JPWO2016186160A1 - Image processing system, image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

Provided is an image processing system capable of realizing both notification and recording for the same operation according to use. An image processing system (300) for detecting a specific motion of a person to be monitored uses an acquisition unit (52) for acquiring motion data indicating the motion of the person and the motion data to determine the degree of motion of the person. A calculation unit (112) for calculating an evaluation value to be shown, a recording unit (120) for recording operation data when the evaluation value satisfies the first determination condition, and an evaluation value of the first determination condition Includes a notification unit (122) for notifying that a specific operation has been detected when different second determination conditions are satisfied.

Description

  The present disclosure relates to an image processing technique, and more particularly, to an image processing technique capable of determining a person's action.

  There is an image processing technique for discriminating a person's movement from an image. This image processing technique is applied to, for example, an image processing apparatus that watches the operation of a cared person who needs care for an elderly person or the like.

  Regarding such an image processing apparatus, Japanese Patent Laying-Open No. 2012-71004 (Patent Document 1) discloses a safety nursing system capable of notifying a nurse corresponding to the condition of each patient. Japanese Laid-Open Patent Publication No. 2001-307257 (Patent Document 2) knows the degree of seriousness, urgency, danger, etc. when a caregiver is notified of an abnormality of a care recipient or a situation requiring care. A care support system that can be used is disclosed.

JP 2012-71004 A JP 2001-307257 A

  By the way, when a dangerous motion such as a fall occurs in the cared person, not only can the carer be notified of this, but also an image (video) when the cared person has the dangerous motion can be recorded. It is desired. At this time, the sensitivity (hereinafter, also referred to as “recording level”) at which the dangerous action is recorded is required to be different from the sensitivity (hereinafter, also referred to as “notification level”) at which the dangerous action is notified. It has been. For example, if an operation that has not been made to fall, such as a cared person hit, is reported, the burden on the caregiver increases. In order to reduce such a burden, it is important to suppress notification of operations that are not so dangerous. On the other hand, in order to identify a dangerous place that causes a fall or the like, it is important to record without omission even an operation that does not cause the fall, such as a care receiver hit.

  The safety nursing system disclosed in Patent Document 1 sets correspondence relationships between a plurality of types of predetermined actions and a plurality of alarm levels according to the condition of each patient. When detecting the predetermined motion of the patient, the safety nursing system notifies that the patient has performed the predetermined motion at an alarm level associated with the detected predetermined motion. That is, the safety nursing system sets an alarm level for each type of operation, and does not set different alarm levels for the same operation. Further, the safety nursing system does not record a predetermined action of the patient.

  The care support system disclosed in Patent Literature 2 includes a step of setting a detection level of a predetermined care recipient's predetermined behavior, and a step of detecting the predetermined behavior of the care receiver based on the detection level and notifying the caregiver Prepare. The caregiver can select a notification method according to the characteristics of the care recipient. The care support system does not record a predetermined operation of the care recipient.

  The present disclosure has been made in order to solve the above-described problems, and an object in one aspect thereof is to provide an image processing system capable of realizing both notification and recording of the same operation according to usage. That is. An object in another aspect is to provide an image processing apparatus capable of realizing both notification and recording for the same operation in accordance with the application. Still another object of the present invention is to provide an image processing method capable of realizing both notification and recording for the same operation in accordance with the application. Still another object of the present invention is to provide an image processing program capable of realizing both notification and recording for the same operation according to usage.

  According to one aspect, an image processing system for detecting a specific motion in a person to be monitored uses an acquisition unit for acquiring motion data indicating the motion of the person, and indicates the degree of motion of the person using the motion data A calculation unit for calculating an evaluation value, a recording unit for recording operation data when the evaluation value satisfies the first determination condition, and a second determination condition whose evaluation value is different from the first determination condition. And a notifying unit for notifying that a specific operation has been detected.

  Preferably, the recording unit determines whether or not the first determination condition is satisfied by comparing the evaluation value with a first threshold value. The notification unit determines whether or not the second determination condition is satisfied by comparing the evaluation value with a second threshold value different from the first threshold value.

  Preferably, the number of determination conditions for the evaluation value in the first determination condition is different from the number of determination conditions for the evaluation value in the second determination condition.

  Preferably, the number of determination conditions for the evaluation value in the second determination condition is larger than the number of determination conditions for the evaluation value in the first determination condition.

Preferably, the second determination condition is a condition that is less likely to be satisfied than the first determination condition.
According to another aspect, an image processing system for detecting a specific motion in a person to be monitored uses an acquisition unit for acquiring motion data indicating the motion of the person and the motion data to determine the degree of motion of the person. A first calculation unit for calculating a first evaluation value to be shown, a second calculation unit for calculating a second evaluation value indicating the degree of movement of the person using the first evaluation value, and a first evaluation value And when one of the second evaluation values satisfies the first determination condition, a recording unit for recording operation data, and when the other of the first evaluation value and the second evaluation value satisfies the second determination condition A notification unit for notifying that a specific operation has been detected.

  According to yet another aspect, an image processing apparatus for detecting a specific motion in a person to be monitored uses an acquisition unit for acquiring motion data indicating the motion of the person, and the degree of motion of the person using the motion data A calculation unit for calculating an evaluation value indicating the value, a recording unit for recording operation data when the evaluation value satisfies the first determination condition, and a second determination in which the evaluation value is different from the first determination condition And a notification unit for notifying that a specific operation has been detected when the condition is satisfied.

  According to still another aspect, an image processing method for detecting a specific motion in a person to be monitored includes a step of acquiring motion data indicating the motion of the person, and an evaluation indicating the degree of motion of the person using the motion data A step of calculating a value, a step of recording operation data when the evaluation value satisfies the first determination condition, and a case where the evaluation value satisfies a second determination condition different from the first determination condition. And notifying that the operation has been detected.

  According to yet another aspect, an image processing program for detecting a specific motion in a person to be monitored obtains motion data indicating the motion of the person in a computer, and the degree of motion of the person using the motion data When the evaluation value satisfies the first determination condition, the step of recording the operation data when the evaluation value satisfies the first determination condition, and when the evaluation value satisfies the second determination condition different from the first determination condition And a step of notifying that a specific operation has been detected.

  In one aspect, notification and recording for the same operation can be realized according to the application.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of a structure of the image processing system according to 1st Embodiment. It is a block diagram which shows an example of a function structure of the image processing system according to 1st Embodiment. It is a figure which shows the image showing a mode from a care receiver standing up until it falls. It is a flowchart showing a part of process which the image processing system according to 1st Embodiment performs. It is a block diagram which shows the main hardware constitutions of the image processing system according to 1st Embodiment. It is a block diagram which shows an example of a function structure of the image processing system according to 2nd Embodiment. It is a flowchart showing a part of process which the image processing system according to 2nd Embodiment performs.

  Embodiments according to the present invention will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. Each embodiment and each modified example described below may be selectively combined as appropriate.

<First Embodiment>
[Configuration of Image Processing System 300]
The configuration of the image processing system 300 will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of an image processing system 300.

  The image processing system 300 is used, for example, to watch over the care receiver 10 who is a person to be monitored. As shown in FIG. 1, the image processing system 300 includes a camera 50, an image processing device 100, and a mobile terminal 200. The camera 50 and the image processing apparatus 100 are connected to each other via a network. The image processing apparatus 100 and the mobile terminal 200 are connected to each other via a network.

  The camera 50 is installed in, for example, a medical facility, a nursing facility, a home, or the like. FIG. 1 shows a state where the camera 50 is photographing the care receiver 10 and the bed 20 from the ceiling. The camera 50 may be attached to the ceiling or to the side wall. The image processing apparatus 100 determines the operation of the care receiver 10 based on a time-series image (video) obtained from the camera 50. As an example, the operation of the care recipient 10 that can be discriminated by the image processing apparatus 100 includes at least one of a fall, a fall from the bed 20, a wake-up, and a floor entry.

  When detecting an operation to be notified, the image processing apparatus 100 records an image (video) showing the operation and transmits information indicating the type of the operation to the mobile terminal 200. In the recording of the operation, it is important to record even the crawl operation without omission so that the dangerous place where the fall or the like is likely to occur can be identified. On the other hand, in the operation notification, if the notification is frequently performed, the burden on the caregiver increases. For example, when the cared person has not fallen, such as when the cared person has nodded, the caregiver is informed of this, increasing the burden on the carer. Therefore, it is preferable that the sensitivity (that is, the recording level) at which the care receiver's target motion is recorded is different from the sensitivity at which the care receiver's target motion is notified (that is, the notification level).

  Therefore, the recording level and the notification level are individually set in image processing system 300 according to the present embodiment. Thereby, the image processing system 300 can implement recording and notification of the same operation according to the application.

  When the mobile terminal 200 receives information indicating the type of operation to be notified from the image processing apparatus 100, the mobile terminal 200 notifies the caregiver 11 of the information. As an example of the notification method, the mobile terminal 200 displays information indicating the type of operation as a message or outputs it as sound. Thereby, when the image processing apparatus 100 detects an operation such as the fall of the care receiver 10, the care worker 11 can quickly rush to the care receiver 10.

  1 shows an example in which the image processing system 300 includes one camera 50, the image processing system 300 may include a plurality of cameras 50. 1 shows an example in which the image processing system 300 includes one image processing apparatus 100, the image processing system 300 may include a plurality of image processing apparatuses 100. Further, although the camera 50 and the image processing apparatus 100 are configured as separate apparatuses in FIG. 1, the camera 50 and the image processing apparatus 100 may be configured integrally. Further, FIG. 1 shows an example in which the image processing system 300 includes a plurality of mobile terminals 200, but the image processing system 300 may be configured by one mobile terminal 200.

[Processing of Image Processing System 300]
Processing executed by the image processing system 300 will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of a functional configuration of the image processing system 300. As illustrated in FIG. 2, the image processing system 300 includes an acquisition unit 52, a detection unit 110, a recording unit 120, and a notification unit 122. Detection unit 110 includes a calculation unit 112, a recording determination unit 114, and a notification determination unit 116. Below, these structures are demonstrated in order.

(Acquisition part 52)
The acquisition unit 52 acquires operation data indicating the operation of the care recipient. The acquisition unit 52 is, for example, a network camera. The acquisition unit 52 acquires an image obtained by photographing a person to be monitored as operation data. The image may be a still image or a moving image. The acquisition unit 52 holds images for the past several seconds and sequentially outputs them to the detection unit 110.

  The acquisition unit 52 is not limited to a camera as long as the acquisition unit 52 can acquire operation data indicating temporal changes in the operation amount of the care recipient. For example, the acquisition unit 52 may be a pressure-sensitive mat placed under a bed, a distance image sensor in a TOF (Time Of Flight) format, an acceleration sensor, a microwave sensor, or the like. The pressure sensitive mat outputs a pressure value as operation data. The distance image sensor outputs a distance image as operation data. The acceleration sensor outputs acceleration as operation data. The microwave sensor outputs the distance to the care recipient as operation data. The acquisition unit 52 does not need to be configured by one sensor, and may be configured by a plurality of sensors.

(Calculation unit 112)
The calculation unit 112 calculates an evaluation value indicating the degree of movement of the care recipient using the motion data. The evaluation value may be indicated by a one-dimensional value or a multi-dimensional value. As an example, the calculation unit 112 calculates an optical flow between time-series images, and uses the maximum value of the optical flow or the size of the optical flow occurrence range as an evaluation value. The optical flow is a vector representing the movement of the care recipient. Details of the evaluation value calculation method using the optical flow will be described later. The calculation unit 112 outputs the calculated evaluation value to the recording determination unit 114 and the notification determination unit 116.

  Note that the evaluation value may be calculated by other methods. In a certain situation, the calculation unit 112 generates a difference image by subtracting time-series images. The calculation unit 112 calculates the size of a range having a pixel value greater than or equal to a predetermined value in the difference image, and uses the size of the range as an evaluation value. In another aspect, the calculation unit 112 specifies positions of human parts such as the head, arms, and legs in the image, and then estimates the care receiver's posture from the positions of the parts. The calculation unit 112 calculates the similarity between the estimated posture and the predetermined action of the care recipient as an evaluation value.

(Recording determination unit 114)
The recording determination unit 114 determines whether to record the image used in the operation determination process based on the evaluation value calculated by the calculation unit 112. Details of the recording determination unit 114 will be described later. When the recording determination unit 114 determines to record an image, the recording determination unit 114 outputs a command for recording the image to the recording unit 120.

(Notification determination unit 116)
The notification determination unit 116 determines whether a predetermined operation (hereinafter also referred to as “target operation”) of the care recipient is detected based on the evaluation value calculated by the calculation unit 112. Details of the notification determination unit 116 will be described later. When the notification determination unit 116 determines that the target motion has been detected, the notification determination unit 116 outputs a command for notifying that the target motion has been detected to the notification unit 122.

(Recording unit 120)
When the recording unit 120 receives a recording command from the recording determination unit 114, the recording unit 120 acquires an image of the past certain time in which the operation to be recorded is reflected from the acquisition unit 52, and records the image.

  In one aspect, the recording unit 120 is configured as a server connected to a network camera via a network. The server communicates with the network camera based on the reception of the recording command from the recording determination unit 114, acquires images from a few seconds before the target motion is detected until the target motion is detected, Remember.

  In another aspect, the recording unit 120 is configured as an HDD (Hard Disk Drive) directly connected to the network camera. The HDD stores images from a few seconds before the target motion is detected until the target motion is detected based on the reception of the recording command from the recording determination unit 114. The recording unit 120 transmits an image recorded in the HDD at regular intervals to the server.

  The image recording method is not limited to the above example. The image recording method is arbitrary as long as the caregiver can confirm what kind of motion data is used to detect the target motion.

(Notification part 122)
When receiving the notification command from the notification determination unit 116, the notification unit 122 notifies the caregiver that the target action has been detected. The alerting | reporting part 122 is integrated as an application in a caregiver's portable terminal, for example. The application is for informing the caregiver of the target action, and is installed in advance on the caregiver's mobile terminal. When the application receives a notification command from the notification determination unit 116, the application notifies that the target action has been detected by means set in itself (for example, notification sound, vibrator, etc.), and the screen of the mobile terminal The detection result is displayed above.

  Note that the notification method is not limited to the above-described method. The notification method is arbitrary as long as it can recognize that the target action of the caregiver is detected. For example, the notification unit 122 may be configured as a speaker that emits an alarm sound, or may be configured as a lamp that notifies the operation of the care receiver by lighting.

[Concrete example]
With reference to FIG. 3, the calculation unit 112, the recording determination unit 114, and the notification determination unit 116 illustrated in FIG. 2 will be described in more detail. FIG. 3 is a diagram illustrating images 32A to 32C representing a state from when the cared person 10 stands up to when it falls. More specifically, the image 32 </ b> A shows the care recipient 10 sitting on the bed 20. The image 32B shows a situation where the care receiver 10 is about to stand up from the bed 20. The image 32C shows a situation where the care receiver 10 is falling.

  The calculation unit 112 calculates an optical flow from the temporally continuous images 32A and 32B. The optical flow represents a motion of an object in an image as a vector. The image 32B shows an optical flow 33AB calculated from temporally continuous images 32A and 32B. The image 32C shows an optical flow 33BC calculated from temporally continuous images 32B and 32C.

  When the cared person falls, a faster movement occurs than in normal times (such as during walking). Therefore, the optical flow 33BC is larger than the optical flow 33AB. Moreover, since the cared person falls from the state of being upright to the floor at the time of the fall, the optical flow is generated in a wider range than at the normal time. Therefore, the generation range of the optical flow 33BC becomes larger than the generation range of the optical flow 33AB. The recording determination unit 114 and the notification determination unit 116 detect the fall of the care recipient by paying attention to the difference in the optical flow between the fall of the care recipient 10 and the normal time.

  In the following description, the motion of the care recipient 10 is described as an example of the motion to be detected, but the motion to be detected is not limited to the motion. In addition, the motions to be detected include motions that fall from the bed 20 during sleep, motions that are rampant on the bed 20 for reasons such as pain, and other motions that cause danger to the care recipient 10.

(Specific Example 1-1)
The calculation unit 112 calculates an optical flow from the time-series image, and calculates an evaluation value indicating the possibility that the care receiver has fallen based on the optical flow. As an example, the evaluation value includes the maximum value of the optical flow and the size of the generation range of the optical flow. The maximum value of the optical flow represents the magnitude of movement of the care recipient. The size of the optical flow generation range represents a range in which movement is generated in the cared person in the image.

  Hereinafter, the maximum value of the optical flow in the i-th image (frame) of the moving image is also referred to as the maximum value Vi, and the size of the optical flow generation range in the i-th image (frame) of the moving image is defined as the generation range Si. Also called.

  When the evaluation value satisfies a first determination condition (hereinafter also referred to as “recording condition”), the recording determination unit 114 determines that the care receiver has fallen and records the operation data. As an example, the recording determination unit 114 determines whether or not the recording condition is satisfied by comparing the evaluation value with a threshold value Tha as shown in the following formula (1). More specifically, the recording condition is satisfied when the maximum value Vi becomes larger than the threshold value Tha.

Vi> Tha (1)
When the evaluation value satisfies a second determination condition (hereinafter, also referred to as “notification condition”), the notification determination unit 116 determines that the cared person has fallen and detects the fall of the cared person. Inform you. As an example, the notification determination unit 116 determines whether or not the notification condition is satisfied by comparing the optical flow maximum value Vi with a threshold Thb, as shown in the following equation (2). More specifically, the notification condition is satisfied when the maximum value Vi becomes larger than the threshold value Thb.

Vi> Thb (2)
(Specific example 1-2)
In Specific Example 1-1, the example in which the maximum value Vi of the optical flow is used as an index for detecting the fall of the care recipient is described. On the other hand, in Example 1-2, an example in which the optical flow generation range Si is used as the index will be described.

  The recording determination unit 114 determines whether the recording condition is satisfied by comparing the generation range Si with the threshold Thp. In the specific example 1-2, as shown in the following formula (3), the recording condition is satisfied when the generation range Si becomes larger than the threshold Thp.

Si> Thp (3)
The notification determination unit 116 determines whether the notification condition is satisfied by comparing the generation range Si with a threshold Thq. In the specific example 1-2, as shown in the following formula (4), the notification condition is satisfied when the generation range Si is larger than the threshold value Thq.

Si> Thq (4)
(Effects of specific examples 1-1 and 1-2)
In this way, by setting different determination conditions for the same evaluation value, the recording level at which the care receiver's target action is recorded is different from the notification level at which the care receiver's target action is notified. It becomes possible to make it.

  Preferably, the notification condition is a condition that is less likely to be satisfied than the recording condition. As an example, the image processing system 300 sets the threshold value Thb (see Expression (2)) to be larger than the threshold value Tha (see Expression (1)), thereby making the notification condition stricter than the recording condition. Alternatively, the image processing system 300 sets the threshold value Thq (see formula (4)) to be larger than the threshold value Thp (see formula (3)), thereby making the notification condition stricter than the recording condition.

  As a result, the image processing system 300 can record an operation that has not been overturned, such as a caregiver crawls, and can notify only the operation that needs to be rushed to the caregiver. As a result, the image processing system 300 can record without leaking information for identifying a dangerous place where a fall or the like is likely to occur, and can further reduce the burden on the caregiver.

(Specific example 2-1)
In specific examples 1-1 and 1-2, the image processing system 300 compares the same evaluation value with different threshold values, thereby changing the recording level and the notification level. On the other hand, in the specific example 2-1, the image processing system 300 varies the recording level and the notification level by varying the number of determination conditions. That is, in the specific example 2-1, the number of determination conditions for the evaluation value under the recording condition is different from the number of determination conditions for the evaluation value under the notification condition.

  More specifically, the recording determination unit 114 determines whether the recording condition is satisfied by comparing the optical flow maximum value Vi with a threshold Thv. For example, as shown in the following equation (5), the recording condition is satisfied when the maximum value Vi becomes larger than the threshold value Thv.

Vi> Thv (5)
As shown in the following formulas (6) and (7), the notification determination unit 116 compares the optical flow maximum value Vi with the threshold Thv and compares the optical flow occurrence range Si with the threshold Ths. Then, it is determined whether or not the notification condition is satisfied. More specifically, as shown in the following formulas (6) and (7), when the maximum value Vi is larger than the threshold value Thv and the generation range Si is larger than the threshold value Ths, notification is performed. The condition is met.

Vi> Thv (6)
Si> Ths (7)
As described above, the image processing system 300 makes the number of determination conditions different so that the recording level at which the target action of the care recipient is recorded and the notification level at which the target action of the care receiver is notified are different. Is possible.

(Specific example 2-2)
The number of determination conditions in the recording condition and the number of determination conditions in the notification condition are not limited to the example of specific example 2-1. These determination condition numbers are arbitrary. Hereinafter, as specific example 2-2, another example in which the number of determination conditions is different will be described.

  The recording determination unit 114 determines whether the recording condition is satisfied by comparing the maximum value Vi of the optical flow with the threshold Thv. More specifically, as shown in the following equation (8), the recording condition is satisfied when the maximum value Vi becomes larger than the threshold value Thv.

Vi> Thv (8)
The notification determining unit 116 determines whether or not the notification condition is satisfied by comparing the maximum value Vi of the optical flow in the past N frames with the threshold Thv. As an example, as shown in the following formula (9), the notification condition is satisfied when all of the maximum values Vi in the past N frames exceed the threshold Thv.

(Effects of specific examples 2-1 and 2-2)
In this way, the image processing system 300 makes the number of determination conditions in the recording condition different from the number of determination conditions in the notification condition, so that the recording level at which the care receiver's target action is recorded and the care receiver's target action are recorded. It is possible to make the notification level different from the notification level.

  Preferably, the determination condition number for the evaluation value in the notification condition is set to be larger than the determination condition number for the evaluation value in the recording condition. As a result, the notification condition is less likely to be satisfied than the recording condition. As a result, the image processing system 300 can record an operation that has not been overturned, such as a caregiver who has struck, and can notify only the operation that needs to be rushed to the caregiver.

[Control Structure of Image Processing System 300]
The control structure of the image processing system 300 will be described with reference to FIG. FIG. 4 is a flowchart showing a part of processing executed by the image processing system 300. In each process shown in FIG. 4, a CPU (Central Processing Unit) 102 (see FIG. 5) of the image processing apparatus 100 and a CPU 202 (see FIG. 5) of the portable terminal 200 execute an image processing program according to the present embodiment. Is realized. Hereinafter, an example in which the CPU 102 executes each process illustrated in FIG. 4 will be described, but each process may be executed by the CPU 202. Or CPU102,202 may perform each process in cooperation. In other aspects, some or all of the processing may be performed by circuit elements or other hardware.

  In step S10, the CPU 102 acquires operation data from the acquisition unit 52 (see FIG. 2). As an example, the acquisition unit 52 is a camera 50 (see FIG. 1), and the operation data is an image.

  In step S <b> 12, the CPU 102 calculates an evaluation value indicating the degree of movement of the person using the operation data as the calculation unit 112 (see FIG. 2). As an example, the evaluation value includes a maximum value of an optical flow calculated from a time-series image, a range where the optical flow is generated, and the like.

  In step S20, the CPU 102 determines whether or not the calculated evaluation value satisfies the recording condition as the above-described recording determination unit 114 (see FIG. 2). For example, the CPU 102 determines that the recording condition is satisfied when the evaluation value exceeds a threshold value. When CPU 102 determines that the evaluation value satisfies the recording condition (YES in step S20), CPU 102 switches control to step S22. If not (NO in step S20), CPU 102 switches control to step S30.

  In step S22, the CPU 102 records operation data as the above-described recording unit 120 (see FIG. 2). The recorded operation data is, for example, an image from a few seconds before the target motion is detected until the target motion is detected.

  In step S30, the CPU 102 determines whether or not the calculated evaluation value satisfies the notification condition as the notification determination unit 116 (see FIG. 2). For example, the CPU 102 determines that the notification condition is satisfied when the evaluation value exceeds a threshold value. When CPU 102 determines that the evaluation value satisfies the notification condition (YES in step S30), CPU 102 switches control to step S32. If not (NO in step S30), CPU 102 ends the image processing according to the present embodiment.

  In step S32, the CPU 102 notifies the caregiver that the target operation of the care receiver has been detected as the above-described notification unit 122 (see FIG. 2).

[Hardware Configuration of Image Processing System 300]
An example of the hardware configuration of the image processing system 300 will be described with reference to FIG. FIG. 5 is a block diagram illustrating a main hardware configuration of the image processing system 300. As shown in FIG. 5, the image processing system 300 includes an image processing apparatus 100 and a mobile terminal 200. The image processing apparatus 100 and the mobile terminal 200 are connected to each other via a network. Hereinafter, the hardware configuration of the image processing apparatus 100 and the hardware configuration of the mobile terminal 200 will be described in order.

(Hardware configuration of image processing apparatus 100)
As shown in FIG. 5, the image processing apparatus 100 includes a ROM (Read Only Memory) 101, a CPU 102, a RAM (Random Access Memory) 103, a network I / F (interface) 104, and a camera I / F 105. And the storage device 106.

  ROM 101 stores an operating system, an image processing program 107 according to the present embodiment, and the like. The CPU 102 controls the operation of the image processing apparatus 100 by executing various programs such as an operating system and the image processing program 107. The RAM 103 functions as a working memory and temporarily stores various data necessary for executing the image processing program 107.

  A communication device such as an antenna or a NIC (Network Interface Card) is connected to the network I / F 104. The image processing apparatus 100 transmits / receives data to / from other communication terminals via the communication device. Other communication terminals include, for example, the mobile terminal 200 and other terminals. The image processing apparatus 100 may be configured to download the image processing program 107 via a network.

  The camera I / F 105 is an interface for connecting the camera 50 to the image processing apparatus 100 in a wired or wireless manner. The image processing apparatus 100 acquires an image from the camera 50 via the camera I / F 105. The camera 50 is, for example, a network camera or other imaging device capable of photographing a subject. The camera 50 may be configured integrally with the image processing apparatus 100, or may be configured separately from the image processing apparatus 100 as shown in FIG.

  The storage device 106 is a storage medium such as a hard disk or an external storage device. As an example, the storage device 106 stores an image processing program 107 for realizing various types of processing according to the present embodiment, an image in which an operation of a detection target of a care receiver is shown, and the like.

  Note that the image processing program 107 may be provided by being incorporated in a part of an arbitrary program, not as a single program. In this case, processing according to the present embodiment is realized in cooperation with an arbitrary program. Even such a program that does not include some modules does not depart from the spirit of the image processing apparatus 100 according to the present embodiment. Furthermore, part or all of the functions provided by the image processing program 107 according to the present embodiment may be realized by dedicated hardware. Furthermore, the image processing apparatus 100 may be configured in the form of a so-called cloud service in which at least one server realizes processing according to the present embodiment.

(Hardware configuration of portable terminal 200)
With reference to FIG. 5, the hardware configuration of the mobile terminal 200 will be described. The portable terminal 200 is, for example, a smartphone, a tablet terminal, or other terminals that can be carried by a care recipient. As illustrated in FIG. 5, the mobile terminal 200 includes a ROM 201, a CPU 202, a RAM 203, a network I / F 204, a display 205, and a storage device 206.

  The ROM 201 stores an operating system, a control program executed by the mobile terminal 200, and the like. The CPU 202 controls the operation of the mobile terminal 200 by executing various programs such as an operating system and a control program for the mobile terminal 200. The RAM 203 functions as a working memory and temporarily stores various data necessary for program execution.

  A communication device such as an antenna or a NIC (Network Interface Card) is connected to the network I / F 204. The portable terminal 200 transmits / receives data to / from other communication terminals via the communication device. Other communication terminals include, for example, the image processing apparatus 100 and other terminals.

  The display 205 is, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or other display devices. The display 205 may be configured as a touch panel. The display 205 displays that the operation has been detected when the operation to be notified of the care recipient is detected. As a notification method, the display 205 displays, for example, the type of operation of the care recipient as a message, or displays an image prompting a warning.

  The storage device 206 includes a storage medium such as eMMC (Embedded MultiMediaCard). The eMMC includes a NAND flash memory and a control circuit. Storage device 206 stores programs and the like for realizing various processes according to the present embodiment.

[Brief Summary]
As described above, in image processing system 300 according to the present embodiment, different determination conditions (that is, recording conditions and notification conditions) are set for the same evaluation value indicating the movement of the care recipient. Thereby, the image processing system 300 can make the recording level in which the care receiver's target motion is recorded different from the notification level in which the care receiver's target motion is notified. As a result, the care recipient can change the recording level and the notification level according to the application.

  In addition, since the same evaluation value is used for the recording determination for determining whether or not to record the target action and the notification determination for determining whether or not to notify the target action, the recording determination process and the notification determination process The implementation of can be simplified. Thereby, the cost of the image processing system 300 can be reduced.

  Furthermore, the notification condition is a condition that is more difficult to satisfy than the recording condition. Therefore, the image processing system 300 can record an operation that has not been overturned, such as a caregiver who has struck, and can notify only the operation that needs to be rushed to the caregiver. As a result, the image processing system 300 can record without leaking information for identifying a dangerous place where a fall or the like is likely to occur, and can further reduce the burden on the caregiver.

<Second Embodiment>
[Overview]
The image processing system 300 according to the first embodiment sets different determination conditions (that is, a recording condition and a notification condition) for the same evaluation value indicating the degree of movement of the care recipient, so that the target action can be performed. The recording level and the notification level of the target action were different. In contrast, the image processing system 300 according to the second embodiment calculates a new evaluation value as the evaluation value for recording or the evaluation value for notification from the evaluation value indicating the degree of movement of the care recipient. The image processing system 300 varies the recording level and the notification level by using the evaluation value as it is and the new evaluation value as a reference.

[Function configuration]
With reference to FIG. 6, the process of the image processing system 300 according to the second embodiment will be described. FIG. 6 is a block diagram showing an example of a functional configuration of the image processing system 300 according to the second embodiment. As illustrated in FIG. 6, the image processing system 300 includes an acquisition unit 52, a detection unit 110, a recording unit 120, and a notification unit 122. The detection unit 110 includes a first calculation unit 112A, a second calculation unit 112B, a recording determination unit 114A, and a notification determination unit 116A. Since acquisition unit 52, recording unit 120, and notification unit 122 are as described in FIG. 2, description of their functional configurations will not be repeated.

  The first calculation unit 112A acquires motion data from the acquisition unit 52, and uses the motion data to indicate a first evaluation value (hereinafter, also referred to as “recording evaluation value”) indicating the degree of movement of the care recipient. Is calculated. Details of the evaluation value for recording will be described later. The first calculation unit 112A outputs the evaluation value for recording to the second calculation unit 112B and the recording determination unit 114A.

  The second calculation unit 112B calculates a second evaluation value (hereinafter, also referred to as “notification evaluation value”) indicating the degree of movement of the care recipient using the recording evaluation value. Details of the evaluation value for notification will be described later. The second calculation unit 112B outputs the evaluation value for notification to the notification determination unit 116A.

  The recording determination unit 114A determines whether or not to record the image used in the operation determination process, based on the evaluation value for recording. Details of the recording determination unit 114A will be described later. If the recording determination unit 114A determines to record an image, the recording determination unit 114A outputs a command for recording the image to the recording unit 120.

  The notification determination unit 116A determines whether a specific operation of the care recipient is detected based on the evaluation value for notification. Details of the notification determination unit 116A will be described later. When the notification determination unit 116A determines that a specific operation has been detected, the notification determination unit 116A outputs a command for notifying that the specific operation has been detected to the notification unit 122.

  In FIG. 6, the example in which the notification evaluation value is calculated from the recording evaluation value has been described, but the recording evaluation value may be calculated from the notification evaluation value. That is, a new evaluation value may be calculated from at least one of the recording evaluation value and the notification evaluation value.

[Concrete example]
Hereinafter, the first calculation unit 112A, the second calculation unit 112B, the recording determination unit 114A, and the notification determination unit 116A illustrated in FIG. 6 will be described in more detail.

(Specific example 3-1)
In Specific Example 3-1, the image processing system 300 calculates the recording evaluation value and the notification evaluation value, and uses the recording evaluation value and the notification evaluation value as a reference, so that the recording level for the operation to be notified is determined. And different notification levels.

  More specifically, the first calculation unit 112A calculates a recording evaluation value indicating the degree of movement of the care recipient using the motion data. As an example, the first calculation unit 112A sets the optical flow occurrence range Si calculated from the time-series images as the recording evaluation value.

  The second calculation unit 112B calculates a notification evaluation value using the calculated recording evaluation value. The second calculation unit 112B calculates the average value of the optical flow occurrence range Si in the past N frames as the notification evaluation value.

  The recording determination unit 114A determines whether or not the recording evaluation value satisfies the recording condition. When the recording condition is satisfied, the recording determination unit 114A records the operation data used for the operation determination of the care receiver. As an example, the recording determination unit 114A determines whether the recording condition is satisfied by comparing the generation range Si with a threshold Ths. For example, as shown in the following formula (10), the recording condition is satisfied when the generation range Si becomes larger than the threshold value Ths.

Si> Ths (10)
The notification determination unit 116A calculates the notification evaluation value using the generation range Si that is the recording evaluation value. As an example, as shown in the following formula (11), the notification determination unit 116A calculates the average value of the generation range Si in the past N frames as the notification evaluation value. The notification determination unit 116A determines whether or not the notification condition is satisfied by comparing the notification evaluation value with the threshold Ths. For example, as shown in the following formula (12), when the notification evaluation value exceeds the threshold Ths, the notification condition is satisfied.

(Specific Example 3-2)
Another specific example in which the recording level and the notification level for the operation to be notified are made different based on the recording evaluation value and the notification evaluation value will be described.

  The recording determination unit 114A determines whether the recording condition is satisfied by comparing the optical flow generation range Si with the threshold Ths. More specifically, as shown in the following formula (13), the recording condition is satisfied when the generation range Si is larger than the threshold value Ths.

Si> Ths (13)
The notification determination unit 116A calculates the notification evaluation value P by multiplying the generation range Si, which is the recording evaluation value, by the optical flow maximum value Vi, as shown in the following equation (14). . Thereafter, the notification determining unit 116A determines whether or not the notification condition is satisfied by comparing the notification evaluation value P with the threshold Thp. As an example, the notification condition is satisfied when the notification evaluation value P becomes larger than the threshold Thp as shown in the following equation (15).

P = Si · Vi (14)
P> Thp (15)
[Control Structure of Image Processing System 300]
With reference to FIG. 7, a control structure of image processing system 300 according to the second embodiment will be described. FIG. 7 is a flowchart representing a part of processing executed by image processing system 300 according to the second embodiment. Each process illustrated in FIG. 7 is realized by the CPU 102 (see FIG. 5) of the image processing apparatus 100 and the CPU 202 (see FIG. 5) of the portable terminal 200 executing the image processing program according to the present embodiment. Hereinafter, an example in which the CPU 102 executes each process illustrated in FIG. 7 will be described, but each process may be executed by the CPU 202. Or CPU102,202 may perform each process in cooperation. In other aspects, some or all of the processing may be performed by circuit elements or other hardware.

  In step S10, the CPU 102 acquires operation data from the acquisition unit 52 (see FIG. 6). As an example, the acquisition unit 52 is a camera 50 (see FIG. 1), and the operation data is an image.

  In step S <b> 12 </ b> A, the CPU 102 uses the operation data as the first calculation unit 112 </ b> A (see FIG. 6) to calculate a recording evaluation value indicating the degree of person movement. As an example, the recording evaluation value includes a maximum value of the optical flow, a range where the optical flow is generated, and the like.

  In step S12B, the CPU 102 uses the recording evaluation value as the second calculation unit 112B (see FIG. 6) described above to calculate a notification evaluation value indicating the degree of person movement. As an example, the evaluation value for notification includes a result of multiplying the maximum value of the optical flow by the size of the optical flow generation range.

  In step S20A, the CPU 102 determines whether the recording evaluation value satisfies the recording condition as the above-described recording determination unit 114A (see FIG. 6). For example, the CPU 102 determines that the recording condition is satisfied when the recording evaluation value exceeds a predetermined threshold. When CPU 102 determines that the recording evaluation value satisfies the recording condition (YES in step S20A), CPU 102 switches control to step S22. If not (NO in step S20A), CPU 102 switches control to step S30A.

  In step S22, the CPU 102 records operation data as the above-described recording unit 120 (see FIG. 6). The recorded operation data is, for example, an image from a few seconds before the target motion is detected until the target motion is detected.

  In step S30A, the CPU 102 determines whether or not the evaluation value for notification satisfies the notification condition as the notification determination unit 116A (see FIG. 6). For example, the CPU 102 determines that the notification condition is satisfied when the notification evaluation value exceeds a predetermined threshold. When CPU 102 determines that the evaluation value for notification satisfies the notification condition (YES in step S30A), CPU 102 switches control to step S32. If not (NO in step S30A), CPU 102 ends the image processing according to the present embodiment.

  In step S32, the CPU 102 notifies the caregiver that the target motion has been detected as the above-described notification unit 122 (see FIG. 6).

[Brief Summary]
As described above, the image processing system 300 according to the present embodiment is based on the evaluation value indicating the degree of movement of the care recipient and the new evaluation value calculated from the evaluation value. The recording level and notification level for the caregiver's target motion are varied. As a result, the image processing system 300 can change the recording level at which the care receiver's target action is recorded and the notification level at which the care receiver's target action is notified depending on the application.

  Preferably, the recording evaluation value and the notification evaluation value are determined under the same determination condition. That is, the recording condition and the notification condition are the same. For example, as shown in the above formulas (10) and (12), the image processing system 300 compares the recording evaluation value and the notification evaluation value with a common threshold value Ths, thereby obtaining the recording evaluation value and the notification. The evaluation value is determined under the same determination condition. Thereby, the image processing system 300 can simplify the setting of the recording condition and the notification condition.

<Third Embodiment>
Although the example in which the acquisition unit 52 (see FIG. 2) is configured as a network camera has been described above, the acquisition unit 52 may be configured with a plurality of sensors. For example, the acquisition unit 52 includes a network camera and a pressure sensor. The image processing system 300 according to the third embodiment includes an evaluation value calculated based on a moving image output by a network camera and an evaluation value calculated based on a pressure distribution output by a pressure sensor. In combination, recording determination and notification determination are performed. Below, the specific example at the time of using these evaluation values together is demonstrated.

[Concrete example]
(Specific Example 4-1)
The first calculation unit 112A (see FIG. 6) calculates an evaluation value indicating the degree of movement of the care receiver from the moving image output by the network camera. The recording determination unit 114A (see FIG. 6) determines that the target action is occurring in the care recipient when the evaluation value satisfies the recording condition. In this case, the recording unit 120 (see FIG. 6) stores the moving image used for the operation determination process.

  The second calculator 112B (see FIG. 6) calculates an evaluation value indicating the degree of movement of the care recipient from the pressure distribution output by the pressure sensor. As an example, the evaluation value includes the maximum value of the pressure in the pressure distribution and the size of a range where a pressure value equal to or greater than a predetermined value occurs. The notification determination unit 116A (see FIG. 6) determines that the target action is occurring in the care receiver when the evaluation value satisfies the notification condition. For example, the notification determination unit 116A determines that the notification condition is satisfied when the evaluation value exceeds a predetermined threshold. In this case, the notification unit 122 (see FIG. 6) notifies the caregiver that the target action of the care receiver has been detected.

(Specific Example 4-2)
The recording determination unit 114A determines that the target action is occurring in the care recipient when one of the evaluation value calculated from the moving image and the evaluation value calculated from the pressure distribution exceeds a predetermined threshold. To do. In this case, the recording unit 120 stores the moving image used for the operation determination process.

  The notification determination unit 116A determines that the target action is occurring in the care recipient when both the evaluation value calculated from the moving image and the evaluation value calculated from the pressure distribution exceed a predetermined threshold. In this case, the notification unit 122 notifies the caregiver that the target action of the care receiver has been detected.

(Specific Example 4-3)
The recording determination unit 114A uses an identifier that has been machine-learned using an evaluation value calculated from a moving image and an evaluation value calculated from a pressure distribution so that undetected movements of the care recipient are reduced. The recording judgment is performed.

  The notification determination unit 116 </ b> A uses an identifier that has been machine-learned so as to reduce false detection of a care receiver's motion using an evaluation value calculated from a moving image and an evaluation value calculated from a pressure distribution. The notification determination is performed.

[Brief Summary]
As described above, image processing system 300 according to the present embodiment performs recording determination and notification determination using not only an image but also a pressure distribution from a pressure sensor. As a result, the image processing system 300 can more accurately determine the operation of the care recipient.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10 care recipients, 11 caregivers, 20 beds, 32A-32C images, 33AB, 33BC optical flow, 50 cameras, 52 acquisition units, 100 image processing devices, 101,201 ROM, 102,202 CPU, 103,203 RAM, 104, 204 Network I / F, 105 Camera I / F, 106, 206 Storage device, 107 Image processing program, 110 Detection unit, 112 Calculation unit, 112A First calculation unit, 112B Second calculation unit, 114, 114A Recording determination Unit, 116, 116A notification determination unit, 120 recording unit, 122 notification unit, 200 portable terminal, 205 display, 300 image processing system.

Claims (9)

An image processing system for detecting a specific action in a person to be monitored,
An acquisition unit for acquiring operation data indicating the operation of the person;
A calculation unit for calculating an evaluation value indicating the degree of movement of the person using the motion data;
A recording unit for recording the operation data when the evaluation value satisfies a first determination condition;
An image processing system comprising: a notifying unit for notifying that the specific operation has been detected when the evaluation value satisfies a second determination condition different from the first determination condition. The recording unit determines whether or not the first determination condition is satisfied by comparing the evaluation value with a first threshold;
The image processing system according to claim 1, wherein the notification unit determines whether or not the second determination condition is satisfied by comparing the evaluation value with a second threshold different from the first threshold.   The image processing system according to claim 1, wherein the number of determination conditions for the evaluation value in the first determination condition is different from the number of determination conditions for the evaluation value in the second determination condition.   The image processing system according to claim 3, wherein the number of determination conditions for the evaluation value in the second determination condition is greater than the number of determination conditions for the evaluation value in the first determination condition.   The image processing system according to claim 1, wherein the second determination condition is a condition that is less likely to be satisfied than the first determination condition. An image processing system for detecting a specific action in a person to be monitored,
An acquisition unit for acquiring operation data indicating the operation of the person;
A first calculation unit for calculating a first evaluation value indicating a degree of movement of the person using the motion data;
A second calculation unit for calculating a second evaluation value indicating a degree of movement of the person using the first evaluation value;
A recording unit for recording the operation data when one of the first evaluation value and the second evaluation value satisfies a first determination condition;
An image processing system comprising: a notifying unit for notifying that the specific operation has been detected when the other of the first evaluation value and the second evaluation value satisfies a second determination condition. An image processing apparatus for detecting a specific action in a person to be monitored,
An acquisition unit for acquiring operation data indicating the operation of the person;
A calculation unit for calculating an evaluation value indicating the degree of movement of the person using the motion data;
A recording unit for recording the operation data when the evaluation value satisfies a first determination condition;
An image processing apparatus comprising: a notifying unit for notifying that the specific operation has been detected when the evaluation value satisfies a second determination condition different from the first determination condition. An image processing method for detecting a specific action in a person to be monitored,
Obtaining motion data indicating the motion of the person;
Calculating an evaluation value indicating the degree of movement of the person using the motion data;
Recording the operation data when the evaluation value satisfies a first determination condition;
An image processing method comprising: notifying that the specific operation has been detected when the evaluation value satisfies a second determination condition different from the first determination condition. An image processing program for detecting a specific action in a person to be monitored,
The image processing program is stored in a computer.
Obtaining motion data indicating the motion of the person;
Calculating an evaluation value indicating the degree of movement of the person using the motion data;
Recording the operation data when the evaluation value satisfies a first determination condition;
An image processing program that, when the evaluation value satisfies a second determination condition different from the first determination condition, executes a step of notifying that the specific operation has been detected.

Images