JP6895090B2 - Detection system and display processing method of detection system - Google Patents

Description

The present invention relates to a technique preferably used in a monitored person monitoring system that monitors a monitored person to be monitored.

Japan (Japan) is an aging society, more specifically, the ratio of the population aged 65 and over to the total population due to the improvement of living standards, sanitary environment and medical standards accompanying the postwar high economic growth. It is a super-aging society with an aging rate of over 21%. In 2005, the total population of about 127.65 million and the elderly population aged 65 and over was about 25.56 million, while in 2020, the total population of about 124.11 million is elderly. The population is estimated to be about 34.56 million. In such an aging society, the nurses and long-term care recipients who need nursing care or nursing care due to illness, injury, or aging are the nurses who need nursing care in a normal society that is not an aging society. The increase is expected more than the above. Japan is also a society with a declining birthrate, for example, with a total fertility rate of 1.43 in 2013. For this reason, elderly care is also taking place in which elderly families (spouse, children, siblings) care for elderly nurses.

Nursing needs are admitted to hospitals and facilities such as welfare facilities for the elderly (short-term admission facilities for the elderly, nursing homes for the elderly, special nursing homes for the elderly, etc. under Japanese law), and receive their nursing and long-term care. In such a facility, a nurse or the like may be injured due to, for example, falling from the bed or falling while walking, or may get out of the bed and wander. It is necessary to respond to such a situation as soon as possible. If such a situation is left unattended, it may develop into a more serious situation. For this reason, at the facility, nurses, caregivers, etc. regularly patrol to confirm their safety and condition.

However, the increase in the number of nurses cannot keep up with the increase in the number of nurses required, and the nursing industry and the long-term care industry are chronically understaffed. Further, as compared with the day shift time zone, the number of nurses, caregivers, etc. decreases in the semi-night shift time zone and the night shift time zone, so that the work load per person increases. Therefore, it is required to reduce the work load. In addition, the situation of elderly care is no exception in the facility, and it is often seen that elderly nurses and the like care for elderly nurses and the like. In general, as the elderly get older, their physical strength weakens, so even if they are healthy, the burden of nursing etc. becomes heavier than that of young nurses, and their movements and judgments become slower.

In order to reduce such labor shortages and the burden on nurses, there is a need for technology that complements nursing work and long-term care work. For this reason, in recent years, research and development of a monitored person monitoring technique for monitoring (monitoring) a monitored person who is a monitoring target to be monitored, such as a nurse requiring nursing, has been researched and developed.

As one of such techniques, for example, there is a nurse call system disclosed in Patent Document 1. The nurse call system disclosed in Patent Document 1 is installed in a bed for a patient to call a nurse, and is installed in a nurse station to answer a call by the nurse call slave unit. A nurse call system having a nurse call master unit, in which a camera that images a patient on the bed from above the bed and a state in which the patient is upright and the patient is away from the bed from the images captured by the camera. The nurse call master unit has a state determining means for determining the occurrence of at least one of them and outputting a caution state generating signal, and the nurse call master unit has a notification means for performing a notification operation in response to the caution state generating signal. Then, this nurse call system receives the mobile terminal carried by the nurse to answer the call from the nurse call slave unit and the caution state generation signal, and transmits the captured image of the camera to the mobile terminal. It has a communication control means to be used.

On the other hand, in terms of safety confirmation, a person living alone living alone is the same as the person requiring long-term care, and is a person to be monitored.

Japanese Unexamined Patent Publication No. 2014-90913

By the way, as in the nurse call system disclosed in Patent Document 1, when the image of the camera is displayed on the terminal device, the observer such as a nurse visually observes the situation of the monitored person such as a nurse required. It is convenient because it can be grasped. However, if the monitored person appears small or the display has a low resolution, it is difficult for the watcher to find the monitored person in the image. In particular, when the terminal device is a mobile terminal device such as a tablet or a smartphone, the display screen is relatively small, so that it is difficult for the observer to find the person to be monitored in the image.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a detection system capable of easily finding a monitored person in an image and a display processing method of the detection system.

The detection system according to the present invention is a detection system that detects a predetermined action of the monitored person based on the image data obtained by photographing the monitored person as the monitored object, and is the image data obtained by photographing the monitored person as the monitored object. The image data of the acquisition unit for acquiring the image data, the detection unit for detecting a predetermined action in the monitored person based on the acquired image data, and the monitored person for which the predetermined action is detected by the detection unit. Based on this, the generation unit that generates position data indicating the position corresponding to the monitored person, the display unit that displays the image data, the monitored person position indicator based on the position data, and the display unit are controlled. The display processing unit has an input area for performing a predetermined operation, and the display processing unit changes the display position so that the input area and the monitored person position indicator do not overlap with each other. To do .

Display processing method according detection system of the present invention, based on the image data obtained by photographing the person being monitored which is monitored, a display processing method in the detection system for detecting a predetermined action of the monitored person, the monitored The step of acquiring the image data of the monitored person in the acquisition unit, the step of detecting the predetermined action in the monitored person in the detection unit based on the acquired image data, and the step of detecting the predetermined action. With respect to the monitored person, the step of generating the position data indicating the position corresponding to the monitored person in the generation unit based on the image data, the image data, and the monitored person position based on the position data. The display unit includes a step of displaying a sign on the display unit and a step of controlling the display unit. The display unit has an input area for performing a predetermined operation, and a step of controlling the display unit includes the input area. The display position is changed so as not to overlap with the monitored person position sign .

The detection system according to the present invention and the display processing method of the detection system can easily find the monitored person in the image.

The above and other objectives, features and advantages of the present invention will become apparent from the following detailed description and accompanying drawings.

It is a figure which shows the structure of the monitored person monitoring system in embodiment. It is a figure which shows the structure of the sensor device in the said monitored person monitoring system. It is a figure for demonstrating the person position data and the person position sign of 1st Embodiment. It is a figure for demonstrating the person position data (person area data) and the person position sign of the 2nd aspect. It is a figure for demonstrating the person position data (person area data) and the person position indicator of the 3rd aspect. It is a figure for demonstrating the person position data (person area data) and the person position sign of 4th aspect. It is a figure for demonstrating the person position data (person area data) and the person position sign of 5th aspect. It is a figure which shows the structure of the mobile terminal apparatus in the said person | monitored person monitoring system. It is a flowchart which shows the operation about the monitoring information of the sensor device in the monitored person monitoring system. It is a flowchart which shows the operation about the monitoring information of the mobile terminal device in the monitored person monitoring system. It is a figure which shows an example of the standby screen displayed on the mobile terminal apparatus in the monitored person monitoring system. It is a figure which shows an example of the monitoring information screen displayed on the mobile terminal apparatus in the monitored person monitoring system. It is a figure which shows an example of the monitoring information screen which expanded the area of a person, which is displayed on the mobile terminal device in the said person monitoring system. It is a figure which shows an example of the monitoring information screen which is displayed on the mobile terminal device in the monitored person monitoring system, and the area of a person and the "corresponding" button overlap. It is a figure which shows an example of the monitoring information screen which displays the area of a person at the center position of the image display area, which is displayed on the mobile terminal device in the said person monitoring system. It is a figure which shows an example of the monitoring information screen which displays the area of a person and the "corresponding" button, which are displayed on the mobile terminal device in the monitored person monitoring system, so that they do not overlap each other.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. It should be noted that the configurations with the same reference numerals in the respective drawings indicate that they are the same configurations, and the description thereof will be omitted as appropriate. In the present specification, when they are generically referred to, they are indicated by reference numerals without subscripts, and when they refer to individual configurations, they are indicated by reference numerals with subscripts.

The monitored person monitoring system in the embodiment is a system that monitors the monitored person (watching target person) Ob, which is the monitoring target (watching target) to be monitored (watched). This monitored person monitoring system includes a sensor device and a terminal device communicably connected to the sensor device, and a predetermined action in a monitored person to be monitored based on an image captured by the imaging unit. Is detected and the detection result is notified to the terminal device. The sensor device in such a monitored person monitoring system has a first communication unit that communicates, an image pickup unit that performs imaging, and person position data relating to the position of a person in the image captured by the image pickup unit in the image. Based on the image captured by the image pickup unit and the person position processing unit that generates the image, a predetermined action of the monitored person to be monitored is detected, and the detection result is the image data of the image captured by the image pickup unit. It also includes a first monitoring processing unit that transmits a monitoring information communication signal accommodating the person position data generated by the person position processing unit by the first communication unit. The terminal device in the monitored person monitoring system is accommodated in a second communication unit that receives the monitoring information communication signal, a display unit that displays the monitoring information communication signal, and a monitoring information communication signal received by the second communication unit. The detection result and the image of the image data are displayed on the display unit, and the person position indicator indicating the position of the person based on the person position data accommodated in the monitoring information communication signal is displayed in the image displayed on the display unit. It includes a second monitoring processing unit having a display processing unit for displaying. The monitoring information communication signal may be directly transmitted from the sensor device to the terminal device, or indirectly from the sensor device to the terminal device via another device (for example, the following management server device). May be sent. The terminal device may be one type of device, but in the following description, the terminal device is two types of devices, a fixed terminal device and a mobile terminal device. The main difference between these fixed terminal devices and mobile terminal devices is that while fixed terminal devices are operated in a fixed manner, mobile terminal devices are carried and operated by observers (users) such as nurses and caregivers, for example. Since these fixed terminal devices and mobile terminal devices are substantially the same, the mobile terminal device will be mainly described below.

FIG. 1 is a diagram showing a configuration of a monitored person monitoring system according to the embodiment. FIG. 2 is a diagram showing a configuration of a sensor device in the monitored person monitoring system of the embodiment. FIG. 3 is a diagram for explaining the person position data and the person position sign of the first aspect. FIG. 4 is a diagram for explaining the person position data (person area data) and the person position sign of the second aspect. FIG. 5 is a diagram for explaining the person position data (person area data) and the person position sign of the third aspect. FIG. 6 is a diagram for explaining the person position data (person area data) and the person position sign of the fourth aspect. FIG. 7 is a diagram for explaining the person position data (person area data) and the person position sign of the fifth aspect. FIG. 8 is a diagram showing a configuration of a mobile terminal device in the monitored person monitoring system of the embodiment.

More specifically, the monitored person monitoring system MS includes, for example, one or a plurality of sensor devices SU (SU-1 to SU-4), a management server device SV, and a fixed terminal device, as shown in FIG. It is equipped with an SP, one or more mobile terminal devices TA (TA-1, TA-2), and a private branch exchange (PBX, Private branch exchange) CX, which are wired or wireless and are LAN (Local Area Network). Etc. (network, communication line) Connected so that communication is possible via NW. The network NW may be provided with repeaters such as repeaters, bridges and routers that relay communication signals. In the example shown in FIG. 1, the plurality of sensor devices SU-1 to SU-4, the management server device SV, the fixed terminal device SP, the plurality of mobile terminal devices TA-1, TA-2, and the premises exchange CX are L2 switches. Concentrators (hubs, HUBs) LS and mixed wired and wireless LANs including access point APs (eg LANs according to the 802.11 standard) NWs are communicatively connected to each other. More specifically, the plurality of sensor devices SU-1 to SU-4, the management server device SV, the fixed terminal device SP, and the private branch exchange CX are connected to the line concentrator LS, and the plurality of portable terminal devices TA-1 and TA-2 are connected. Is connected to the concentrator LS via the access point AP. Then, the network NW constitutes a so-called intranet by using a group of Internet protocols such as TCP (Transmission Control Protocol) and IP (Internet Protocol).

The monitored person monitoring system MS is arranged at an appropriate place according to the monitored person Ob. The monitored person (person to be watched over) Ob is, for example, a person who needs nursing care due to illness or injury, a person who needs nursing care due to deterioration of physical ability, or a person living alone. In particular, from the viewpoint of enabling early detection and early response, the monitored person Ob may be a person who needs the detection when a predetermined inconvenient event such as an abnormal state occurs in the person. preferable. Therefore, the monitored person monitoring system MS is suitably arranged in a building such as a hospital, a welfare facility for the elderly, and a dwelling unit according to the type of the monitored person Ob. In the example shown in FIG. 1, the monitored person monitoring system MS is arranged in a plurality of living room RMs in which a plurality of monitored person Obs move in, or in a building of a nursing care facility having a plurality of rooms such as a nurse station.

The premises exchange (line switching machine) CX is connected to the network NW, controls extension telephones such as outgoing calls, incoming calls, and calls between mobile terminal devices TAs, performs extension calls between the mobile terminal devices TAs, and then performs extension calls between the mobile terminal devices TAs. , For example, is connected to an external telephone TL such as a fixed telephone or a mobile phone via a public telephone network PN such as a fixed telephone network or a mobile phone network, and makes, receives, and makes a call between the external telephone TL and the mobile terminal device TA. It is a device that controls an external line telephone such as, etc., and performs an external line telephone between the external line telephone TL and the mobile terminal device TA. The private branch exchange CX is, for example, a digital exchange, an IP-PBX (Internet Protocol Private Branch eXchange), or the like.

The sensor device SU is provided with a communication function for communicating with other devices SV, SP, TA via the network NW, detects a predetermined action in the monitored person Ob, and notifies the detection result to the management server device SV. , A device that accepts a nurse call, notifies the management server device SV to that effect, makes a voice call with the terminal devices SP and TA, generates an image including a moving image, and distributes the moving image to the terminal devices SP and TA. Is. Then, in the present embodiment, when transmitting an image to the terminal devices SP and TA, the sensor device SU is a person related to the position of the person in the image on the image in order to transmit the image to the terminal devices SP and TA. The position data is generated and transmitted to the management server device SV together with the image. For example, as shown in FIG. 2, such a sensor device SU includes an imaging unit 11, a sensor side sound input / output unit (SU sound input / output unit) 12, a nurse call reception operation unit 13, and a sensor side control process. A unit (SU control processing unit) 14, a sensor-side communication interface unit (SU communication IF unit) 15, and a sensor-side storage unit (SU storage unit) 16 are provided.

The imaging unit 11 is a device that is connected to the SU control processing unit 14 and performs imaging under the control of the SU control processing unit 14 to generate an image (image data). The image includes a still image (still image data) and a moving image (moving image data). The image pickup unit 11 is arranged so as to be able to monitor the space (location space, living room RM of the arrangement location in the example shown in FIG. 1) where the monitored person Ob, which is the monitoring target to be monitored, is scheduled to be located, and the location An image (image data) that gives a bird's-eye view of the imaging target is generated by imaging the space as an imaging target from above, and the image (target image) of the imaging target is output to the SU control processing unit 14. Preferably, since it is highly probable that the entire monitored person Ob can be imaged, the imaging unit 11 is scheduled to position the head of the monitored person Ob in the bedding (for example, a bed) on which the monitored person Ob lies. It is arranged so that the imaging target can be imaged from directly above the preset head position (usually, the pillow arrangement position). The sensor device SU acquires an image of the monitored person Ob from above the monitored person Ob, preferably an image taken from directly above the planned head position, by the imaging unit 11.

Such an imaging unit 11 may be a device that generates an image of visible light, but in the present embodiment, it is a device that generates an infrared image so that the monitored person Ob can be monitored even in a relatively dark place. .. In the present embodiment, such an imaging unit 11 is, for example, an imaging optical system that forms an infrared optical image of an imaging target on a predetermined imaging surface, and the light receiving surface is made to coincide with the imaging surface. An image sensor that is arranged and converts an infrared optical image of the imaging target into an electrical signal, and image data that represents an infrared image of the imaging target by image processing the output of the image sensor. It is a digital infrared camera provided with an image processing unit and the like for generating an image. In the present embodiment, the imaging optical system of the imaging unit 11 is preferably a wide-angle optical system (so-called wide-angle lens (including a fisheye lens)) having an angle of view capable of imaging the entire living room RM in which the imaging unit 11 is arranged. ..

The SU sound input / output unit 12 is a circuit for inputting / outputting sound. That is, the SU sound input / output unit 12 is a circuit that is connected to the SU control processing unit 14 and generates and outputs a sound corresponding to an electric signal representing the sound according to the control of the SU control processing unit 14, and is external. This is a circuit for acquiring the sound of the above and inputting it to the sensor device SU. The SU sound input / output unit 12 includes, for example, a speaker that converts an electric signal (sound data) of sound into a mechanical vibration signal (acoustic signal) of sound, and a microphone that converts a mechanical vibration signal of sound in an audible region into an electric signal. And so on. The SU sound input / output unit 12 outputs an electric signal representing an external sound to the SU control processing unit 14, and also converts the electric signal input from the SU control processing unit 14 into a mechanical vibration signal of the sound and outputs it. ..

The nurse call reception operation unit 13 is connected to the SU control processing unit 14, and is a switch circuit such as a push button type switch for inputting a nurse call to the sensor device SU. The nurse call reception operation unit 13 may be connected to the SU control processing unit 14 by wire, or may be connected to the SU control processing unit 14 by, for example, short-range wireless communication such as a Bluetooth (registered trademark) standard.

The SU communication IF unit 15 is a communication circuit connected to the SU control processing unit 14 and for performing communication under the control of the SU control processing unit 14. The SU communication IF unit 15 generates a communication signal containing data to be transferred input from the SU control processing unit 14 according to the communication protocol used in the network NW of the monitored person monitoring system MS, and this generated communication. The signal is transmitted to other devices SV, SP, and TA via the network NW. The SU communication IF unit 15 receives communication signals from other devices SV, SP, and TA via the network NW, extracts data from the received communication signals, and the SU control processing unit 14 can process the extracted data. It is converted into data in a different format and output to the SU control processing unit 14. The SU communication IF unit 15 is configured to include, for example, a communication interface circuit according to the IEEE802.11 standard or the like.

The SU storage unit 16 is a circuit that is connected to the SU control processing unit 14 and stores various predetermined programs and various predetermined data under the control of the SU control processing unit 14. The various predetermined programs include, for example, a SU control program that controls each part of the sensor device SU according to the function of each part, and a person related to the position of a person in the image captured by the imaging unit 11 in the image. A person position processing program that generates position data, an image (image data) captured by the imaging unit 11 by detecting a predetermined action in the monitored person Ob, and a person position data generated by the person position processing program. At the same time, the SU monitoring processing program that notifies the predetermined terminal devices SP and TA via the management server device SV, and when the nurse call reception operation unit 13 receives the nurse call, the management server device SV is notified to that effect. A nurse call processing program that makes a voice call with the terminal devices SP and TA by using the SU sound input / output unit 12 and the like, and a moving image generated by the imaging unit 11 to the terminal devices SP and TA that requested the moving image. A control processing program such as a SU streaming processing program to be distributed by streaming is included. In the various predetermined data, each program such as the sensor device identifier (sensor ID) which is an identifier for identifying and identifying the sensor device SU of the own machine and the communication address of the management server device SV is executed. It contains the data necessary for the program. The SU storage unit 16 includes, for example, a ROM (Read Only Memory) which is a non-volatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) which is a rewritable non-volatile storage element, and the like. The SU storage unit 16 includes a RAM (Random Access Memory) or the like that serves as a working memory of the so-called SU control processing unit 14 that stores data or the like generated during execution of the predetermined program.

The SU control processing unit 14 controls each unit of the sensor device SU according to the function of each unit, detects a predetermined action in the monitored person Ob, and outputs the detection result to the image (image data) and the person position data. At the same time, it notifies the management server device SV, receives a nurse call, notifies the management server device SV to that effect, makes a voice call with the terminal devices SP and TA, and generates an image including a moving image to the terminal. This is a circuit for delivering a moving image to the devices SP and TA. The SU control processing unit 14 includes, for example, a CPU (Central Processing Unit) and peripheral circuits thereof. When the control processing program is executed, the SU control processing unit 14 has a sensor side control unit (SU control unit) 141, a sensor side monitoring processing unit (SU monitoring processing unit) 142, a person position processing unit 143, and a nurse call. It is functionally provided with a processing unit (nurse call processing unit) 144 and a sensor-side streaming processing unit (SU streaming processing unit) 145.

The SU control unit 141 controls each unit of the sensor device SU according to the function of each unit, and controls the entire sensor device SU.

The SU monitoring processing unit 142 detects a predetermined predetermined action (state, situation) in the monitored person Ob based on the image, and generates the image and the person generated by the person position processing unit 143 as described later. It notifies (notifies, transmits) to the management server device SV together with the position data.

More specifically, in the present embodiment, the predetermined actions include, for example, waking up when the monitored person Ob wakes up, getting out of bed when the monitored person Ob leaves the bedding, and falling when the monitored person Ob falls from the bedding. And, there are four actions of a fall in which the monitored person Ob has fallen. The SU monitoring processing unit 142 detects, for example, the head of the monitored person Ob based on the target image captured by the imaging unit 11, and based on the time change of the size of the detected head of the monitored person Ob. Detects getting up, getting out of bed, falling and falling of the monitored person Ob. More specifically, first, the location area of the bedding and the first to third threshold values Th1 to Th3 are stored in advance in the SU storage unit 16 as one of the various predetermined data. The first threshold value Th1 is a value for discriminating between the size of the head in the lying posture and the size of the head in the sitting posture in the region where the bedding is located. The second threshold value Th2 is a value for discriminating whether or not the size of the head in the standing posture in the living room RM excluding the area where the bedding is located. The third threshold value Th3 is a value for discriminating whether or not the size of the head in the lying posture in the living room RM excluding the area where the bedding is located. Then, the SU monitoring processing unit 142 extracts a moving object region from the target image as a region of the person of the monitored person Ob by, for example, the background subtraction method or the frame subtraction method. Next, the SU monitoring processing unit 142 uses the extracted moving body region, for example, by a circular or elliptical Hough transform, by, for example, by pattern matching using a prepared head model, or, for example, for head detection. The head region of the monitored person Ob is extracted by the neural network learned in. Next, the SU monitoring processing unit 142 detects getting up, getting out of bed, falling, and falling from the position and size of the extracted head. For example, in the SU monitoring processing unit 142, the position of the extracted head is within the location region of the bedding, and the size of the extracted head is determined from the size of the lying posture by using the first threshold Th1. When the size of the sitting posture changes with time, it is determined that the person has woken up, and the wake-up is detected. For example, in the SU monitoring processing unit 142, when the position of the extracted head changes with time from the inside of the bedding location area to the outside of the bedding location area, the size of the extracted head is the second threshold value. When the time changes from a certain size to the size of the standing posture by using Th2, it is determined that the person has left the bed, and the person has left the bed. For example, in the SU monitoring processing unit 142, when the position of the extracted head changes with time from the inside of the bedding location area to the outside of the bedding location area, the size of the extracted head is the third threshold value. When the time changes from a certain size to the size of the lying posture by using Th3, it is determined that the person has fallen, and the fall is detected. For example, in the SU monitoring processing unit 142, the position of the extracted head is in the living room RM excluding the area where the bedding is located, and the size of the extracted head is determined by using the third threshold value Th3. When the time changes from the size to the size of the lying posture, it is determined that the person has fallen, and the fall is detected.

Since the area of the person on the image is within the range of a predetermined size (width, size), the moving object area extracted as the area of the person is also within the range of the predetermined size (width, size). It becomes. Therefore, the range to be extracted as the moving body region is set in advance, and when the moving body region is extracted, if the moving body region extracted from the target image by, for example, the background subtraction method or the frame subtraction method is out of the range. A moving body region extracted from this target image by, for example, the background subtraction method or the frame subtraction method may be excluded from the moving body region extracted as a person's region.

When the predetermined action is detected in this way, the SU monitoring processing unit 142 transmits the detection result, the target image used for obtaining the detection result, and the person position data with respect to the target image in the SU communication IF. Unit 15 notifies the management server device SV. More specifically, the SU monitoring processing unit 142 was used to detect the sensor ID of the own machine, the detection result (in this embodiment, one or more of getting up, getting out of bed, falling and falling), and the predetermined action. The target image and a communication signal (first monitoring information communication signal) accommodating the person position data for the target image are transmitted to the management server device SV via the SU communication IF unit 15. The image may be at least one of a still image and a moving image, and in the present embodiment, as will be described later, the still image is first notified and the moving image is distributed according to the user's request. First, the moving image may be delivered, or the still image and the moving image may be transmitted, and the still image and the moving image may be displayed on the terminal devices SP and TA by splitting the screen.

The person position data is generated by the person position processing unit 143 as described later, and is accommodated in the first monitoring information communication signal as metadata in the image data of the target image. For example, the image data and the person position data are stored in an Exif (Exif (Exifable image file format) file). More specifically, the Exif file is a file conforming to the JPEG (Joint Photographic Experts Group) file format, and so-called Exif information such as shooting conditions is stored in the application marker segment. The person position data is stored in this application marker segment. Further, for example, the image data and the person position data are stored as form data in a multipart format file in HTTP (Hypertext Transfer Protocol). More specifically, the image data is accommodated as one form data in the multipart format file, and the person position data is accommodated as another form data in the multipart format file.

The person position processing unit 143 generates person position data regarding the position of the person in the image captured by the image pickup unit 11. In the present embodiment, the image is the target image used to detect a predetermined behavior in the monitored person. As the person position data, for example, the data of the following first to fifth aspects can be used.

The person position data of the first aspect is the coordinate data of the position itself of the person in the image on the image. When the person position data of the first aspect is used, more specifically, the person position processing unit 143 first, like the SU monitoring processing unit 142, first, for example, background subtraction from the target image captured by the imaging unit 11. The moving body area MB is extracted as a person's area by the method or the frame subtraction method. Then, as shown in FIG. 3, the person position processing unit 143 obtains the center of gravity position P1 in the extracted moving body region MB, and uses the coordinate data (x, y) of the obtained center of gravity position P1 as the person position in the first aspect. Obtained as data (position of the person in the image on the image = center of gravity position P1). The coordinate data (x, y) is similarly given as the person position data of the following second to fifth aspects at a pixel position having the upper left vertex of the target image as the coordinate origin. When the person position data of the first aspect is used, the person position indicator displayed on the mobile terminal device TA is an image of a point represented by the coordinate data (x, y) of the center of gravity position P1 as described later. It becomes HP1.

The person position data of the second aspect is the person area data relating to the area of the person including the position of the person in the image on the image (for example, the center of gravity position P1 or the like), and the person area data is the person. It is coordinate data representing a rectangle including all or a part of the area of. When the person position data of the second aspect is used, more specifically, the person position processing unit 143 extracts the moving body area MB as the area of the person in the same manner as described above, and as shown in FIG. A rectangle including all or a part of the moving body region MB (for example, an circumscribing rectangle with respect to the moving body region MB) is obtained, and coordinate data representing the obtained rectangle is obtained as the person position data (person area data) of the second aspect. The coordinate data representing this rectangle is, for example, composed of coordinate data (x0, y0) and (x1, y1) of both end points P20 and P21 of one diagonal line in the rectangle. Further, for example, the coordinate data representing the rectangle is composed of the coordinate data of each of the four vertices in the rectangle (not shown). When the person position data of the second aspect is used, the person position indicator displayed on the mobile terminal device TA is represented by the person area data (x0, y0) and (x1, y1) as described later. Image, that is, a rectangular image (an image of a rectangular outline) HP2.

The person position data of the third aspect is the person area data relating to the area of the person including the position of the person in the image on the image (for example, the center of gravity position P1 or the like), and the person area data is the person. It is coordinate data representing the outline of the area of. When the person position data of the third aspect is used, more specifically, the person position processing unit 143 extracts the moving body area MB as the area of the person in the same manner as described above, and as shown in FIG. The contour line of the moving body region MB is obtained, and the coordinate data representing the obtained contour line is obtained as the person position data (person area data) of the third aspect. The coordinate data representing the contour line of the moving body region MB is the coordinate data (x0, y0), (x1, y1), (x1, y1) of each of the vertices P30, P31, P32, ..., P3C in the contour line of the moving body region MB. It consists of x2, y2), ..., (Xpc, ypc). More specifically, when the moving body region MB has a relatively complicated shape, the number of each vertex in the contour line of the moving body region MB becomes large, and the data given to the person position data in the first monitoring information communication signal. It may exceed the capacity (for example, the data capacity of several to a dozen coordinate data). Therefore, the person position processing unit 143 forms a mosaic by gathering the pixels constituting the moving body area MB as a person's area with a plurality of adjacent pixels, thereby forming the moving body area MB with a plurality of mosaics ( Mosaic the moving body area MB). By enlarging this mosaic, the number of mosaics constituting the moving body region MB is reduced, and as a result, the number of vertices in the contour line of the moving body region MB composed of the plurality of mosaics is also reduced. Therefore, the person position processing unit 143 sequentially increases the size of the mosaic and adjusts the size of the mosaic until the data capacity is equal to or less than the data capacity given to the person position data in the first monitoring information communication signal. When the person position data of the third aspect is used, the person position indicator displayed on the mobile terminal device TA is the person area data (x0, y0), (x1, y1), (x2) as described later. , Y2), ..., (Xpc, ypc), that is, the image HP3 of the contour line in the area of the person.

The person position data of the fourth aspect is the person area data relating to the area of the person including the position of the person in the image on the image (for example, the center of gravity position P1 or the like), and the person area data is the person. It is coordinate data representing a circle including all or a part of the area of. When the person position data of the fourth aspect is used, more specifically, the person position processing unit 143 extracts the moving body area MB as the area of the person as described above, and as shown in FIG. 6, A circle including all or a part of the moving body region MB (for example, an circumscribed circle with respect to the moving body region MB) is obtained, and coordinate data representing the obtained circle is obtained as the person position data (person area data) of the fourth aspect. The coordinate data representing this circle includes, for example, the coordinate data (x, y) of the center point P4 in the circle and the radius r. Further, for example, the coordinate data representing the circle includes the coordinate data (x, y) of the center of gravity position P4 in the moving body region MB and the radius r of a preset fixed value. When the person position data of the fourth aspect is used, the person position indicator displayed on the mobile terminal device TA is an image represented by the person area data (x, y), r, that is, that is, as described later. , Circular image (image of circular contour line) HP4.

The person position data of the fifth aspect is the person area data relating to the area of the person including the position of the person in the image on the image (for example, the center of gravity position P1 or the like), and the person area data is the person. It is coordinate data representing an ellipse including all or a part of the area of. When the person position data of the fifth aspect is used, more specifically, the person position processing unit 143 extracts the moving body area MB as the area of the person in the same manner as described above, and as shown in FIG. 7, An ellipse including all or a part of the moving body region MB is obtained, and the coordinate data representing the obtained ellipse is obtained as the person position data (person area data) of the fifth aspect. More specifically, the person position processing unit 143 provides one appropriate first setting point P51 on the contour line in the moving body region MB as the area of the person, and on the contour line farthest from the first setting point P51. A point is obtained as a second set point P53, and a line segment P51 P53 connecting the first set point P51 and the second set point P53 is defined as an elliptical long axis. Next, the person position processing unit 143 passes through the center point P50 of the line segment P51 P53 connecting the first set point P51 and the second set point P53 as the long axis, and the straight line perpendicular to the line segment P51 P53 and the contour. Two intersections with the line segment are obtained, the lengths of each of these two intersections and the center point P50 are obtained, and the intersection that gives a longer length from the two intersections is defined as the intersection P52 between the minor axis and the ellipse. To do. As a result, the major axis, the minor axis, and the intersection of the major axis and the minor axis are obtained, and an ellipse is obtained. The coordinate data representing the ellipse includes, for example, an intersection P50 between the major axis and the minor axis, one intersection P51 with the major axis on the ellipse (first set point) P51, and one intersection P52 with the minor axis on the ellipse. Each coordinate data (x0, y0), (x1, y1), (x2, y2) of. Further, for example, the coordinate data representing the ellipse includes coordinate data (x0, y0) at the intersection P50 between the major axis and the minor axis, and an angle formed by the major axis (or minor axis) and the x-axis (or y-axis). It consists of a major axis length and a minor axis length. When the person position data of the fifth aspect is used, the person position indicator displayed on the mobile terminal device TA is the person area data (x0, y0), (x1, y1), (x2, y2). The represented image, that is, an elliptical image (an image of an elliptical contour line) HP5.

Then, the person position processing unit 143 notifies the SU monitoring processing unit 142 of the person position data generated in this way. For example, in the present embodiment, the person position data of the second aspect is used, and the person position processing unit 143 notifies the SU monitoring processing unit 142 of the person position data of the second aspect. Hereinafter, the case where the person position data of the second aspect is used will be described, but the case where the person position data of another aspect is used can also be described in the same manner.

When the nurse call processing unit 144 receives the nurse call at the nurse call reception operation unit 13, the nurse call processing unit 144 notifies the management server device SV to that effect, and uses the SU sound input / output unit 12 or the like to connect with the terminal devices SP and TA. A voice call is made between them. More specifically, when the nurse call reception operation unit 13 is input-operated, the nurse call processing unit 144 accommodates the sensor ID of the own machine and the nurse call reception information indicating that the nurse call has been received. 1 The nurse call notification communication signal is transmitted to the management server device SV via the SU communication IF unit 15. Then, the nurse call processing unit 144 uses the SU sound input / output unit 12 and the like to perform a voice call with the terminal devices SP and TA by, for example, VoIP (Voice over Internet Protocol).

When the SU streaming processing unit 145 receives a request for video distribution from the fixed terminal device SP or the mobile terminal device TA via the SU communication IF unit 15, the SU streaming processing unit 145 receives the request for the fixed terminal device SP or the mobile terminal device TA. The moving image (for example, a live moving image) generated by the imaging unit 11 is distributed via the SU communication IF unit 15 by streaming reproduction.

In FIG. 1, four first to fourth sensor devices SU-1 to SU-4 are shown as an example, and the first sensor device SU-1 is Mr. A who is one of the monitored persons Ob. The second sensor device SU-2 is arranged in the living room RM-1 (not shown) of Ob-1, and is arranged in the living room RM-2 (not shown) of Mr. B Ob-2 who is one of the monitored persons Ob. The third sensor device SU-3 is installed in the living room RM-3 (not shown) of Mr. C Ob-3, who is one of the monitored persons Ob, and the fourth sensor device SU-4 is installed. It is arranged in the living room RM-4 (not shown) of Mr. D Ob-4, who is one of the monitored persons Ob.

The management server device SV has a communication function for communicating with other devices SU, TA, and SP via the network NW, and receives the detection result, the target image, and the person position data regarding the monitored person Ob from the sensor device SU. Then, the information (monitoring information) related to the monitoring of the monitored person Ob is managed, and the detected detection result, the target image and the person position data regarding the received monitored person Ob are notified (re-reported) to the predetermined terminal devices SP and TA. It is a device that notifies and transmits). More specifically, the management server device SV has a correspondence relationship between the sensor device SU (sensor ID) of the notification source and the terminal devices SP and TA (terminal ID) of the notification destination (re-notification destination) (notification destination correspondence relationship). , And the correspondence relationship (communication address correspondence relationship) between each device SU, SP, TA (each ID) and its communication address is stored in advance. The terminal ID is an identifier for identifying and identifying the terminal devices SP and TA. First, when the management server device SV receives the first monitoring information communication signal, it is accommodated in the sensor device of the notification source (source) in the received first monitoring information communication signal and the received first monitoring information communication signal. The data is associated with each other and stored (recorded) as monitoring information of the monitored person Ob. Then, the management server device SV identifies the terminal devices SP and TA of the notification destination corresponding to the sensor device SU of the notification source in the received first monitoring information communication signal from the notification destination correspondence relationship, and the notification destination The second monitoring information communication signal is transmitted to the terminal devices SP and TA. The second monitoring information communication signal includes the sensor ID accommodated in the received first monitoring information communication signal, the detection result, the target image and the person position data, and the received as a download destination of the moving image. The communication address corresponding to the sensor device SU having the sensor ID accommodated in the first monitoring information communication signal is accommodated. The communication address is obtained from the communication address correspondence relationship. Further, when the management server device SV receives the first nurse call notification communication signal, it is accommodated in the sensor device of the notification source in the received first nurse call notification communication signal and the received first nurse call notification communication signal. The data is associated with each other and stored (recorded) as monitoring information of the monitored person Ob. Then, the management server device SV identifies the terminal devices SP and TA of the notification destination corresponding to the sensor device of the notification source in the received first nurse call notification communication signal from the notification destination correspondence relationship, and the notification destination The second nurse call notification communication signal is transmitted to the terminal devices SP and TA. The second nurse call notification communication signal includes the sensor ID and the nurse call reception information contained in the received first nurse call notification communication signal. The second nurse call notification communication signal may include a communication address corresponding to the sensor device SU having the sensor ID accommodated in the received first nurse call notification communication signal as the download destination of the moving image. The management server device SV provides the client with data according to the request of the client (terminal device SP, TA, etc. in this embodiment). Such a management server device SV can be configured by, for example, a computer having a communication function.

The fixed terminal device SP includes a communication function for communicating with other devices SU, SV, and TA via a network NW, a display function for displaying predetermined information, an input function for inputting predetermined instructions and data, and the like. The user interface (UI) of the monitored person monitoring system MS by inputting predetermined instructions and data given to the management server device SV and the mobile terminal device TA, displaying the monitoring information obtained by the sensor device SU, and the like. ) Is a device that functions as. Such a fixed terminal device SP can be configured by, for example, a computer having a communication function. The fixed terminal device SP as an example of the terminal device operates in the same manner as the mobile terminal device TA, but in this specification, the mobile terminal device TA which is another example of the terminal device will be described.

The mobile terminal device TA has a communication function for communicating with other devices SV, SP, and SU via a network NW, a display function for displaying predetermined information, an input function for inputting predetermined instructions and data, and a voice call. It is equipped with a call function to perform, inputting predetermined instructions and data to be given to the management server device SV and the sensor device SU, and monitoring information (including moving images) obtained by the sensor device SU by a report from the management server device SV. It is a device for displaying, answering a nurse call, and calling out by a voice call with the sensor device SU.

In the present embodiment, such a portable terminal device TA includes, for example, a terminal-side communication interface unit (TA communication IF unit) 31 and a terminal-side control processing unit (TA control processing unit) 32, as shown in FIG. , Terminal side storage unit (TA storage unit) 33, terminal side sound input / output unit (TA sound input / output unit) 34, terminal side input unit (TA input unit) 35, and terminal side display unit (TA display unit). 36 and a terminal side interface unit (TAIF unit) 37 are provided.

The TA communication IF unit 31 is a communication circuit that is connected to the TA control processing unit 32 and performs communication under the control of the TA control processing unit 32, similarly to the SU communication IF unit 15. The TA communication IF unit 31 is configured to include, for example, a communication interface circuit according to the IEEE802.11 standard or the like.

Like the SU sound input / output unit 12, the TA sound input / output unit 34 is connected to the TA control processing unit 32, and generates and outputs a sound corresponding to an electric signal representing the sound according to the control of the TA control processing unit 32. This is a circuit for acquiring an external sound and inputting it to the portable terminal device TA.

The TA input unit 35 is a circuit connected to the TA control processing unit 32, accepting a predetermined operation and inputting it to the mobile terminal device TA, for example, a plurality of input switches to which a predetermined function is assigned. .. The predetermined operation includes, for example, an ID input operation for logging in, a voice call request operation and its termination operation, a live video request operation and its termination operation, and the notified monitored person. Input operation to the effect (“correspondence”) that there is an intention to perform response (coping, response) such as lifesaving, nursing, nursing care, and assistance to Ob, and instructions (command, command, to enlarge and display the person in the image). It includes various operations necessary for monitoring, such as input operations for enlarged display instructions (commands). The TA display unit 36 is connected to the TA control processing unit 32, and according to the control of the TA control processing unit 32, a predetermined operation content input from the TA input unit 35 and a subject monitored by the monitored person monitoring system MS. The monitoring information regarding the monitoring of the observer Ob (for example, the type of a predetermined action detected by the sensor device SU, the image (still image and moving image) of the monitored person Ob, and the person position indicating the position of the person based on the person position data). It is a circuit that displays signs, reception of nurse calls, etc.), and is a display device such as an LCD (liquid crystal display) and an organic EL display. In the present embodiment, the touch panel is composed of the TA input unit 35 and the TA display unit 36. In this case, the TA input unit 35 is a position input device that detects and inputs an operation position such as a resistance film method or a capacitance method. In this touch panel, a position input device is provided on the display surface of the TA display unit 36, and one or a plurality of input content candidates that can be input to the TA display unit 36 are displayed. When the observer) touches the display position displaying the input content to be input, the position input device detects the position, and the display content displayed at the detected position is displayed on the mobile terminal device TA as the user's operation input content. Entered.

The TAIF unit 37 is a circuit that is connected to the TA control processing unit 32 and inputs / outputs data to / from an external device according to the control of the TA control processing unit 32. For example, the Bluetooth (registered trademark) standard is used. An interface circuit, an interface circuit that performs infrared communication such as the IrDA standard, and an interface circuit that uses the USB standard.

The TA storage unit 33 is a circuit that is connected to the TA control processing unit 32 and stores various predetermined programs and various predetermined data under the control of the TA control processing unit 32. The various predetermined programs include, for example, a TA control program that controls each part of the mobile terminal device TA according to the function of each part, and the detection result received from the sensor device SU via the management server device SV. A sensor device by using a TA monitoring processing program that stores (records) monitoring information related to monitoring of the monitored person Ob such as the nurse call and displays the detection result and the nurse call, a TA sound input / output unit 34, and the like. A control processing program such as a TA call processing program that makes a voice call with the SU, or a streaming processing program that receives video distribution from the sensor device SU and displays the distributed video on the TA display unit 36 by streaming playback. Is included. In the TA monitoring processing program, the detection result and the image of the image data accommodated in the second monitoring information communication signal are displayed on the TA display unit 36, and the person based on the person position data accommodated in the second monitoring information communication signal. A display processing program for displaying the position indicator in the image displayed on the TA display unit 36 is included. In the various predetermined data, data necessary for executing each program such as the terminal ID of the own machine, the screen information displayed on the TA display unit 36, and the monitoring information related to the monitoring of the monitored person Ob, etc. Is included. The TA storage unit 33 includes, for example, a ROM, an EEPROM, or the like. The TA storage unit 33 includes a RAM or the like that serves as a working memory of the so-called TA control processing unit 32 that stores data or the like generated during execution of the predetermined program. Then, the TA storage unit 33 functionally includes a terminal-side monitoring information storage unit (TA monitoring information storage unit) 331 for storing the monitoring information.

The TA monitoring information storage unit 331 stores the monitoring information of the monitored person Ob transmitted and received to and from each of the devices SV, SP, and SU. More specifically, in the present embodiment, the TA monitoring information storage unit 331 has the sensor ID, the detection result, the target image, which are accommodated in the second monitoring information communication signal received from the management server device SV, as the monitoring information. The communication address of the sensor device SU to which the person position data and the moving image are downloaded, the reception time of the second monitoring information communication signal, etc. are stored in association with each other, and the second nurse call notification communication received from the management server device SV. The sensor ID and nurse call reception information stored in the signal, the reception time of the second nurse call notification communication signal, and the like are stored in association with each other.

The TA control processing unit 32 controls each unit of the mobile terminal device TA according to the function of each unit, receives and displays the monitoring information for the monitored person Ob, and is a circuit for answering a nurse call or calling out. Is. The TA control processing unit 32 includes, for example, a CPU and peripheral circuits thereof. When the control processing program is executed, the TA control processing unit 32 includes the terminal side control unit (TA control unit) 321 and the terminal side monitoring processing unit (TA monitoring processing unit) 322, the call processing unit 323, and the terminal side streaming processing. A unit (TA streaming processing unit) 324 is functionally provided. The TA monitoring processing unit 322 functionally includes a display processing unit 3221.

The TA control unit 321 controls each part of the mobile terminal device TA according to the function of each part, and controls the entire control of the mobile terminal device TA.

The TA monitoring processing unit 322 stores (records) the detection result received from the sensor device SU via the management server device SV and the monitoring information regarding the monitoring of the monitored person Ob such as the nurse call, and stores (records) the detection result and the above. It displays a nurse call. More specifically, when the TA monitoring processing unit 322 receives the second monitoring information communication signal from the management server device SV, the TA monitoring processing unit 322 receives the monitoring information of the monitored person Ob contained in the received second monitoring information communication signal. It is stored (recorded) in the TA monitoring information storage unit 331. The TA monitoring processing unit 322 displays a screen corresponding to each information accommodated in the received second monitoring information communication signal on the TA display unit 36 by the display processing unit 3221. More specifically, the display processing unit 3221 displays the detection result and the target image stored in the received second monitoring information communication signal on the TA display unit 36, and stores the detection result and the target image in the received second monitoring information communication signal. The person position sign based on the person position data is displayed in the target image displayed on the TA display unit 36. Then, in the present embodiment, when the TA input unit 35 receives the enlarged display instruction, the display processing unit 3221 enlarges the size of the area of the person displayed on the TA display unit 36 and increases the size of the person. Display the area. When the TA monitoring processing unit 322 receives the second nurse call notification communication signal from the management server device SV, the TA monitoring processing unit 322 stores the monitoring information of the monitored person Ob contained in the received second nurse call notification communication signal in the TA monitoring information storage. It is stored (recorded) in the unit 331. The TA monitoring processing unit 322 displays the nurse call reception screen stored in advance in the TA storage unit 33 on the TA display unit 36 in response to the nurse call reception information stored in the received second nurse call notification communication signal. .. Then, when the TA monitoring processing unit 322 receives a predetermined input operation from the TA input unit 35, the TA monitoring processing unit 322 executes a predetermined processing according to the input operation.

The call processing unit 323 makes a voice call with the sensor device SU by using the TA sound input / output unit 34 and the like. More specifically, the call processing unit 323 uses the TA sound input / output unit 34 and the like to transmit the first monitoring information communication signal and the first nurse call notification communication signal to the management server device SV. Or, a voice call is made with, for example, VoIP with a sensor device SU or the like selected and designated by a user (monitorer) of the mobile terminal device TA.

The TA streaming processing unit 324 receives the distribution of the moving image from the sensor device SU, and displays the distributed moving image on the TA display unit 36 by streaming reproduction.

Such a mobile terminal device TA can be configured by a portable communication terminal device such as a so-called tablet computer, a smartphone, or a mobile phone.

In the present embodiment, the TA input unit 35 and the TA display unit 36 constituting the touch panel receive an instruction input unit (first instruction input unit) that receives an enlarged display instruction which is an instruction to enlarge and display a person appearing in the image. ), And also corresponds to an example of a second instruction input unit that receives an operation instruction that is an instruction to perform a predetermined operation.

Next, the operation of this embodiment will be described. FIG. 9 is a flowchart showing an operation related to the monitoring information of the sensor device in the monitored person monitoring system of the embodiment. FIG. 10 is a flowchart showing an operation related to monitoring information of the mobile terminal device in the monitored person monitoring system of the embodiment. FIG. 11 is a diagram showing an example of a standby screen displayed on the mobile terminal device in the monitored person monitoring system of the embodiment. FIG. 12 is a diagram showing an example of a monitoring information screen displayed on the mobile terminal device in the monitored person monitoring system of the embodiment. FIG. 13 is a diagram showing an example of a monitoring information screen in which the area of a person is enlarged, which is displayed on the mobile terminal device in the monitored person monitoring system of the embodiment.

In such a monitored person monitoring system MS, each device SU, SV, SP, and TA executes necessary initialization of each part when the power is turned on, and starts its operation. In the sensor device SU, by executing the control processing program, the SU control processing unit 14 includes the SU control unit 141, the SU monitoring processing unit 142, the person position processing unit 143, the nurse call processing unit 144, and the SU streaming processing unit 145. Functionally configured. In the mobile terminal device TA, the TA control processing unit 32 is functionally configured with the TA control unit 321 and the TA monitoring processing unit 322, the call processing unit 323, and the TA streaming processing unit 324 by executing the control processing program. The TA monitoring processing unit 322 is functionally configured with a display processing unit 3221.

The operation of the sensor device SU will be described. The sensor device SU detects a predetermined operation of the monitored person Ob by operating as follows for each frame or every few frames, and determines whether or not a nurse call is accepted.

In FIG. 9, first, the sensor device SU acquires an image (image data) for one frame from the imaging unit 11 as the target image by the SU control unit 141 of the SU control processing unit 14 (S11).

Next, the sensor device SU is executed by the SU monitoring processing unit 142 of the SU control processing unit 14 based on the acquired target image to perform an action detection process for detecting a predetermined action on the monitored person Ob (S12).

Next, the sensor device SU determines whether or not a predetermined action in the monitored person Ob is detected by the action detection process S12 by the SU monitoring processing unit 142. As a result of this determination, if the action is not detected (No), the sensor device SU then executes the process S16, while if the action is detected (Yes), the sensor device SU executes the process S16. The apparatus SU executes the next process S14 and the process S15 in sequence, and then executes the process S16.

In this process S14, the sensor device SU generates the person position data regarding the position of the person in the acquired target image in the image by the person position processing unit 143 of the SU control processing unit 14. The person position data may be any of the above-mentioned person position data of the first to fifth aspects, but in the present embodiment, for example, the person position data of the second aspect is adopted. In this case, the person position processing unit 143 first extracts the moving object region MB as the region of the person from the acquired target image by, for example, the background subtraction method or the frame subtraction method. Alternatively, the person position processing unit 143 acquires the moving body region MB extracted by the SU monitoring processing unit 142 in the processing S12 from the SU monitoring processing unit 142. Next, the person position processing unit 143 obtains a rectangle including all or a part of the moving body area MB, and obtains coordinate data representing the obtained rectangle as person position data (person area data). The coordinate data representing the rectangle is, for example, composed of the coordinate data of both end points of one diagonal line in the rectangle (P20 (x0, y0), P21 (x1, y1) in the example shown in FIG. 4). Then, the person position processing unit 143 notifies the SU monitoring processing unit 142 of the obtained person position data (person area data). When there are a plurality of moving body region MBs, the person position data may be a plurality of people corresponding to each of the plurality of moving body region MBs.

In the next process S15 of the process S14, the sensor device SU monitors the SU in order to notify the predetermined terminal devices SP and TA via the management server device SV of the predetermined actions detected in the processes S12 and S13. The unit 142 transmits the first monitoring information communication signal to the management server device SV. More specifically, the SU monitoring processing unit 142 is a first monitoring information communication signal accommodating the sensor ID of the own machine, the detection result, the target image, and the person position data (rectangular person area data in this example). Is transmitted to the management server device SV via the SU communication IF unit 15. Therefore, the first monitoring information communication signal for notifying the detection result includes a target image as an image of the monitored person Ob together with the detection result, and the person position data regarding the position of the person reflected in the target image is the target. Included with image.

In the process S16, the sensor device SU determines whether or not the nurse call is accepted by the nurse call processing unit 144 of the SU control processing unit 14. That is, the processes S11 to S17 shown in FIG. 9 are repeatedly executed every frame or every few frames, but the nurse is executed between the previous execution of the process S16 and the current execution of the process S16. It is determined whether or not the call reception operation unit 13 has been operated. As a result of this determination, if the nurse call reception operation unit 13 is not operated and the nurse call is not accepted (No), the sensor device SU ends the current processing, while the nurse call reception operation unit 13 Is operated and the nurse call is accepted (Yes), the sensor device SU ends the current process after executing the next process S17.

In this process S17, in order to notify the predetermined terminal devices SP and TA via the management server device SV of the nurse call whose reception is found in the process S16, the sensor device SU is first set by the nurse call processing unit 144. The nurse call notification communication signal is transmitted to the management server device SV. More specifically, the nurse call processing unit 144 transmits the first nurse call notification communication signal containing the sensor ID of the own device and the nurse call reception information to the management server device SV via the SU communication IF unit 15. ..

The sensor device SU operates as described above with respect to the detection of a predetermined action in the monitored person Ob and the reception of the nurse call.

When the management server device SV receives the first monitoring information communication signal from the sensor device SU via the network NW, the management server device SV receives the sensor ID, the determination result, the target image, and the person position data (this) stored in the first monitoring information communication signal. In the embodiment, rectangular person area data) and the like are stored (recorded) as monitoring information of the monitored person Ob monitored by the sensor device SU having this sensor ID. Then, the management server device SV identifies the terminal devices SP and TA of the notification destination corresponding to the sensor device SU of the notification source in the received first monitoring information communication signal from the notification destination correspondence relationship, and the notification destination The second monitoring information communication signal is transmitted to the terminal devices SP and TA. Further, when the management server device SV receives the first nurse call notification communication signal from the sensor device SU via the network NW, the management server device SV receives the sensor ID, the nurse call reception information, and the like accommodated in the first nurse call notification communication signal. It is stored (recorded) as monitoring information of the monitored person Ob monitored by the sensor device SU having this sensor ID. Then, the management server device SV identifies the terminal devices SP and TA of the notification destination corresponding to the sensor device SU of the notification source in the received first nurse call notification communication signal from the notification destination correspondence relationship, and this notification destination. The second nurse call notification communication signal is transmitted to the terminal devices SP and TA of the above.

When the fixed terminal device SP and the mobile terminal device TA receive the second monitoring information communication signal from the management server device SV via the network NW, the fixed terminal device SP and the mobile terminal device TA relate to the monitoring of the monitored person Ob accommodated in the second monitoring information communication signal. The monitoring information is displayed. The operation of displaying this monitoring information by the mobile terminal device TA will be described in detail below. Further, when the fixed terminal device SP and the mobile terminal device TA receive the second nurse call notification communication signal from the management server device SV via the network NW, the fixed terminal device SP and the mobile terminal device TA receive the sensor ID accommodated in the second nurse call notification communication signal. It displays that the nurse call has been accepted from the monitored person Ob monitored by the sensor device SU. By such an operation, the monitored person monitoring system MS roughly detects a predetermined action in each monitored person Ob by each sensor device SU, management server device SV, fixed terminal device SP, and mobile terminal device TA. Each monitored person Ob is monitored.

Next, the operation of displaying the monitoring information regarding the monitoring of the monitored person Ob and the operation related thereto in the terminal devices SP and TA will be described. Here, the operation of the mobile terminal device TA will be typically described.

As described above, when the power is turned on and the operation is started, the mobile terminal device TA accepts a login operation by a monitor (user) such as a nurse or a caregiver, and displays the TA monitoring processing unit 322. The TA display unit 36 displays a standby screen for waiting for a communication signal addressed to the own unit by the unit 3221. As shown in FIG. 11, the standby screen 51 has, for example, a menu bar area 511 for displaying a menu bar, and a standby main area for displaying a message (for example, "no notification") and an icon indicating that the user is in standby. 512, a time area 513 that displays the current time, a date area 514 that displays today's date, and a user name area 515 that displays the user name that is currently logged in to the mobile terminal device TA. To be equipped with. The menu bar area 511 is provided with an off-hook button 5111 for inputting an instruction for making an extension call with another mobile terminal device TA or making an outside line call with the outside line telephone TL.

Then, in FIG. 10, the mobile terminal device TA determines whether or not the TA communication IF unit 31 has received the communication signal by the TA control unit 321 of the TA control processing unit 32 (S21). As a result of this determination, if the communication signal is not received (No), the mobile terminal device TA returns the process to S21, and if the result of the determination is that the communication signal is received (Yes), the mobile terminal device TA returns the process to S21. The mobile terminal device TA executes the following process S22. That is, the mobile terminal device TA is waiting for the reception of the communication signal.

In the process S22, the mobile terminal device TA determines the type of the received communication signal by the TA control unit 321. As a result of this determination, when the received communication signal is the second monitoring information communication signal (second monitoring information), the mobile terminal device TA sequentially executes the following processes S23 and S24, and then processes S27. When the received communication signal is the second nurse call notification communication signal (second NC notification), the following processes S25 and S26 are sequentially executed, and then the process S27 is executed, and the received communication signal is received. When the communication signal is not the second monitoring information communication signal and the second nurse call notification communication signal (other), this process is performed after executing process S29 which performs appropriate processing according to the communication signal received in process S21. finish.

In the process S23, the mobile terminal device TA is monitored by the TA monitoring process unit 322 of the TA control process unit 32 for the monitored person Ob, which is accommodated in the second monitoring information communication signal received from the management server device SV in the process S21. The monitoring information related to the above is stored (recorded) in the TA monitoring information storage unit 331.

Next to the processing S23, the TA monitoring processing unit 322 displays a screen corresponding to each information accommodated in the second monitoring information communication signal received in the processing S21 by the display processing unit 3221, for example, the monitoring information shown in FIG. The screen 52 is displayed on the TA display unit 36 (S24).

The monitoring information screen 52 is a screen for displaying the monitoring information regarding the monitoring of the monitored person Ob. On the monitoring information screen 52, for example, as shown in FIG. 12, the menu bar area 511, the arrangement location of the sensor device SU having the sensor ID accommodated in the second monitoring information communication signal received in the process S21, and the above. The monitored person name area 521 displaying the name of the monitored person Ob monitored by the sensor device SU having the sensor ID, and the reception time of the second monitoring information communication signal received in the process S21 (or the predetermined action). The elapsed time from the detection time), the detection information display area 522 that displays the detection result accommodated in the second monitoring information communication signal received in the process S21, and the second monitoring information communication signal received in the process S21. The accommodated image (that is, the target image captured by the sensor device SU having the sensor ID) (here, a still image) and the position of the person accommodated in the second monitoring information communication signal received in the process S21. It includes an image area 523 that displays a person position indicator HP2 based on data (rectangular person area data in this example), a "corresponding" button 524, a "speak" button 525, and a "view LIVE" button 526. ..

In order to display the location of the sensor device SU and the name of the monitored person Ob in the monitored person name area 521, the TA storage unit 33 has a sensor ID and a location of the sensor device SU having the sensor ID. And the names of the monitored persons Ob monitored by the sensor device SU having the sensor ID are associated with each other and stored in advance.

In the detection information display area 522, the detection result (in this embodiment, each name of getting up, getting out of bed, falling, and falling) stored in the second monitoring information communication signal received in the process S21 may be displayed as it is. , In the present embodiment, it is displayed by an icon symbolically representing the detection result. In order to display with this icon, each action and an icon symbolically representing the action are associated with each other and stored in advance in the TA storage unit 33. In the example shown in FIG. 12, a wake-up icon symbolically representing wake-up is displayed in the detection information display area 522.

In the image area 523, the person position sign HP2 based on the person position data is displayed in the target image being displayed. In the example shown in FIG. 12, the bedding BD and the monitored person Ob are shown in the target image. In the present embodiment, the person position data accommodated in the second monitoring information communication signal received in the process S21 is rectangular person area data, and the display processing unit 3221 is viewed from the upper left in the front view of the target image being displayed. A target displaying a rectangular image HP2 represented by person area data (here, each coordinate data (x0, y0), (x1, y1) of both ends of a diagonal line) with a vertex as a coordinate origin as a person position indicator HP2. Display by superimposing on the image. Then, in the present embodiment, in the area surrounded by the person position sign HP2, the enlarged display instruction, which is an instruction to enlarge and display the person appearing in the target image being displayed, is input to the mobile terminal device TA. It is a button of.

The display processing unit 3221 may display an image in the area of a person by a second display method different from the display method of the target image (first display method). By displaying the inside and outside of the area of the person in different display methods, the user (monitorer) of the mobile terminal device TA can easily find the area of the person in the image, in other words, the monitored person Ob in the image. For example, the first display method is a normal display, the second display method is a mosaic display, and the display processing unit 3221 sets the image in the rectangular person position sign HP as an image in the area of the person. The image outside the rectangular person position sign HP is displayed as usual, and the image inside the rectangular person position sign HP is displayed as a mosaic. Further, for example, the first display method is color display, the second display method is monochrome display (monochromatic display), and the display processing unit 3221 displays the image in the rectangular person position marker HP in the area of the person. As the image inside, the image outside the rectangular person position indicator HP is displayed in color, and the image inside the rectangular person position indicator HP is displayed in monochrome. Further, for example, the first display method is monochrome display, the second display method is color display, and the display processing unit 3221 sets the image in the rectangular person position marker HP as an image in the area of the person. The image outside the rectangular person position indicator HP is displayed in monochrome, and the image inside the rectangular person position indicator HP is displayed in color. Further, for example, the first display method is a positive display, the second display method is a negative display, and the display processing unit 3221 sets the image in the rectangular person position sign HP as an image in the area of the person. The image outside the rectangular person position sign HP is displayed as positive, and the image inside the rectangular person position sign HP is displayed as negative. Further, for example, the first display method is a negative display, the second display method is a positive display, and the display processing unit 3221 sets the image in the rectangular person position sign HP as an image in the area of the person. The image outside the rectangular person position sign HP is displayed as a negative, and the image inside the rectangular person position sign HP is displayed as a positive.

On the monitoring information screen 52, the "correspondence" button 524 intends to take a predetermined response (response, response) such as lifesaving, nursing, nursing care, and assistance to the detection result displayed on the monitoring information screen 52. This is a button for inputting implementation intention information indicating that the user of the mobile terminal device TA exists into the mobile terminal device TA. The "speak" button 525 is a button for requesting a voice call, and is for inputting an instruction for connecting the sensor device SU of the sensor ID and the mobile terminal device TA so as to be able to make a call via the network NW. Button. The "view LIVE" button 526 is a button for requesting a live moving image, and is a button for inputting an instruction to display a moving image captured by the sensor device SU of the sensor ID.

Returning to FIG. 10, on the other hand, in the process S25, the mobile terminal device TA is accommodated in the second nurse call notification communication signal received from the management server device SV in the process S21 by the TA monitoring process unit 322 of the TA control process unit 32. The monitoring information regarding the monitoring of the monitored person Ob is stored (recorded) in the TA monitoring information storage unit 331.

Next to this process S25, the TA monitoring processing unit 322 transmits the nurse call stored in advance in the TA storage unit 33 according to the nurse call reception information stored in the second nurse call notification communication signal received in the processing S21. A schematic nurse call reception screen indicating that the reception has been received is displayed on the TA display unit 36 (S26).

Then, in the process S27 executed after each of the processes S24 and S26, the mobile terminal device TA uses the TA control processing unit 32 to perform an input operation on the touch panel including the TA input unit 35 and the TA display unit 36. Determine if it has been accepted. As a result of this determination, if the input operation is not accepted (No), the mobile terminal device TA returns the process to the process S27, while if the result of the determination is that the input operation is accepted, the mobile terminal device TA returns the process to the process S27. The device TA executes the next process S28.

In this process S28, the mobile terminal device TA executes an appropriate process according to the content of the input operation by the TA control processing unit 32, and ends this process.

For example, the mobile terminal device TA allows the TA control processing unit 32 to perform an input operation (for example, a tap operation) of an area surrounded by the person position sign HP2, which also functions as a button for inputting an enlarged display instruction. Upon receipt, the display processing unit 3221 displays the area of the person on the TA display unit 36 in an enlarged manner larger than the size of the area of the person being displayed. For example, the area surrounded by the person position sign HP2 shown in FIG. 12 is enlarged about 4 times (enlarged by 2 times each in the vertical direction and the horizontal direction), and the area surrounded by the person position sign HPE shown in FIG. 13 is enlarged. Is displayed. The pixel value of the pixel generated by the enlargement is generated by interpolating from the pixel value of the pixel adjacent to the pixel value, for example. Since the area of the person is enlarged and displayed in this way, the user (monitorer) of the mobile terminal device TA can easily find the monitored person Ob in the image, and can easily recognize the state of the monitored person Ob in the image. ..

Further, for example, when the TA control processing unit 32 receives the input operation of the "corresponding" button 524 (that is, when the corresponding intention is received), the portable terminal device TA is currently displayed on the TA display unit 36. The monitoring information of the monitored person Ob is stored in the TA monitoring information storage unit 331 with the fact that "corresponding" has been accepted, and corresponds to the monitoring information of the monitored person Ob displayed on the TA display unit 36. A communication signal (correspondence reception notification communication signal) accommodating the sensor ID and information indicating that "correspondence" is accepted (correspondence reception information) is transmitted to the management server device SV. The management server device SV that has received the correspondence reception notification communication signal transmits the sensor ID contained in the received correspondence reception notification communication signal and the communication signal containing the correspondence reception information (correspondence reception well-known communication signal) by broadcast communication. It is transmitted to the terminal devices SP and TA. As a result, regarding the sensor ID corresponding to the monitoring information of the monitored person Ob displayed on the TA display unit 36, the fact that "corresponding" is accepted is synchronized between the terminal devices SP and TA.

Further, for example, when the TA control processing unit 32 receives the input operation of the "speaking" button 525, the mobile terminal device TA monitors the monitored person Ob displayed on the TA display unit 36 by the call processing unit 323. A communication signal (call request communication signal) containing information such as requesting a voice call is transmitted to the sensor device SU, and the sensor device SU corresponding to the communication signal is connected to the sensor device SU via the network NW so that a voice call can be made. As a result, a voice call can be made between the mobile terminal device TA and the sensor device SU. When the TA control processing unit 32 receives an input operation of the "end" button shown in the figure, which is a button for inputting an instruction to end the voice call, the mobile terminal device TA receives the TA by the call processing unit 323. A communication signal (call end communication signal) containing information such as requesting the end of the voice call is transmitted to the sensor device SU that monitors the monitored person Ob displayed on the display unit 36. As a result, the voice call between the mobile terminal device TA and the sensor device SU is terminated.

Further, for example, when the TA control processing unit 32 receives the input operation of the "view LIVE" button 526, the mobile terminal device TA is monitored by the TA streaming processing unit 324 and is currently displayed on the TA display unit 36. A communication signal (video distribution request communication signal) containing information such as requesting live video distribution is transmitted to the sensor device SU that monitors the person Ob, and the sensor device SU and network NW corresponding to the communication signal are transmitted. It is connected so that the moving image can be downloaded via the sensor device SU, and the moving image is delivered live from the sensor device SU, and the delivered moving image is displayed on the TA display unit 36 by streaming reproduction. In the monitoring information screen 52 that displays the live moving image, the moving image is displayed in the image area 523 instead of the still image, and the "Live end" button is displayed instead of the "Watch LIVE" button 526. .. As a result, a live moving image is displayed on the mobile terminal device TA. The "Live end" button in the illustration is a button for requesting the end of the moving image, and ends (stops) the distribution of the moving image captured by the sensor device SU of the sensor ID and ends (stops) the display. ) Is a button for inputting an instruction to be made. When the TA control processing unit 32 receives the input operation of the "Live end" button, the mobile terminal device TA monitors the monitored person Ob currently displayed on the TA display unit 36 by the TA streaming processing unit 324. A communication signal (video distribution end communication signal) containing information such as requesting the end of video distribution is transmitted to the sensor device SU, and a still image is displayed on the TA display unit 36. As a result, the mobile terminal device TA ends the display of the live moving image.

The mobile terminal device TA operates as described above with respect to the detection result received from the sensor device SU via the management server device SV and each notification (each re-notification) of the nurse call reception.

As described above, the monitored person monitoring system MS, the terminal device SP, TA and the display method implemented therein in the present embodiment are based on the first monitoring information communication signal received by the TA communication IF unit 31. 2 The image of the image data accommodated in the monitoring information communication signal is displayed on the TA display unit 36, and the person position indicator HP2 based on the person position data accommodated in the second monitoring information communication signal is displayed on the TA display unit. A display processing unit 3221 for displaying in an image is provided. Therefore, in the monitored person monitoring system MS, the terminal device SP, TA, and the display method, the person position indicator HP2 is displayed on the displayed image, so that the user (monitorer) can display the person position indicator HP2. As a clue, the monitored person Ob in the image can be found, and the monitored person Ob in the image can be easily found. Since the sensor device SU in the present embodiment accommodates not only the image but also the person position data (person area data in the present embodiment) in the first monitoring information communication signal, the terminal device SP that has received the second monitoring information communication signal. , TA can display the person position indicator HP2 on the displayed image. Therefore, as described above, the users of the terminal devices SP and TA can find the monitored person Ob in the image by using the person position sign HP2 as a clue, and can easily find the monitored person Ob in the image.

In the above-described embodiment, depending on the position of the person in the target image on the target image, for example, as shown in FIG. 14, the area of the person (in the example shown in FIG. 14, the area in the person position marker HP2). In some cases, only a part of the above is displayed on the TA display unit 36. Alternatively, a button for inputting an operation instruction, such as a "corresponding" button 524, a "speaking" button 525, and a "view LIVE" button 526, into the mobile terminal device TA is an image area. When displayed in 523, depending on the position of the person in the target image on the target image, for example, as shown in FIG. 14, the area of the person (in the example shown in FIG. 14, the area in the person position marker HP2). A part or all of the above may overlap with the button, and a part or all of the area of the person may be displayed on the TA display unit 36. In order to eliminate such a case, the display processing unit 3221 may be configured as in the following first and second modified forms. FIG. 14 is a diagram showing an example of a monitoring information screen in which a person's area and a “corresponding” button are overlapped, which is displayed on a mobile terminal device in the monitored person monitoring system of the embodiment. FIG. 15 is a diagram showing an example of a monitoring information screen that displays a person's area at a central position of an image display area displayed on a mobile terminal device in the monitored person monitoring system of the embodiment. FIG. 16 is a diagram showing an example of a monitoring information screen displayed on a mobile terminal device in the monitored person monitoring system of the embodiment, which displays a person's area and a “corresponding” button so as not to overlap each other.

In the first modification, the display processing unit 3221 has the center position SC of the image area 523 for displaying the target image on the TA display unit 36 and the position HPC of the person appearing in the target image on the target image. The target image and the person position indicator HP are displayed on the TA display unit 36 so as to coincide with each other (SC = HPC). The position HPC is, for example, the center of gravity position P1 in the moving body region MB. Further, for example, when the person position data is rectangular person area data, the position HPC is the position of the intersection of diagonal lines in the rectangle. Further, for example, when the person position data is circular person area data, the position HPC is a center point in the circle. Further, for example, when the person position data is elliptical person area data, the position HPC is the position of the intersection of the major axis and the minor axis in the ellipse. More specifically, the pixel position (coordinate data) of the central position SC is stored in advance in the TA storage unit 33, and the display processing unit 3221 stores the pixel position (coordinate data) in the TA storage unit 33, for example, as shown in FIG. The display positions of the target image and the person position marker HP are adjusted so that the position HPC (the position of the intersection of the diagonal lines in the rectangle in this example) is located at the pixel position of the center position SC. Is displayed on the TA display unit 36. According to this, since the user (monitorer) of the mobile terminal device TA only needs to gaze at the central position of the image area 523, it becomes easy to find the monitored person Ob in the image.

In the second modification, the display processing unit 3221 inputs an operation instruction, which is an instruction to perform a predetermined operation, to the mobile terminal device TA in the remaining area excluding the area of the person in the display area of the TA display unit 36. Display the button for Preferably, the display processing unit 3221 displays the button in the remaining area of the TA display unit 36 for displaying the target image, excluding the area of the person. For example, when the button is displayed in the image area 523, for example, as shown in FIG. 14, a part of the person area (the area in the person position indicator HP2 in the example shown in FIG. 14) is a "corresponding" button. When overlapping with 524, as shown in FIG. 16, the display processing unit 3221 is a residual area excluding the person's area (in this example, the area in the person's position indicator HP2), for example, the upper end of the image area 523. A "corresponding" button 524 is displayed in the area. Since the display positions of the buttons such as the "corresponding" button 524, the "speaking" button 525, and the "view LIVE" button 526 are preset, the area where the person's area and the button overlap is determined. It is known in advance. Therefore, the remaining area excluding the area of the person can be predicted and is stored in the TA storage unit 33 in advance. When all or a part of the area of the person overlaps with the button, the display processing unit 3221 may display the button in the remaining area stored in advance in the TA storage unit 33. According to this, the area of the person is not hidden by the button, and the user (monitorer) of the mobile terminal device TA can easily find the monitored person Ob in the image.

The present specification discloses various aspects of the technology as described above, of which the main technologies are summarized below.

The terminal device according to one aspect receives the image data of the image of the monitored person to be monitored and the monitoring information communication signal accommodating the person position data regarding the position of the person in the image on the image. A unit, a display unit for displaying, and an image of image data accommodated in the monitoring information communication signal received by the communication unit are displayed on the display unit, and are based on the person position data accommodated in the monitoring information communication signal. It is provided with a display processing unit that displays a person position indicator indicating the position of the person in the image displayed on the display unit. Preferably, in the terminal device described above, the person position data is accommodated in the monitoring information communication signal as metadata of the image data. Preferably, in the terminal device described above, the image data and the person position data are housed in an Exif (Exif image file format) file. Preferably, in the terminal device described above, the image data and the person position data are stored as form data in a multipart format file in HTTP (Hypertext Transfer Protocol). Preferably, in the terminal device described above, the person position data is the person area data of the area of the person including the position of the person in the image on the image. Preferably, in the terminal device described above, the person position data is person area data relating to the person's area including the position of the person in the image on the image, and the person area data is the person's area. It is coordinate data representing any one of a rectangle, a circle and an ellipse including all or a part of, and the person position indicator is an image represented by the person area data. Preferably, in the terminal device described above, the person position data is person area data relating to the person's area including the position of the person in the image on the image, and the person area data is the person's area. It is coordinate data representing the contour line of, and the person position indicator is an image represented by the person area data.

Such a terminal device displays an image of image data accommodated in the monitoring information communication signal received by the communication unit on the display unit, and determines the position of the person based on the person position data accommodated in the monitoring information communication signal. A display processing unit for displaying a person position indicator to be represented in the image displayed on the display unit is provided. Therefore, in the terminal device, the person position sign is displayed on the image being displayed, so that the user of the terminal device can find the monitored person in the image by using the person position sign as a clue, and in the image. It becomes easier to find the person to be monitored.

In another aspect, in the terminal device described above, the person position data is person area data relating to the area of the person including the position of the person in the image on the image, and the person position indicator is said. It is an image represented by the person area data, and the display processing unit displays the image in the person's area by a second display method different from the image display method. Preferably, in the terminal device described above, the second display method is a method of displaying an image in the area of the person in a mosaic. Preferably, in the terminal device described above, the image display method is a method of displaying the image in color or monochrome, and the second display method is a method of displaying an image in the area of the person in monochrome or color. How to display. Preferably, in the terminal device described above, the method of displaying the image is a method of displaying the image positively or negatively, and the second display method is a method of displaying the image in the area of the person negatively or positively. How to display.

In such a terminal device, the display processing unit displays an image in the area of a person by a second display method different from the image display method (first display method). Therefore, the user of the terminal device can easily find the area of the person in the image, in other words, the monitored person in the image.

In another aspect, in these terminal devices, the display processing unit has a central position of an image display area for displaying the image in the display unit and a position on the image of a person appearing in the image. The image and the person position sign are displayed on the display unit so that the images and the person position signs match each other.

In such a terminal device, the display processing unit displays a person position sign at the center position of the image display area. Therefore, the user of the terminal device only needs to pay attention to the central position of the image display area, so that it becomes easy to find the monitored person in the image.

In another aspect, the terminal device described above further includes an instruction input unit that receives an enlarged display instruction that is an instruction to enlarge and display a person appearing in the image, and the person position data is the person appearing in the image. The person area data relating to the area of the person including the position on the image, the person position indicator is an image represented by the person area data, and the display processing unit is the instruction input unit. When the enlarged display instruction is received, the area of the person is displayed in an enlarged size larger than the area of the person displayed on the display unit.

Such a terminal device further includes an instruction input unit, and when the display processing unit receives an enlarged display instruction at the instruction input unit, the terminal device is enlarged beyond the size of the area of the person displayed on the display unit. Display the area of the person. Therefore, since the area of the person is enlarged and displayed, the user of the terminal device can easily find the monitored person in the image.

In another aspect, the terminal device described above further includes a second instruction input unit that receives an operation instruction that is an instruction to perform a predetermined operation, and the second instruction input unit constitutes a touch panel together with the display unit. The person position data displayed by a button on the display unit is the person area data relating to the area of the person including the position of the person in the image on the image, and the person position indicator is the person. It is an image represented by area data, and the display processing unit displays the button in the remaining area excluding the area of the person in the display area of the display unit. Preferably, in the terminal device described above, the display processing unit displays the button in the remaining area excluding the area of the person in the image display area for displaying the image of the display unit.

In such a terminal device, the display processing unit displays a button for receiving an operation instruction in the remaining area excluding the area of the person. Therefore, the area of the person is not hidden by the button, and the user of the terminal device can easily find the person to be monitored in the image.

The display method of the terminal device according to the other aspect is the monitoring information communication that accommodates the image data of the image of the monitored person to be monitored and the person position data regarding the position of the person in the image on the image. The receiving process of receiving the signal and the image of the image data accommodated in the monitoring information communication signal received in the receiving process are displayed on the display unit to be displayed, and the person position data accommodated in the monitoring information communication signal is displayed. The present invention includes a display processing step of displaying a person position indicator indicating the position of the person based on the image in the image displayed on the display unit.

In such a display method of the terminal device, the image of the image data accommodated in the monitoring information communication signal received in the receiving process is displayed on the display unit, and the person based on the person position data accommodated in the monitoring information communication signal. The present invention includes a display control step of displaying a person position indicator indicating the position of the above in the image displayed on the display unit. Therefore, in the display method of the terminal device, the person position sign is displayed on the displayed image, so that the user of the terminal device can find the monitored person in the image by using the person position sign as a clue. , It becomes easier to find the monitored person in the image.

The sensor device according to the other aspect is a person position processing that generates person position data relating to a communication unit that performs communication, an image pickup unit that performs imaging, and a position of a person in the image captured by the image pickup unit. The communication unit includes an image data of an image captured by the imaging unit, and a monitoring processing unit that transmits a monitoring information communication signal accommodating the person position data generated by the person position processing unit.

Since such a sensor device accommodates not only the image but also the person position data in the monitoring information communication signal, the terminal device that has received the monitoring information communication signal can display the person position indicator on the image being displayed. Therefore, the user of the terminal device can find the monitored person in the image by using the person position sign as a clue, and it becomes easy to find the monitored person in the image.

The monitored person monitoring system according to the other aspect includes the above-mentioned sensor device and any of the above-mentioned terminal devices.

Since such a monitored person monitoring system includes the above-mentioned sensor device and any of the above-mentioned terminal devices, it is possible to easily find the monitored person in the image.

This application is based on Japanese Patent Application No. 2016-25856 filed on February 15, 2016, the contents of which are included in the present application.

In order to express the present invention, the present invention has been appropriately and sufficiently described through the embodiments with reference to the drawings above, but those skilled in the art can easily change and / or improve the above embodiments. It should be recognized that it can be done. Therefore, unless the modified or improved form implemented by a person skilled in the art is at a level that deviates from the scope of rights of the claims stated in the claims, the modified form or the improved form is the scope of rights of the claims. It is interpreted as being comprehensively included in.

MS Monitored person monitoring system SU, SU-1 to SU-4 Sensor device SV Management server device SP Fixed terminal device TA, TA-1, TA-2 Mobile terminal device 11 Imaging unit 12 Sensor side sound input / output unit (SU sound) Input / output section)
13 Nurse call reception operation unit 14 Sensor side control processing unit (SU control processing unit)
15 Sensor side communication interface unit (SU communication IF unit)
16 Sensor side storage unit (SU storage unit)
141 Sensor side control unit (SU control unit)
142 Sensor side monitoring processing unit (SU monitoring processing unit)
143 Person position processing unit 144 Nurse call processing unit 145 Sensor side streaming processing unit (SU streaming processing unit)
31 Terminal-side communication interface section (TA communication IF section)
32 Terminal side control processing unit (TA control processing unit)
33 Terminal side storage unit (TA storage unit)
34 Terminal side sound input / output unit (TA sound input / output unit)
35 Terminal side input section (TA input section)
36 Terminal side display (TA display)
37 Terminal side interface section (TAIF section)
321 Terminal side control unit (TA control unit)
322 Terminal side monitoring processing unit (TA monitoring processing unit)
323 Call processing unit 324 Terminal side streaming processing unit (TA streaming processing unit)
3221 Display processing unit

Claims (8)

In a detection system that detects a predetermined behavior of a monitored person based on image data of the monitored person to be monitored.
An acquisition unit that acquires image data of the monitored person to be monitored, and
Based on the acquired image data, a detection unit that detects a predetermined action in the monitored person, and a detection unit.
With respect to the monitored person whose predetermined action is detected by the detection unit, a generation unit that generates position data indicating a position corresponding to the monitored person based on the image data, and a generation unit.
A display unit that displays the image data and a monitored person position sign based on the position data, and
A display processing unit that controls the display unit is provided.
The display unit has an input area for performing a predetermined operation .
The display processing unit changes the display position so that the input area and the monitored person position sign do not overlap .
Detection system. The generation unit generates the position data in response to the detection of a predetermined action by the detection unit.
The detection system according to claim 1. The predetermined behavior detected by the detection unit includes at least one of getting up from the bedding, getting out of the bedding, falling from the bedding, and falling of the monitored person in the monitored person.
The detection system according to claim 1 or 2. The display unit further displays identification information for identifying a predetermined action detected by the detection unit.
The detection system according to any one of claims 1 to 3. The display unit displays image data indicating an area including the monitored person in the image data as the monitored person position indicator.
The detection system according to any one of claims 1 to 4. The display unit displays rectangular image data indicating an area including the monitored person as image data indicating the area including the monitored person in the image data.
The detection system according to claim 5. The display unit includes a mobile terminal device included in the detection system.
The detection system according to any one of claims 1 to 6. It is a display processing method in a detection system that detects a predetermined behavior of a monitored person based on image data obtained by photographing the monitored person to be monitored.
The step of acquiring the image data of the monitored person to be monitored in the acquisition unit, and
A step of detecting a predetermined action of the monitored person in the detection unit based on the acquired image data, and
With respect to the monitored person in which the predetermined behavior is detected, a step of generating position data indicating a position corresponding to the monitored person in the generation unit based on the image data, and a step of generating.
A step of displaying the image data and a monitored person position sign based on the position data on the display unit, and
A step for controlling the display unit is provided.
The display unit has an input area for performing a predetermined operation .
The step of controlling the display unit changes the display position so that the input area and the monitored person position sign do not overlap .
Display processing method in the detection system.

Images