JP2023000589A - Information processing system, information processor, control method, and control program - Google Patents

Abstract

To provide an information processing system, an information processor, a control method, and a control program capable of displaying a captured image of a care receiver while a state of each care receiver can be recognized.SOLUTION: An information processing system, which is a monitoring system 1, comprises: detection devices 10 each including a camera 104 for photographing a photographic image of a subject person, which is a care receiver 80, and a bed sensor 103 for detecting a state of the subject person; and a server including a monitor 204 for displaying a plurality of individual images respectively indicating the states of the plurality of subject persons on one first screen, and a control unit which, while maintaining the states of the plurality of subject persons included in the displayed first screen so as to be recognizable by the individual images. also causes the monitor 204 to display the photographic images.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system, an information processing device, a control method, and a control program.

Care recipients and care recipients (hereinafter referred to as "care recipients") who reside or are hospitalized in nursing care facilities such as welfare facilities for the elderly and hospitals may fall while walking within the nursing care facilities. or fall out of bed and get injured, or detect abnormalities in biometric information. Therefore, in order to enable nurses and caregivers to rush to the resident immediately when the resident is in such a state, development of a system for constantly detecting the state of the resident is underway.

Patent Literature 1 listed below discloses the following prior art. The biometric information received from a plurality of biometric information recording devices is displayed in each display frame, and the biometric information display frame determined to be abnormal is changed to an alert display frame with a changed color or background color. This makes it possible to immediately grasp the occurrence of an abnormal situation.

JP 2015-100568

However, in the prior art disclosed in Patent Document 1, if there is a problem with the measurement accuracy of the sensor that measures the biological information, the biological information that does not reflect the actual state of the user is displayed, and the biological information is abnormal. There is a problem that it may be falsely reported.

This kind of problem can be solved by installing a camera in the caregiver's living room, and by displaying the captured image of the caregiver in the event that an abnormality is detected along with the biological information measured in real time. It can be solved by grasping the state.

However, by displaying the image captured by the camera on the screen where the real-time biological information of each caregiver is displayed, the pop-up screen, etc. that displays the image captured by the camera will not display the biological information. It will be covered on the screen where you are. As a result, there is a problem that a part of the biometric information of the care recipient other than the care recipient whose photographed image is displayed cannot be seen. This problem becomes more conspicuous by displaying a plurality of pop-up screens or the like for displaying captured images.

The present invention has been made to solve such problems. That is, it is an object of the present invention to provide an information processing system, an information processing apparatus, a control method, and a control program capable of displaying a photographed image of a care recipient while the state of each care recipient can be recognized.

The above problems of the present invention are solved by the following means.

(1) Displaying a photographing unit for photographing a photographed image of a subject, a detecting unit for detecting the state of the subject, and a plurality of individual images respectively showing the states of the plurality of subjects on a single first screen. and a display unit for further displaying the photographed images on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images. An information processing system comprising a control unit.

(2) The control unit performs first display control for displaying the captured image as a second screen outside the first screen displayed on the display unit, and A second display control for displaying the individual image in a screen so as not to overlap the individual image, and a third display control for incorporating the captured image into a part of the individual image showing the state of the subject photographed in the captured image. The information processing system according to (1) above, wherein the display unit is caused to display by any of display control.

(3) a receiving unit that receives a change instruction for changing at least one of the display size and resolution of the photographed image for each subject, and the control unit controls the photographed image based on the change instruction; The information processing system according to the above (1) or (2), wherein at least one of the display size and resolution of is changed for each subject and displayed on the display unit.

(4) A reception unit that receives an enlargement instruction for designating a place to be enlarged in the displayed captured image, and the control unit displays the displayed captured image based on the enlargement instruction. The information processing system according to (1) or (2) above, wherein the photographed image is displayed on the display unit by enlarging the specified location without changing the size.

(5) A sound storage unit for storing sounds, the imaging unit has a sound output unit for outputting the stored sounds, and the control unit outputs the stored sounds when a predetermined condition is satisfied. The information processing system according to any one of (1) to (4) above, wherein the sound is output from the sound output unit.

(6) The above ( The information processing system according to any one of 1) to (4).

(7) detecting disconnection of communication between a transmission unit communicatively connected to a storage device and transmitting the captured image to the storage device in order to store the captured image, and the storage device and the transmission unit; a disconnection detection unit, wherein the photographing unit has a preliminary storage unit for storing the captured image; and the control unit stores the captured image in the preliminary storage unit when the disconnection of communication is detected. The information processing system according to any one of the above (1) to (5), wherein the information is stored in the .

(8) an acquisition unit that acquires a photographed image of a subject and the state of the subject; and a display unit that displays a plurality of individual images respectively showing the states of the plurality of subjects on one first screen. a control unit for further displaying the photographed images on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images; Information processing device having

(9) The control unit performs first display control for displaying the captured image as a second screen outside the first screen displayed on the display unit, and A second display control for displaying the individual image in a screen so as not to overlap the individual image, and a third display control for incorporating the captured image into a part of the individual image showing the state of the subject photographed in the captured image. The information processing apparatus according to (8) above, wherein the display section is caused to display by any one of display control.

(10) A reception unit that receives a change instruction for changing at least one of the display size and resolution of the photographed image for each subject, and the control unit controls the photographed image based on the change instruction. The information processing apparatus according to (8) or (9) above, wherein at least one of the display size and resolution of is changed for each target person and displayed on the display unit.

(11) A reception unit that receives an enlargement instruction for designating a place to be enlarged in the displayed captured image, and the control unit displays the displayed captured image based on the enlargement instruction. The information processing apparatus according to (8) or (9) above, wherein the photographed image is displayed on the display unit by enlarging the specified location without changing the size.

(12) A step (a) of acquiring a photographed image of a subject and a state of the subject, and displaying a plurality of individual images respectively showing the states of the plurality of subjects on a single first screen on a display unit; a step (b) of displaying the photographed image on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images; and a step (c) of further displaying the .

(13) In step (c), first display control for displaying the photographed image as a second screen outside the first screen displayed on the display section; A second display control for displaying the individual image in the first screen so as not to overlap the individual image, and displaying the photographed image by incorporating it into a part of the individual image showing the state of the subject photographed in the photographed image. The control method according to (12) above, wherein the display section is caused to display by any one of the third display controls.

(14) Step (d) of receiving a change instruction for changing at least one of the display size and resolution of the photographed image for each subject; and based on the change instruction, the display size and resolution of the photographed image. The control method according to (12) or (13) above, comprising a step (e) of changing at least one of the above for each subject and causing the display unit to display the change.

(15) Step (f) of receiving an enlargement instruction for designating a place to be enlarged in the displayed captured image; and without changing the display size of the displayed captured image based on the enlargement instruction and a step (g) of enlarging the specified location and displaying the photographed image on the display unit.

(16) A control program for causing a computer to execute the control method according to any one of (12) to (15) above.

A plurality of individual images respectively showing the states of a plurality of subjects are displayed on the display unit as one first screen, and the states of the plurality of subjects included in the displayed first screen can be recognized from the individual images. , the photographed image is further displayed on the display unit. As a result, it is possible to display the photographed images of the care recipients in a state in which the condition of each care recipient can be recognized.

It is a figure showing the whole surveillance system composition. 1 is a diagram showing the inside of a cared person's room in which a detection device is arranged; FIG. It is a block diagram which shows the hardware constitutions of a detection apparatus. It is a block diagram which shows the hardware constitutions of a server. It is a figure which shows a management table. It is a figure which shows an event management table. It is a figure which shows a monitor screen. FIG. 10 is a diagram showing an example of a state in which a captured image pop-up screen is displayed on a monitor together with a monitor screen; FIG. 10 is a diagram showing another example of a state in which the captured image pop-up screen is displayed on the monitor together with the monitor screen; FIG. 10 is a diagram showing still another example of a state in which the captured image pop-up screen is displayed on the monitor together with the monitor screen; It is a figure which shows an expansion place designation|designated screen. It is a block diagram which shows the hardware constitutions of a portable terminal. It is a figure which shows the example of a terminal monitor screen. FIG. 10 is a diagram showing an example of a state in which a captured image pop-up screen is displayed on the input display unit together with the terminal monitor screen; 4 is a flow chart showing the operation of the server; 4 is a flow chart showing the operation of the mobile terminal;

Hereinafter, an information processing system, an information processing device, a control method, and a control program according to embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation, and may differ from the actual ratios.

FIG. 1 is a diagram showing the overall configuration of a monitoring system 1 according to an embodiment. The monitoring system 1 constitutes an information processing system. A monitoring system 1 used in a nursing care facility or the like includes a detection device 10 , a server 20 , a plurality of mobile terminals 30 and a wireless AP (access point) 31 . The detection device 10, the server 20, and the mobile terminal 30 are connected to communicate with each other via a LAN (Local Area Network) and a wireless AP31. The portable terminal 30 is carried by a nursing caregiver 70 who nurses or cares for a nursing care recipient 80 . The detection device 10 constitutes a detection unit alone or together with the control unit 201 (see FIG. 4) of the server 20 . The server 20 and the mobile terminal 30 each constitute an information processing device. A cared person 80 constitutes a subject. In the following description, the subject is assumed to be the care recipient 80, but the subject also includes clinical trial participants.

(Detection device 10)
FIG. 2 is a diagram showing the inside of a room (hereinafter also referred to as a “living room”) of a care recipient 80 in which the detection device 10 is arranged. FIG. 3 is a block diagram showing the hardware configuration of the detection device 10. As shown in FIG. In the example of FIG. 3, detection device 10 includes bed sensor 103 and camera 104, but bed sensor 103 and camera 104 may each be separate devices. In this case, bed sensor 103 and camera 104 may each have a control unit and a communication unit. A camera 104 constitutes an imaging unit.

As shown in FIG. 2 , the bed sensors 103 included in the detection device 10 are arranged on the bed 90 in the living room of the cared person 80 . Specifically, the bed sensor 103 is placed under the mattress on the bed 90, for example. The camera 104 is placed, for example, on the ceiling of the living room or on the upper part of the wall, and photographs the bed 90 of the cared person 80 below as a monitoring area.

As shown in FIG. 3, the detection device 10 has a control section 101, a communication section 102, a bed sensor 103, and a camera 104, which are interconnected by a bus. The control unit 101 constitutes a sound storage unit and a disconnection detection unit. The communication unit 102 constitutes a transmission unit.

The control unit 101 includes a CPU, RAM, ROM, and the like. The control unit 101 may have an HDD or an SSD. The control unit 101 controls each unit of the detection device 10 and performs arithmetic processing according to a program. The action of the control unit 101 will be described later.

The communication unit 102 is an interface circuit (for example, a LAN card, a wireless communication circuit, etc.) for communicating with, for example, the server 20 via a LAN.

The bed sensor 103 detects body movements of the cared person 80 on the bed 90 . Thus, the bed sensor 103 detects sleep, waking, getting up, getting out of bed, being in bed, breathing rate, heart rate, etc. of the care recipient 80 as the condition of the care recipient 80 . The bed sensor 103 can be composed of a sheet-like bag containing air and a piezoelectric sensor that detects variations in the air pressure.

The camera 104 captures an image of the bed 90 of the care recipient 80 as a monitoring area and outputs a captured image (image data). The captured images include still images and moving images. A moving image may include a plurality of still images captured intermittently in time series. The camera 104 is a near-infrared camera, but a visible light camera may be used instead, or both of them may be used.

Camera 104 can include sound input/output unit 1041 and spare storage unit 1042 . The sound input/output unit 1041 is composed of, for example, a microphone and a speaker. The sound input/output unit 1041, for example, collects the sound in the living room with a microphone. The sound input/output unit 1041, for example, outputs sound into the living room through a speaker. The sound input/output unit 1041 constitutes a sound output unit and a sound collection unit. A preliminary storage unit 1042 stores a captured image in a predetermined case to be described later. The spare storage unit 1042 can store sound data.

The action of the control unit 101 will be described.

The control unit 101 transmits the detection result of the state of the cared person 80 detected by the bed sensor 103 of the detection device 10 (hereinafter simply referred to as "detection result") as data to the server 20 via the communication unit 102. Send. The communication unit 102 transmits a captured image captured by the camera 104 (hereinafter also simply referred to as a “captured image”) to the server 20 as data.

Based on the captured image, the control unit 101 extracts the silhouette of the care recipient 80 using a known time subtraction method or background subtraction method, and extracts the silhouette of the care recipient 80 using a known technique such as machine learning. By estimating movement, the behavior of the care recipient 80 may be detected.

The control unit 101 cooperates with the control unit 201 (see FIG. 4) of the server 20, and based on an enlargement instruction (described later) input by the user to the operation unit 205 (see FIG. 4) of the server 20, a specified The camera 104 may be controlled to capture a captured image in which the location is zoomed up.

The control unit 101 cooperates with the control unit 201 of the server 20, or independently, when a predetermined condition is satisfied, transfers the sound stored in the storage unit 203 or the preliminary storage unit 1042 of the server 20 to the sound input/output unit. Output from 1041. Predetermined conditions include, for example, that it is time to eat, that an event (to be described later) is detected, and that a predetermined abnormality is detected. The predetermined abnormality includes, for example, the fact that the sound input/output unit 1041 collects a sound having a loudness exceeding a predetermined threshold value due to the falling of an object, etc., and the abnormal behavior such as rampage based on the captured image. It includes things that have been detected. For example, when it is time to eat, the control unit 101 can cause the sound input/output unit 1041 to output a voice such as "It's time to eat." The control unit 101 receives, for example, from the server 20 via the communication unit 102 that a “wake up” event has been detected, or when an object falls over, the sound input/output unit 1041 outputs a signal that exceeds a predetermined threshold value. For example, when a low sound is collected, the sound input/output unit 1041 may output a voice such as “Please wait for a while”.

The control unit 101 stores the sound collected by the sound input/output unit 1041 in the preliminary storage unit 1042, or associates the data of the sound with the sensor ID or the care recipient 80 described later, and sends the data to the server 20. It can be transmitted and stored in the storage unit 203 of the server. The stored sound data can be used to detect abnormal behavior of the cared person 80, such as the cared person 80 shouting in the middle of the night.

The control unit 101 can communicably connect the communication unit 102 and an external storage device that is not included in the monitoring system 1, and transmit and store the captured image to the storage device. A storage device is, for example, a cloud server. The storage device can be replaced by the storage unit 203 of the server. In a predetermined case, the control unit 101 can store the captured image in the preliminary storage unit 1042 instead of the storage device. Predetermined cases include, for example, a case where the control unit 101 detects a disconnection of communication due to a partial power failure or the like between the communication unit 102 and the storage device.

(Server 20)
FIG. 4 is a block diagram showing the hardware configuration of the server 20. As shown in FIG. The server 20 has a control section 201, a communication section 202, a storage section 203, a monitor 204, and an operation section 205, which are interconnected by a bus. The communication unit 202 constitutes an acquisition unit. A monitor 204 constitutes a display unit and a reception unit.

The control unit 201 includes a CPU, RAM, ROM, and the like. The control unit 201 controls each unit of the server 20 and performs arithmetic processing according to a program. The action of the control unit 201 will be described later.

The communication unit 202 is configured by an interface circuit for communicating with, for example, the mobile terminal 30 or the like via a LAN. The communication unit 202 receives detection results and captured images from the detection device 10 .

Storage unit 203 is configured by an HDD or SSD, and stores various programs and various data.

The monitor 204 is configured, for example, together with the operation unit 205 by a touch panel in which a touch sensor is superimposed on a display surface such as liquid crystal. The monitor 204 displays a monitor screen 2041 (see FIG. 7) showing the condition of the cared person 80 in each living room. The condition of the cared person 80 includes, for example, detection results of sleeping, waking, getting up, getting out of bed, being in bed, breathing rate, and heart rate of the cared person 80 . The status of the cared person 80 may include information indicating an event, which will be described later. As will be described later, the monitor 204 can display a captured image while the monitor screen 2041 is being displayed.

The operation unit 205 is configured by, for example, a touch panel as described above, and receives various operation inputs by the user.

The action of the control unit 201 will be described.

FIG. 5 is a diagram showing the management table 2031. As shown in FIG. The control unit 201 identifies a unique sensor ID that identifies each detection device 10, an installation location that is the room number in which each detection device 10 is installed, and the care recipient 80 living in the installation location. is stored in the storage unit 203 . The management table 2031 may further include a unique camera ID that identifies the camera 40 installed at the installation location in association with it. The management table 2031 can be referred to when the monitor screen 2041 or the like is created.

In the example of FIG. 5, for example, the sensor ID of SU-1, room No. 101, which is the place of installation, and Mr. A, who is the care recipient 80, are associated.

The control unit 201 determines abnormal breathing and abnormal heartbeat based on the detection results received from the detection device 10 . Specifically, the control unit 201 can determine that the respiratory rate is abnormal when the respiratory rate is outside the predetermined range. If the heart rate is outside the predetermined range, the control unit 201 can determine that the heart rate is abnormal. Based on the detection results, the control unit 201 can detect waking up, getting up, getting out of bed, abnormal breathing, and abnormal heartbeat as events. An event is a change in the state of the care recipient 80 , which requires the care of the caregiver 70 and should be notified to the caregiver 70 . Changes in the state of the cared person 80 include waking up, getting up, getting out of bed, abnormal breathing, abnormal heartbeat, care calls, and the like. Note that detection of events based on detection results may be performed by the detection device 10 . In this case, the event detection result is transmitted from the detection device 10 to the server 20 . The event detection result is information about the event that occurred, and constitutes event occurrence information.

The control unit 201 can store the captured image in the storage unit 203 . The control unit 201 may cause the storage unit 203 to store all of the captured images. The control unit 201 may temporarily store the captured images in the ring buffer and rewrite the captured images with the new captured images in order from the oldest captured images.

FIG. 6 is a diagram showing the event management table 2032. As shown in FIG. The control unit 201 controls the sensor ID, the detection result of the event, the detection time of the event, the location in the storage unit 203 where the still image when the event was detected, and the moving image including when the event was detected. is stored in the storage unit 203 as an event management table. The event management table may further include an associated event ID consisting of unique alphanumeric characters identifying the event. The moving image including the time when the event is detected can be a captured image which the control unit 201 reads the captured images stored in the ring buffer retroactively from the time when the event is detected and stored in the storage unit 203. The time to look back can be any time between 10 and 30 seconds, for example. The retroactive time may be set differently for each care recipient.

FIG. 7 is a diagram showing the monitor screen 2041. As shown in FIG. Based on the detection results received from the detection device 10 , the control unit 201 creates a monitor screen 2041 showing the condition of the cared person 80 in each living room and displays it on the monitor 204 . A monitor screen 2041 displays a plurality of individual images 2042 each showing the condition of the cared person 80 on one screen. On the monitor screen 2041, the state of the cared person 80 can be updated and displayed at predetermined intervals. A monitor screen 2041 constitutes the first screen.

In the example of FIG. 7, sleep, awakening, getting up, getting out of bed, breathing rate, and heart rate are displayed in characters (text) as the states of the cared person 80 to be cared for. Instead of characters, a simple illustration (icon), an image (for example, a still image or a live image) of the care recipient 80 taken by the camera 40, and a state of the care recipient 80 such as "sleep" are shown. Combinations with letters and the like may be displayed. For example, the state of the cared person 80 in room 104 is shown to be asleep, with a respiratory rate of 11 and a heart rate of 67. Information indicating an event is indicated by a background pattern of pin dots. In FIG. 7, waking up (room 102, etc.) and getting up (room 202, etc.) are shown as events.

On the monitor screen 2041, the characters "event notification" can be displayed for the care-receiver 80 who is the target of event notification due to the event notification settings (or changes in the event notification settings). For the cared-for person 80 who is not subject to event notification, a hatched character "EVENT NOTIFICATION" may be displayed.

As will be described later, a terminal monitor screen 3041 (see FIG. 13) corresponding to the monitor screen 2041 can be displayed on each mobile terminal 30. FIG.

When the control unit 201 detects an event, the control unit 201 displays information indicating the event as a pop-up screen displaying a captured image (hereinafter referred to as a “captured image pop-up screen 2044 (see FIG. 8)”) on the monitor. It can be displayed on the monitor 204 together with the screen 2041 . A captured image pop-up screen 2044 constitutes the second screen. The control unit 201 also displays the captured image pop-up screen 2044 when a captured image display request is input by selecting (for example, touching) the camera icon 2043 included in each individual image 2042 . It can be further displayed on the monitor 204 together with the monitor screen 2041 displayed.

The control unit 201 maintains the photographed image pop-up screen 2044 so that the individual images 2042 of the plurality of cared persons 80 included in the monitor screen 2041 displayed on the monitor 204 can be recognized, and further displays the images on the monitor 204. display. That is, the control unit 201 controls the sleep, awakening, rising, getting out of bed, breathing rate, and heart rate of all the care recipients 80 included in the monitor screen 2041, and information identifying the care recipients 80 (for example, , room number, and name) are displayed on the monitor 204 so that the display of the captured image pop-up screen 2044 is not obstructed. Note that being able to recognize the condition of the care recipient 80 includes being able to recognize the condition of the care recipient 80 by scrolling the screen on the monitor 204 or the like.

FIG. 8 is a diagram showing an example of a state in which the captured image pop-up screen 2044 is displayed on the monitor 204 together with the monitor screen 2041. As shown in FIG. FIG. 8 shows a photographed image pop-up screen 2044 displayed when the camera icon 2043 is selected in the individual image 2042 showing the state of Mr. A in room 101. As shown in FIG.

In the example of FIG. 8, the monitor 204 is controlled (first control) to display the captured image pop-up screen 2044 outside the monitor screen 2041 displayed on the monitor 204, and the captured image pop-up screen 2044 is displayed. It is The photographed image pop-up screen 2044 is displayed on the monitor 204 so that the states of all cared persons 80 can be recognized by displaying the photographed image pop-up screen 2044 outside the monitor screen 2041 .

FIG. 9 is a diagram showing another example of the state in which the captured image pop-up screen 2044 is displayed on the monitor 204 together with the monitor screen 2041. As shown in FIG. As in the example of FIG. 8, FIG. 9 shows a captured image pop-up screen 2044 displayed when the camera icon 2043 is selected in the individual image 2042 showing the state of Mr. A in room 101. As shown in FIG.

In the example of FIG. 9, the monitor 204 is controlled (second control) so that the captured image pop-up screen 2044 does not overlap the individual image 2042 in the monitor screen 2041 displayed on the monitor 204, and the captured image pop-up screen is displayed. A screen 2044 is displayed. The photographed image pop-up screen 2044 is displayed on the monitor 204 so that the photographed image pop-up screen 2044 does not overlap with any of the individual images 2042 so that the states of all the care recipients 80 can be recognized. .

FIG. 10 is a diagram showing still another example of the state in which the captured image pop-up screen 2044 is displayed on the monitor 204 along with the monitor screen 2041. As shown in FIG. As in the example of FIG. 8, FIG. 10 shows a captured image pop-up screen 2044 displayed when the camera icon 2043 is selected in the individual image 2042 showing the state of Mr. A in room 101. As shown in FIG.

In the example of FIG. 10 , a photographed image pop-up screen 2044 is displayed on the monitor screen 2041 displayed on the monitor 204 . The monitor 204 is controlled (third control) so as to incorporate and display the image 2042 . By such control, a captured image pop-up screen 2044 is displayed. The photographed image pop-up screen 2044 is incorporated in the individual image 2042 of the care recipient 80 included in the photographed image of the photographed image pop-up screen 2044 and displayed so that the states of all the care recipients 80 can be recognized. , a captured image pop-up screen 2044 is displayed on the monitor 204 .

Control unit 201 displays photographed image pop-up screen 2044 based on a change instruction for changing at least one of the display size and resolution of the photographed image for each care recipient 80, which is input by the user through operation unit 205. At least one of the size (that is, the display size of the captured image) and the resolution of the captured image displayed on the captured image pop-up screen 2044 is changed for each cared person 80 and displayed on the monitor 204 . The instruction to change the display size of the captured image is, for example, by selecting the "larger size" button 2045 displayed on the captured image pop-up screen 2044 and inputting the changed magnification in a predetermined input field (not shown). can be done with The instruction to change the display size of the captured image may include the post-change magnification (for example, 150%). An instruction to change the resolution of the captured image can be given by selecting a “high image quality” button 2046 displayed on the captured image pop-up screen 2044, for example. When an instruction to change the display size of the captured image is received from the operation unit 205, the control unit 201 controls the monitor 204 based on the change instruction to increase the display size of the captured image pop-up screen 2044 (or smaller). When receiving an instruction to change the display size of the captured image, the control unit 201 may change the display size of the captured image pop-up screen 2044 at a fixed magnification. When increasing the display size of the captured image pop-up screen 2044, the control unit 201 enlarges the captured image pop-up screen 2044 to display the states of all the care-receivers 80 included in the monitor screen 2041. display of the monitor screen 2041 and the captured image pop-up screen 2044 by the monitor 204 is controlled so as not to be disturbed by the When an instruction to change the resolution of the captured image is received from the operation unit 205, the control unit 201 normally lowers the resolution of the captured image received from the detection device 10 and changes the captured image displayed on the captured image pop-up screen 2044. can be changed so as to be displayed on the captured image pop-up screen 2044 without lowering the resolution of the captured image received from the detection device 10 . The control unit 201 may cooperate with the control unit 101 of the detection device 10 to increase the resolution of the captured image displayed on the captured image pop-up screen 2044 by performing control to increase the resolution of the camera 104 . .

The control unit 201 adjusts the display size of the displayed captured image pop-up screen 2044 (that is, the size of the captured image) based on an enlargement instruction for designating a location to be enlarged in the captured image input by the user through the operation unit 205. A photographed image pop-up screen 2044 is displayed on the monitor 204 by enlarging the specified location on the photographed image without changing the display size). The enlargement instruction is, for example, by selecting the "enlarge" button (not shown) displayed on the captured image pop-up screen 2044 to display the enlargement location designation screen 2047 (see FIG. 11), and then performing the photographing on the enlargement location designation screen 2047. This can be done by specifying the location in the image to be magnified by selection.

FIG. 11 is a diagram showing an enlarged location designation screen 2047. As shown in FIG. As shown in the example of FIG. 11, the enlarged location designation screen 2047 can be displayed on the captured image pop-up screen 2044 as a screen in which sections 1 to 32 are selectably displayed. The user can input an enlargement instruction to the operation unit 205 for specifying a place to be enlarged in the captured image by selecting (touching) any section on the enlargement location designation screen 2047 .

Based on the enlargement instruction, the control unit 201 can use the captured image received from the detection device 10 to create an enlarged image of the designated location on the captured image, and display the image on the captured image pop-up screen 2044 .

(Mobile terminal 30)
FIG. 12 is a block diagram showing the hardware configuration of the mobile terminal 30. As shown in FIG. The mobile terminal 30 has a control section 301, a wireless communication section 302, a storage section 303, an input display section 304, and an audio input/output section 305, which are interconnected by a bus. The mobile terminal 30 may be composed of a portable communication terminal device such as a tablet computer, smart phone, or mobile phone. The wireless communication unit 302 constitutes an acquisition unit. The input display unit 304 constitutes a reception unit.

The control unit 301 includes a CPU, RAM, ROM, and the like. The control unit 301 controls each unit of the mobile terminal 30 and performs arithmetic processing according to a program. The action of the control unit 301 will be described later.

The wireless communication unit 302 wirelessly communicates with each device via the wireless AP 31 or directly using wireless communication using standards such as Wi-Fi and Bluetooth (registered trademark).

A storage unit 303 is configured by an SSD or an SD card, and stores various programs and various data.

The input display unit 304 is configured by, for example, a touch panel in which a touch sensor is superimposed on a display surface such as liquid crystal, and displays various data and receives various inputs.

The voice input/output unit 305 is, for example, a speaker and a microphone, and enables voice communication by the caregiver 70 with another mobile terminal 30 via the wireless communication unit 302 .

The action of the control unit 301 will be described.

The control unit 301 receives the result of detection by the detection device 10 and the captured image from the server 2- through the wireless communication unit 302. FIG.

Based on the detection results, the control unit 301 creates a terminal monitor screen 3041 showing the state of the cared person 80 in each living room, and displays it on the input display unit 304 . A terminal monitor screen 3041 constitutes the first screen. A terminal monitor screen 3041 is a screen corresponding to the monitor screen 2041 displayed on the monitor 204 of the server 20 . Therefore, the state of the cared person 80 and the like displayed on the terminal monitor screen 3041 are the same as those displayed on the monitor screen 2041, and redundant descriptions will be omitted or simplified.

FIG. 13 is a diagram showing an example of the terminal monitor screen 3041. As shown in FIG. The terminal monitor screen 3041 displays a plurality of individual images 3042 each showing the condition of the cared person 80 on one screen. On the terminal monitor screen 3041, the status of the cared person 80 can be updated and displayed at predetermined intervals. In the terminal monitor screen 3041, display of the respiratory rate, heart rate, and event notification settings may be omitted.

In the example of FIG. 13, each individual image 3042 includes a camera icon 3043 . When a captured image display request is input by selecting (for example, touching) the camera icon 3043 , the control unit 301 displays the captured image pop-up screen 3044 on the input display unit 304 . It can also be displayed with the terminal monitor screen 3041 that is present.

FIG. 14 is a diagram showing an example of a state in which the captured image popup screen 3044 is displayed on the input display unit 304 together with the terminal monitor screen 3041. As shown in FIG. FIG. 14 shows a photographed image pop-up screen 3044 displayed when the camera icon 3043 is selected in the individual image 3042 showing the state of Mr. A in the 101st room.

In the example of FIG. 14, the input display unit 304 is controlled (first control) to display the captured image popup screen 3044 outside the terminal monitor screen 3041 displayed on the monitor 204, and the captured image popup screen is displayed. 3044 is displayed. A captured image pop-up screen 3044 constitutes the second screen. The captured image pop-up screen 3044 is displayed on the input display unit 304 so that the states of all the care recipients 80 can be recognized by displaying the captured image pop-up screen 3044 outside the monitor screen 2041 .

Note that the captured image pop-up screen 3044 may be displayed so as not to overlap the individual image 3042 in the displayed terminal monitor screen 3041 in the same manner as the captured image pop-up screen 2044 displayed on the monitor 204 of the server 20. good. The photographed image pop-up screen 3044 may be displayed in the displayed terminal monitor screen 3041 by incorporating it into the individual image 3042 showing the state of the cared person 80 photographed in the photographed image of the photographed image pop-up screen 3044. good.

The control unit 301 receives an event detection result (event notification) from the server 20 when an event is detected by the server 20 . The control unit 301 can further display the captured image pop-up screen 3044 on the input display unit 304 together with the displayed terminal monitor screen 3041 by receiving the event detection result from the server 20 .

The control unit 301 displays the captured image pop-up screen 3044 based on the change instruction for changing at least one of the display size and resolution of the captured image for each care recipient 80 input by the user in the input display unit 304. At least one of the display size (that is, the display size of the captured image) and the resolution of the captured image displayed on the captured image pop-up screen 3044 is changed for each cared person 80 and displayed on the input display unit 304 . When the input display unit 304 receives an instruction to change the display size of the captured image, the control unit 301 controls the input display unit 304 based on the change instruction to change the display size of the captured image popup screen 3044. Make it bigger (or smaller). When receiving an instruction to change the display size of the captured image, the control unit 301 may change the display size of the captured image pop-up screen 3044 at a fixed magnification. When increasing the display size of the captured image pop-up screen 3044, the control unit 301 enlarges the captured image pop-up screen 3044 to display the states of all the caregivers 80 included in the terminal monitor screen 3041. The display of the terminal monitor screen 3041 and the captured image pop-up screen 3044 by the input display unit 304 is controlled so as not to be hindered by this. When the input display unit 304 receives an instruction to change the resolution of the captured image, the control unit 301 normally lowers the resolution of the captured image received from the server 20 and displays the captured image on the captured image pop-up screen 3044. can be changed so that the captured image received from the server 20 is displayed on the captured image pop-up screen 3044 without lowering the resolution of the captured image. The control unit 301 cooperates with the control unit 201 of the server 20 and the control unit 101 of the detection device 10 to increase the resolution of the camera 104, thereby increasing the resolution of the captured image displayed on the captured image pop-up screen 3044. may be changed to increase

The control unit 301 adjusts the display size of the displayed captured image pop-up screen 3044 (that is, the captured image (display size)), and displays on the monitor 204 a photographed image pop-up screen 3044 in which the specified location on the photographed image is enlarged. The enlargement instruction is, for example, by selecting an “enlarge” button (not shown) displayed on the captured image pop-up screen 3044 to display an enlargement location designation screen 2047 (see FIG. 11), and performing a photographing operation on the enlargement location designation screen 2047. This can be done by designating a place in the image to be magnified.

Based on the enlargement instruction, the control unit 301 can use the captured image received from the server 20 to create an enlarged image of the specified location on the captured image and display it on the captured image pop-up screen 3044 .

Operation of the server 20 will be described.

FIG. 15 is a flow chart showing the operation of the server 20. As shown in FIG. This flowchart can be executed by the control unit 201 of the server 20 according to a program.

The control unit 201 acquires the detection result and the captured image from the detection device 10 by receiving them through the communication unit 202 (S101).

The control unit 201 creates a monitor screen 2041 based on the obtained detection result and displays it on the monitor 204 (S102).

The control unit 201 detects an event based on the detection result, or determines whether a request to display a captured image has been input through the operation unit 205 (S103). If the control unit 201 does not detect an event and determines that a request to display a captured image has not been input (S103: NO), it executes step S105.

When the control unit 201 detects an event or determines that a request to display a captured image has been input (S103: YES), the captured image pop-up screen 2044 is displayed outside the monitor screen 2041 being displayed. (S104).

The control unit 201 determines whether an instruction to change the display size or resolution of the captured image displayed as the captured image pop-up screen 2044 has been input to the operation unit 205 (S105). If the control unit 201 determines that an instruction to change the display size and resolution of the captured image has not been input (S105: NO), it executes step S107.

If the control unit 201 determines that an instruction to change the display size or resolution of the captured image has been input (S105: YES), the control unit 201 changes the display size or resolution of the captured image pop-up screen 2044 based on the change instruction. (S106).

The control unit 201 determines whether or not an enlargement instruction for designating a location to be enlarged in the captured image displayed as the captured image pop-up screen 2044 has been input to the operation unit 205 (S107). If the control unit 201 determines that an enlargement instruction has not been input (S107: NO), the process ends.

If the controller 201 determines that an enlargement instruction has been input (S107: YES), it enlarges the specified location on the captured image pop-up screen 2044 based on the enlargement instruction (S108).

FIG. 16 is a flow chart showing the operation of the mobile terminal 30. As shown in FIG. This flowchart can be executed by the control unit 301 of the mobile terminal 30 according to the program.

The control unit 301 acquires the detection result and the captured image from the server 20 by receiving them through the wireless communication unit 302 (S201).

Based on the obtained detection result, the control unit 301 creates a terminal monitor screen 3041 and displays it on the input display unit 304 (S202).

The control unit 301 receives the event detection result from the server 20 via the wireless communication unit 302, or determines whether a display request for a captured image has been input through the input display unit 304 (S203). When the control unit 301 determines that the event detection result has not been received and the request to display the captured image has not been input (S203: NO), the control unit 301 executes step S205.

When the control unit 301 receives the event detection result or determines that a request to display a captured image has been input (S203: YES), the control unit 301 displays the captured image pop-up screen 3044 on the displayed terminal monitor screen 3041. is displayed outside (S204).

The control unit 301 determines whether an instruction to change the display size or resolution of the captured image displayed as the captured image pop-up screen 3044 has been input to the input display unit 304 (S205). If the control unit 301 determines that an instruction to change the display size and resolution of the captured image has not been input (S205: NO), it executes step S207.

If the control unit 301 determines that an instruction to change the display size or resolution of the captured image has been input (S205: YES), the control unit 301 changes the display size or resolution of the captured image popup screen 3044 based on the change instruction. (S206).

The control unit 301 determines whether or not an enlargement instruction for designating a location to be enlarged in the captured image displayed as the captured image pop-up screen 3044 has been input to the input display unit 304 (S207). If the control unit 301 determines that an enlargement instruction has not been input (S207: NO), the process ends.

If the controller 301 determines that an enlargement instruction has been input (S207: YES), it enlarges the specified location on the captured image popup screen 3044 based on the enlargement instruction (S208).

This embodiment has the following effects.

A plurality of individual images respectively showing the states of a plurality of subjects are displayed on the display unit as one first screen, and the states of the plurality of subjects included in the displayed first screen can be recognized from the individual images. , the photographed image is further displayed on the display unit. As a result, it is possible to display the photographed images of the care recipients in a state in which the condition of each care recipient can be recognized.

A first display control for displaying the photographed image as a second screen outside the first screen displayed on the display unit, and a second display control for displaying the captured image so as not to overlap the individual image within the first screen displayed on the display unit. 2 display control, or 3rd display control in which the photographed image is incorporated into a part of the individual image showing the state of the subject photographed in the photographed image and displayed on the display unit. As a result, it is possible to more easily display the photographed images of the care recipients while maintaining the state of each of the care recipients to be recognized.

Furthermore, a change instruction for changing at least one of the display size and resolution of the captured image for each subject is accepted. Then, based on the change instruction, at least one of the display size and resolution of the photographed image is changed for each subject and displayed on the display unit. Alternatively, an enlargement instruction for designating a location to be enlarged in the captured image is received, and based on the enlargement instruction, the specified location is enlarged without changing the display size of the displayed captured image and the captured image is displayed. displayed in the department. Depending on the subject's health condition, there are those who want to see the details of the subject and those who just want to see the overall overview. There is a problem that the target person cannot be monitored. A person who wants to see details includes a person who has a broken finger, a person who is partially paralyzed, and the like. As a target person who only needs to see the overall overview, a person who only needs to confirm whether or not he or she has left the bed can be considered. For this reason, it is possible to appropriately monitor the subject by enabling adjustment of the display size and resolution of the photographed image for each subject, or by enabling enlargement of a designated portion.

Furthermore, a sound output unit that stores sound and outputs the stored sound when a predetermined condition is satisfied is provided in the photographing unit. As a result, for example, the safety of the subject can be further ensured by outputting an appropriate pre-stored sound when the fall of the subject is detected.

Furthermore, a sound collecting unit for collecting sound is provided in the photographing unit, and the collected sound is stored. Thereby, a subject's abnormality can be detected based on a sound.

Further, in order to store the photographed image, the photographed image is transmitted to a communicably connected storage device, and disconnection of communication with the storage device is detected. The photographed image is stored in the provided preliminary storage section. As a result, captured images can be stored even when communication is cut off due to a power outage or the like caused by a disaster.

The configuration of the server 20 and the monitoring system 1 including the server 20 described above is the main configuration for describing the features of the above-described embodiment, and is not limited to the above-described configuration. , can be modified in various ways. For example, the bed sensor 103 transmits microwaves to a predetermined irradiation area and receives reflected waves to detect the Doppler shift of the microwaves caused by the body movement (for example, breathing movement) of the cared person 80. It may be replaced with a body motion sensor that detects as Further, the above-described flowchart may include steps other than the steps shown in the flowchart, or may not include some steps. Also, the order of steps is not limited to the embodiment described above. Furthermore, each step may be combined with other steps and executed as one step, may be executed while being included in other steps, or may be divided into a plurality of steps and executed.

Also, the means and methods for performing various processes in the above-described embodiments can be realized either by dedicated hardware circuits or programmed computers. The program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. Further, the program may be provided as independent application software, or may be incorporated as a function into the software of the device such as the detection unit.

1 surveillance system,
10 detection device,
101 control unit,
102 communication unit,
103 bed sensor,
104 camera,
1041 sound input/output unit,
1042 spare storage unit,
20 servers,
201 control unit,
202 communication unit;
203 storage unit,
2031 management table,
2032 event management table;
204 monitor,
2041 monitor screen,
2042 individual images,
2043 camera icon,
2044 captured image pop-up screen,
2047 Enlarged location specification screen,
30 mobile terminals,
301 control unit,
302 wireless communication unit,
303 storage unit,
304 input display unit,
3041 terminal monitor screen,
3042 individual images,
3043 camera icon,
3044 captured image pop-up screen,
305 audio input/output unit;

Claims (16)

a photographing unit for photographing a photographed image of a subject;
a detection unit that detects the state of the subject;
a display unit that displays a plurality of individual images respectively showing the states of the plurality of subjects on a single first screen;
a control unit for further displaying the photographed image on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images;
An information processing system having The control unit performs first display control for displaying the captured image as a second screen outside the first screen displayed on the display unit, and A second display control for displaying the individual image so as not to overlap, and a third display control for displaying the photographed image by incorporating it into a part of the individual image showing the state of the subject photographed in the photographed image. 2. The information processing system according to claim 1, wherein the display unit is caused to display by any one of them. a reception unit that receives a change instruction for changing at least one of the display size and resolution of the captured image for each subject;
3. The information processing system according to claim 1, wherein the control unit changes at least one of display size and resolution of the captured image for each subject based on the change instruction, and causes the display unit to display the captured image. . a reception unit that receives an enlargement instruction for designating a location to be enlarged in the displayed photographed image;
2. The control unit causes the display unit to display the photographed image by enlarging a designated place without changing the display size of the displayed photographed image based on the enlargement instruction. 3. The information processing system according to 2. Having a sound storage unit for storing sounds,
The imaging unit has a sound output unit that outputs the stored sound,
5. The information processing system according to claim 1, wherein said control unit causes said sound output unit to output said stored sound when a predetermined condition is satisfied. Having a sound storage unit for storing sounds,
The imaging unit has a sound collecting unit that collects sound,
The information processing system according to any one of claims 1 to 4, wherein the control unit stores the collected sounds in the sound storage unit. a transmission unit communicably connected to a storage device for transmitting the captured image to the storage device for storing the captured image;
a disconnection detection unit that detects disconnection of communication between the storage device and the transmission unit;
The photographing unit has a preliminary storage unit that stores the photographed image,
6. The information processing system according to any one of claims 1 to 5, wherein said control unit stores said captured image in said spare storage unit when disconnection of said communication is detected. an acquisition unit that acquires a captured image of a subject and the state of the subject;
a display unit that displays a plurality of individual images respectively showing the states of the plurality of subjects on a single first screen;
a control unit for further displaying the photographed image on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images;
Information processing device having The control unit performs first display control for displaying the captured image as a second screen outside the first screen displayed on the display unit, and A second display control for displaying the individual image so as not to overlap, and a third display control for displaying the photographed image by incorporating it into a part of the individual image showing the state of the subject photographed in the photographed image. 9. The information processing apparatus according to claim 8, wherein the display unit is caused to display by any one of them. a reception unit that receives a change instruction for changing at least one of the display size and resolution of the captured image for each subject;
10. The information processing according to claim 8, wherein the control unit changes at least one of a display size and resolution of the captured image for each target person based on the change instruction, and causes the display unit to display the captured image. Device. a reception unit that receives an enlargement instruction for designating a location to be enlarged in the displayed photographed image;
9. The control unit causes the display unit to display the photographed image by enlarging a specified location without changing the display size of the displayed photographed image based on the enlargement instruction. 10. The information processing device according to 9. a step (a) of obtaining a photographed image of a subject and a state of the subject;
a step (b) of displaying a plurality of individual images respectively showing the states of the plurality of subjects on a single first screen on a display unit;
a step (c) of further displaying the photographed image on the display unit while maintaining the states of the plurality of subjects included in the first screen displayed on the display unit so as to be recognizable by the individual images; ,
A control method with In the step (c), first display control for displaying the photographed image as a second screen outside the first screen displayed on the display unit, and the first screen displayed on the display unit A second display control for displaying the individual image so as not to overlap the individual image within the second display control, and a third display for displaying the photographed image by incorporating it into a part of the individual image showing the state of the subject photographed in the photographed image. 13. The control method according to claim 12, wherein the display unit is caused to display by either control. a step (d) of receiving a change instruction for changing at least one of the display size and resolution of the photographed image for each subject;
a step (e) of changing at least one of the display size and resolution of the photographed image for each subject based on the change instruction and displaying the photographed image on the display unit;
14. The control method according to claim 12 or 13, comprising: a step (f) of receiving an enlargement instruction for designating a place to be enlarged in the displayed photographed image;
a step (g) of enlarging a specified location and displaying the photographed image on the display unit without changing the display size of the photographed image being displayed, based on the enlargement instruction;
The control method according to any one of claims 12 to 14, having A control program for causing a computer to execute the control method according to any one of claims 12 to 15.

Images