JP2020112902A - Information processing device, control method thereof, program, information processing system, and control method thereof - Google Patents

Abstract

To appropriately issue announcement related to a cared person.SOLUTION: When a predetermined announcement issuing condition is satisfied so as to issue announcement by a sensor arranged in a room of each cared person, face authentication information and an announcement condition are pre-registered for each person not to issue incorrect announcement or not to miss announcement in a case where a cared person enters not his/her own room but a room of another person and falls asleep on a bed in the room of the another person. Then, announcement is issued based on an announcement issuing condition determined for each person after identifying the person by face authentication.SELECTED DRAWING: Figure 12

Description

The present invention relates to a method for a caregiver to check the condition of a cared person.

At the nursing care site, the biometric information of the cared person is obtained using a bed sensor or the like installed in the cared person's room, etc., and based on the obtained biometric information and the settings related to the alerting made in advance. There is a mechanism for determining whether or not to issue a notification about an abnormality of the care recipient, and notifying the caregiver of the physical condition of the care recipient when it is determined to issue the notification. (For example, refer to Patent Document 1).

Japanese Patent Laid-Open No. 2018-61842

In the invention described in Patent Document 1, a sensor for obtaining biological information, a setting related to reporting, and a person are associated with each other, and when the person reaches a predetermined state set in advance, a report is issued. To be done.

However, when the invention described in Patent Document 1 is applied to a plurality of persons, although it is possible to make settings for reporting for each person, when the sensor acquires biometric information of persons who have no correspondence , There is a risk of false alarms and loss of information.

For example, assume a situation where the settings for the bed sensor in room 201 and the settings for reporting in the bed sensor in room 202 are different.

When the care receiver living in room 201 sleeps in room 202, there is a risk that the care giver may not be notified at the time when the alarm should be issued or may be notified at the time when the alarm should not be issued.

Therefore, it is an object of the present invention to more appropriately issue an alert regarding a cared person.

An information processing apparatus according to the present invention is an information processing apparatus capable of communicating with an image capturing apparatus that captures an image of a person and a storage unit that stores a report condition for issuing a report corresponding to the person. Identification means for identifying a person based on an image obtained by imaging, acquisition means for obtaining information on the person identified by the identification means, information on the person obtained by the acquisition means, and the storage unit And a determination means for determining the presence or absence of a report based on the report condition stored in.

According to the present invention, there is an effect that it is possible to more appropriately issue an alarm regarding a cared person.

It is a block diagram which shows an example of a structure of the watching system in embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of an information processing apparatus. It is a block diagram which shows an example of the hardware constitutions of a network camera. It is a block diagram which shows an example of the hardware constitutions of a portable terminal. It is a block diagram showing an example of composition of a module of an information processor. It is a flowchart which shows the flow of a watching process. It is a flowchart which shows the flow of a watching process. It is a block diagram which shows the structure of a resident information table. It is a block diagram which shows the structure of an all resident collation table. It is a block diagram which shows the structure of an all resident condition table. It is a block diagram which shows the structure of a sensor table. It is a screen display example which shows an example of a screen display in a portable terminal.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram showing an example of the configuration of a watching system (corresponding to an information processing system) according to an embodiment of the present invention.

In the watching system according to the embodiment of the present invention, the information processing apparatus 100, the network camera 101, the recording server 102, the mobile terminal 103, and the sensor 104 communicate with each other via a network 110 (for example, WAN=wide area network). Enabled and connected.

The network camera 101 is a camera that shoots an arbitrary place and transmits the video/image obtained by the shooting to the recording server 102.

The recording server 102 is a server that receives and stores the video/image transmitted from the network camera 101, and stores the video/image, and transmits the video/image to the information processing apparatus 100.

The information processing device 100 is a device that receives a video/image stored in the recording server 102 and performs a process of watching over a care receiver.

The mobile terminal 103 is a terminal that displays the result of the watching process in the information processing device 100. Normally, it is owned by the caregiver, but may be owned by a person concerned with the caregiver, such as the family of the caregiver.

The processes performed by the information processing device 100, the network camera 101, and the recording server 102 may be performed in a single device such as the network camera 101.

The sensor 104 is, for example, a motion sensor for detecting a person or a bed sensor for detecting biological information.

The sensor for detecting biological information is not a wearable sensor but a sensor installed in each room.

Alternatively, the recording server 102 may not be provided, and a video/image obtained by shooting with the network camera 101 may be stored in a cloud or the like.

In addition, the network camera 101 may be provided in plurality instead of one. Also, a plurality of sensors 104 may be provided.

The network camera 101 may detect a person without the presence sensor.

In addition, the mobile terminal 103 may simultaneously display the respective videos/images displayed by the plurality of network cameras 101 in the display areas corresponding to the respective network cameras 101.

This is the end of the description of the configuration of the watching system shown in FIG.

The hardware configuration of the information processing apparatus 100 shown in FIG. 1 will be described below with reference to FIG.

The CPU 201 centrally controls each device and controller connected to the system bus 204.

Further, the ROM 202 or the external memory 211 will be described later, which is necessary to realize a function executed by each server or each PC, such as a BIOS (Basic Input/Output System) and an OS (Operating System) which are control programs of the CPU 201. Various programs are stored. In addition, information necessary for carrying out the present invention is stored. The external memory may be a database.

The RAM 203 functions as a main memory, a work area, etc. of the CPU 201. The CPU 201 loads various programs necessary for executing the processing from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.

The input controller 205 controls inputs from a keyboard (KB) 209 and a pointing device such as a mouse (not shown).

The video controller 206 controls display on a display such as the display 210. The display may be a display such as a liquid crystal display. These are used by the administrator as needed.

The memory controller 207 is an external storage device (hard disk (HD)) that stores a boot program, various applications, font data, user files, edit files, various data, etc., a flexible disk (FD), or a PCMCIA (Personal Computer). Controls access to an external memory 211, such as a CompactFlash (registered trademark) memory connected to a Memory Card International Association (Card) card slot via an adapter.

The communication I/F controller 208 connects and communicates with an external device via a network, and executes a communication control process on the network. For example, communication using TCP/IP (Transmission Control Protocol/Internet Protocol) is possible.

The CPU 201 can display on the display 210, for example, by executing an outline font rasterization process in the display information area in the RAM 203. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display 210.

Various programs to be described later for implementing the present invention are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 203 as necessary.

The information necessary for carrying out the present invention may be stored in a device such as a cloud different from the information processing device 100.

This is the end of the description of the hardware configuration of the information processing apparatus 100 shown in FIG. The above configuration is an example, and can be appropriately changed according to the purpose and use.

Hereinafter, the hardware configuration of the network camera 101 shown in FIG. 1 will be described with reference to FIG.

The CPU 301 centrally controls each device and controller connected to the system bus 304. Further, the ROM 302 or the external memory 305 has a control program for the CPU 301, such as a BIOS (Basic Input/Output System), an operating system program (hereinafter, OS), and a later-described necessary for realizing the function executed by the network camera 101. Various programs to be executed are stored. The RAM 303 functions as a main memory, a work area, etc. of the CPU 301.

The CPU 301 loads various programs and the like necessary for executing processing into the RAM 303 and executes the programs to realize various operations.

A memory controller (MC) 306 is a CF memory connected to a hard disk (HD) or a PCMCIA card slot for storing a boot program, various applications, font data, user files, edit files, various data, images, etc. via an adapter. It controls access to the external memory 305 such as a smart media (registered trademark) or the like.

The camera unit 307, which is connected to the image processing unit 308, photoelectrically converts light obtained by passing through a lens directed to a monitoring target by a light receiving cell such as a CCD or a CMOS, and then outputs an RGB signal. And a complementary color signal to the image processing unit 308.

The image processing unit 308 performs white balance adjustment, gamma processing, sharpness processing, and YC signal processing based on the RGB signal and the color capturing signal, and further generates a luminance signal Y and a chroma signal (hereinafter, YC signal). Then, the YC signal is compressed in a predetermined compression format (for example, JPEG format or Motion JPEG format), and the compressed data is temporarily stored in the external memory 305 as an image.

The communication I/F controller (communication I/FC) 309 connects and communicates with an external device via a network, executes communication control processing on the network, and stores an image stored in the external memory 305. Is transmitted to an external device by the communication I/F controller 309.

This is the end of the description of the hardware configuration of the network camera 101 shown in FIG. The above configuration is an example, and can be appropriately changed according to the purpose and use.

The hardware configuration of the mobile terminal 103 shown in FIG. 1 will be described below with reference to FIG.

The CPU 401 centrally controls each device and controller connected to the system bus 404.

Further, the ROM 402 or the flash memory 414 includes a control program of the CPU 401, such as a BIOS (Basic Input/Output System), an operating system program (hereinafter referred to as OS), and a function to be executed by the mobile terminal, which will be described later. Various programs to be executed are stored. The RAM 403 functions as a main memory, a work area, etc. of the CPU 401.

The CPU 401 implements various operations by loading a program or the like required for execution of processing into the RAM 403 and executing the program.

The input controller 405 also controls inputs from the touch panel 410, the microphone 411, and the camera 412. The touch panel 410 controls input related to a user's touch operation, the microphone 411 controls voice input, and the camera 412 controls input of captured still images and moving images. The output controller 406 controls output to the touch panel 410 and the speaker 413.

The touch panel 410 detects a touch operation by the user and displays the image sent from the output controller 406 described above. The touch panel 410 is a component in which a display device and a position input device are integrated. It is also possible to detect a touch operation on a plurality of locations (hereinafter, referred to as multi-touch).

A memory controller (MC) 407 controls access to a flash memory 414 that stores a boot program, browser software, various applications, font data, user files, edit files, various data, and the like. Although a flash memory is described in the present embodiment, a storage medium such as a hard disk (HD), a flexible disk (FD), or a card-type memory connected to a PCMCIA card slot via an adapter may be used.

The communication I/F controller 409 connects and communicates with an external device via a network, and executes communication control processing on the network. For example, internet communication using TCP/IP is possible.

The sensor controller 408 controls input from the sensor 415 included in the mobile terminal 103. Various sensors exist in the sensor 415 of the mobile terminal 103, and are, for example, a direction sensor and an acceleration sensor.

Note that the CPU 401 enables display on the touch panel 410 by executing outline font rasterization processing in a display information area in the RAM 403, for example.

Various programs and the like used by the mobile terminal 103 of the present invention to execute various processes described later are recorded in the flash memory 414, and are executed by the CPU 401 by being loaded into the RAM 403 as necessary. .. Further, definition files and various information tables used by the program according to the present invention are stored in the flash memory 414.

The definition file and various information tables used by the program according to the present invention may be stored in a device other than the information processing device 100, such as a cloud.

This is the end of the description of the hardware configuration of the mobile terminal 103 shown in FIG. The above configuration is an example, and can be appropriately changed according to the purpose and use.

Hereinafter, a block diagram showing an example of the configuration of modules of the information processing apparatus 100 according to the embodiment of the present invention will be described with reference to FIG. The details of each function will be described together with the flowcharts described later.

The information processing device 100 includes an acquisition unit 501, a determination unit 502, an authentication unit 503, a notification unit 504, and a management unit 505.

The acquisition unit 501 is a functional unit that controls to acquire a table, an image, sensor data, and the like.

In the present embodiment, the determination unit 502 is a functional unit that controls to determine whether or not a person can be identified by face authentication, whether or not an instruction from a user is accepted, and whether or not data related to the person satisfies a reporting condition. The authentication unit 503 is a functional unit that controls to perform face authentication.

The notification unit 504 is a functional unit that controls to notify the mobile terminal 103 and the like, and the management unit 505 is a functional unit that controls to manage a table and the like.

This is the end of the description of the block diagram showing the exemplary configuration of the modules of the information processing apparatus 100 shown in FIG.

Hereinafter, the watching process according to the embodiment of the present invention will be described with reference to FIGS. 6 and 7.

In the present embodiment, the caregiver is called a staff member and the care receiver is called a resident.

In addition, it is assumed that the network camera 101 is installed in each room where there is a resident.

In step S601, the acquisition unit 501 acquires resident information from the resident information table 800 shown in FIG. 8 and sensor-related information from the sensor table 1100 shown in FIG.

The resident information table 800 includes an ID 801, a name 802, a heart rate 803, a breathing rate 804, a bed leaving 805, a main room 806, a permission area 807, a location area 808, a face image path 809, a staff ID 810, a resident ID 811, and a resident ID. It is composed of a temporary storage flag 812.

The ID 801 is an ID given to a person in a care facility, such as a resident or a staff member. If necessary, it may be given to the resident's family.

The resident ID 801 is a list of numbers, but “A” is added to the head of the staff ID 801. “B” is added to the head of the resident family ID 801.

The name 802 is the name of the person corresponding to the ID 801 given to the resident or staff.

The heart rate 803 is one of the reporting conditions and is a threshold of the heart rate of the resident. Whether or not to notify the mobile terminal 103 is determined based on the threshold value.

The respiratory rate 804 is one of the reporting conditions and is a threshold of the occupant's respiratory rate. Whether or not to notify the mobile terminal 103 is determined based on the threshold value.

Bed leaving 805 is one of the notification conditions, and when "Yes" is stored, the mobile terminal 103 is notified when the resident leaves the bed.

The reporting condition is not limited to the heartbeat rate 803, the breathing rate 804, and the leaving bed 805, and may be any item relating to the biometric information and operation of the resident.

The main room 806 is the main room of the person corresponding to the ID 801. Usually, this is a room in which the bed corresponding to the ID 801 is located and the network camera 101 is installed.

The permission area 807 is an area where the person corresponding to the ID 801 is permitted to enter the room, and a plurality of permission areas may be set. It is also possible to set a prohibited area.

The location area 808 is the current location area (use area) of the person corresponding to the ID 801. Since the location area 808 is associated with the network camera 101, the location area 808 can be specified if it is possible to recognize which network camera is used to capture the image.

The face image path 809 is a path in which the face image of the person corresponding to the ID 801 is stored.

The staff member ID 810 is the ID of the staff member who watches over the person corresponding to the ID 801.

The responsible resident ID 811 is the ID of the resident monitored by the person corresponding to the ID 801.

The temporary storage flag 812 is a flag indicating that the resident information table 800 is temporarily stored. OFF is stored unless otherwise specified.

Since the resident information table 800 is created in association with the network camera 101 installed in the resident's main room, basically, information on the resident in the main room 806 and the staff watching over the resident is stored. It is stored.

For example, when an employee whose information is not stored in the resident information table 800 enters the room, the information of the employee is temporarily stored in the resident information table 800.

The sensor table 1100 includes data items 1101 and sensors 104.

The data item 1101 is an item for which data can be acquired by the sensor 104, and is a condition item related to the presence/absence of notification to the mobile terminal 103.

The sensor 104 is a sensor that acquires data of an item corresponding to the data item 1101.

In step S602, the determination unit 502 determines whether the instruction to end the watching process illustrated in FIGS. 6 and 7 has been received.

If it is determined that the end instruction is received, the process is ended, and if it is not determined that the end instruction is received, the process proceeds to step S603.

In step S603, the acquisition unit 501 acquires a video image captured by the network camera 101.

In step S604, the acquisition unit 501 acquires that the sensor 104 has detected a person.

In step S605, the authentication unit 503 performs face authentication on the face included in the image acquired in step S603 using the face image stored in the face image path 809 of the resident information table 800.

In step S606, the determination unit 502 determines whether the person can be identified by the face authentication in step S605. That is, it is determined whether there is a person included in the resident information table 800 in the video acquired in step S603.

When it is determined that the person can be identified, the process proceeds to step S621, and when it is not determined that the person can be identified, the process proceeds to step S607.

In step S607, the acquisition unit 501 acquires the information of all the residents from the all-resident comparison table 900 shown in FIG.

The all resident collation table 900 includes an ID 901, a name 902, a location area 903, a face image path 904, a staff member ID 905, and a resident ID 906.

The ID 901, name 902, location area 903, face image path 904, staff ID 905 in charge, and tenant ID in charge 906 are the same as ID 801, name 802, location area 808, face image path 809, staff ID 810, and tenant ID 811 in charge. In addition, it is information used when face recognition is performed on the person detected in step S604.

The resident information table 800 is a table corresponding to one network camera 101, but the all resident matching table 900 aggregates information used for face authentication stored in a plurality of resident information tables 800.

In step S608, the authentication unit 503 performs face authentication on the face included in the video acquired in step S603 using the face image stored in the face image path 904 of the all resident matching table 900.

In step S609, the determination unit 502 determines whether the person can be identified by the face authentication in step S608. That is, it is determined whether there is a person included in the all-resident matching table 900 in the video acquired in step S603.

If it is determined that the person can be identified, that is, if it is determined that the person detected in step S604 can be identified, the process proceeds to step S613, and if it is not determined that the person can be identified, the process proceeds to step S610.

In step S610, the notification unit 504 notifies the mobile terminal 103 that the person detected in step S604 is a person who is not registered in the resident information table 800 or the all resident verification table 900.

For example, as in 1208 of the screen display example 1200 of FIG. 12, the mobile terminal 103 is caused to display “Authentication is not possible. There is one who is not registered in the database”.

A screen display example 1200 will be described with reference to FIG.

Reference numeral 1201 indicates the status of each area (for example, a room).

1202 is the name of the area. For example, in room 501 in which Mr. Nakajima resides, it is displayed as "501 Nakajima".

It should be noted that 1211 showing the 4th floor common area is displayed so as to indicate that Mr. Nakajima is in the 4th floor common area 1211.

Reference numeral 1203 indicates the status of the area. For example, if Mr. Nakajima is not in room 501 and is in the common area on the 4th floor, which is a permission area, the message "Absent. I am in the common area on the 4th floor." is displayed.

An icon 1204 is used to confirm the status of the area with a video. When the user clicks on 1204, an image obtained by shooting with the network camera 101 installed at 1202 is displayed on the mobile terminal 103 as at 1210.

In the present embodiment, it is assumed that one network camera 101 is installed in each area, but a plurality of network cameras 101 may be installed in one area, and a plurality of 1204 is provided in one area. It may be displayed.

Reference numeral 1205 indicates that in the room 502 in which Mr. Shimaoka resides, Mr. Shimaoka's location area 808 is room 502 and is sleeping. Further, the room temperature is measured by the sensor 104 installed in the room 502, and a display is displayed to warn that the room temperature exceeds 30 degrees.

Apart from 1205, an alert “502 Room temperature is high” as shown in 1214 may be displayed on the mobile terminal 103 to ask the staff for urgency.

Reference numeral 1206 indicates that Mr. Sugano, who is in room 301, has left the entrance.

Separately from 1206, an alert as shown in 1214 “Mr. Sugano at the entrance 301 has left the entrance. Please chase immediately.” may be displayed on the mobile terminal 103 so that the staff can pay attention.

In addition, 1212 showing outside the entrance is displayed so as to indicate that Mr. Sugano is outside the entrance.

1207 indicates that Mr. Kamiya, who is in room 302, is not in room 302, and the area 808 is a dining room. It also shows that Mr. Takayanagi, who is in Room 402, is in Room 302.

Apart from 1207, an alert as shown in 1214, “Mr. Takayanagi of 302 402 is entering the room. Change of room? Yes/No” may be displayed on the mobile terminal 103.

By selecting Yes, Mr. Takayanagi can receive an instruction to change the room from Room 402 to Room 302.

In addition, in 1209 showing the dining room, it is displayed so that Mr. Kamiya is in the dining room 1209.

In addition, in 1213 indicating the room 402 in which Mr. Takayanagi resides, it is displayed that Mr. Takayanagi's location area 808 is room 302 instead of room 402.

Reference numeral 1208 indicates that there is an unregistered person in the all resident verification table 900 in the unoccupied room 202. When a person who is not registered in a table such as the resident information table 800, the all resident matching table 900, or the all resident condition table 1000 shown in FIG. 10 enters the room, for example, “authentication is not possible. Is displayed.” is displayed.

The all resident condition table 1000 includes an ID 1001, a heart rate 1002, a breathing rate 1003, a bed 1004, a main room 1005, and a permission area 1006.

ID 1001, heart rate 1002, respiration rate 1003, leaving bed 1004, main room 1005, and permission area 1006 are assigned to IDs in the same manner as ID 801, heart rate 803, respiration rate 804, leaving bed 805, main room 806, and permission area 807. This is an item indicating the reporting condition of the corresponding person.

Reference numeral 1216 included in 1208 is a face image of the unregistered person cut out from the image acquired in step S603. By displaying in this way, it is possible to visually confirm who the unregistered person is. A mode in which a face image is displayed by clicking 1204 may be used.

Apart from 1208, an alert such as “There is a person who is not registered in the 202 database. New occupant? Yes/No” as shown in 1214 may be displayed on the mobile terminal 103.

By selecting Yes, it is possible to register the information of unregistered persons in the resident information table 800, the total resident matching table 900, the total resident condition table 1000, and the like.

Reference numeral 1214 is an alert displayed on the mobile terminal 103. It is displayed when a notification is received by the watching process by the information processing device 100.

When the staff member selects the handled box 1215, the handled box 1215 becomes black and it can be seen at a glance that the handling is completed.

In this way, by causing the mobile terminal 103 to perform the display shown in the screen display example 1200, the staff and the family of the resident can easily confirm the status of the resident.

The information displayed on the mobile terminal 103 may of course be set for each mobile terminal 103 and each user. For example, the resident's family need only be able to confirm the status of the resident in their relatives.

In step S611, the determination unit 502 determines whether the instruction to newly register the person detected in step S604 has been received via the mobile terminal 103.

For example, a screen urging registration is displayed on the mobile terminal 103, such as “There is a person who is not registered in the database 202 in alert 20214 of screen display example 1200. New resident? Yes/No”. It is determined whether the registration instruction has been accepted.

If it is determined that the instruction to newly register is accepted, the process proceeds to step S612, and if it is not determined that the instruction to newly register is accepted, the process returns to step S602.

In step S612, the management unit 505 stores information about the person detected in step S604 in the table based on the new registration instruction in step S611.

When the information is stored in the resident information table 800 when the information is stored in association with the network camera 101, when the information is stored in association with another network camera 101, the all resident matching table 900 and the all resident condition table 1000 are stored. Store the information in.

The resident information table 800 is a table corresponding to one network camera 101, but the total resident matching table 900 and the total resident condition table 1000 are a collection of a plurality of resident information tables 800. This is a table divided into information used for face authentication (all resident matching table 900) and information used for determining whether the reporting condition is satisfied (all resident condition table 1000).

By dividing into the all resident matching table 900 and the all resident condition table 1000, whether only the information used at the time of face authentication is acquired in step S607 and the alarm condition is satisfied when the person can be identified by the face authentication? It is possible to obtain information used when determining.

That is, if it is not determined in step S609 that the person can be identified by the face authentication, the entire resident condition table 1000 does not have to be acquired, and the load on the network can be reduced.

Further, by storing the information of the person who is not identified by the face authentication in step S606 and step S609 in the table, the person is identified by the face authentication in step S606 or step S609. For example, it is effective when registering information of a new resident.

It should be noted that the information contained in the resident information table 800, the total resident matching table 900, and the total resident condition table 1000 is synchronized with other tables when any one of the tables is edited. ..

For example, when the person's location area 808 included in the resident information table 800 is edited from “201” to “dining room”, the location area 903 of the all-resident matching table 900 is also automatically edited to “dining room”. ..

In step S613, the acquisition unit 501 acquires information about all the residents from the all-resident condition table 1000.

In step S614, the management unit 505 compares the information stored in the all resident matching table 900 and the information stored in the all resident condition table 1000 for the person detected in step S604. Temporarily store in 800.

ON is stored in the temporary storage flag 812 so that it is understood that the temporary storage is performed.

By temporarily storing in the resident information table 800, it is possible to watch a person who is not stored in the resident information table 800 without repeatedly reading various tables.

In step S615, the management unit 505 updates the resident area 808.

The resident's location area may be displayed on the mobile terminal 103 with a screen as shown in a screen display example 1200.

In step S616, the determination unit 502 determines whether the person's location area 808 is included in the permission area 807.

If it is determined that the location area 808 is included in the permission area 807, the process proceeds to step S622, and if it is not determined that the location area 808 is included in the permission area 807, the process proceeds to step S617.

In step S617, the notification unit 504 notifies the mobile terminal 103 that the person detected in step S604 is in an area where the person is not permitted (area where there is no use permission).

For example, as in 1206 of the screen display example 1200, the mobile terminal 103 is caused to display “I have left the entrance!”.

In addition, a screen prompting Mr. Sugano to chase is displayed on the mobile terminal 103, as in the alert 1214 of the screen display example 1200, "Mr. Kanno outside the entrance 301 has left the entrance. Please chase immediately."

The owner of the mobile terminal 103 can confirm that the resident is not in the permission area 807, that the staff is not in the permission area 807, and that the person who is not in the permission area is currently in the area 808. You can know

In step S618, the determination unit 502 determines whether the instruction to edit the permission area 807 of the person detected in step S604 is received via the mobile terminal 103.

For example, a screen prompting to edit the information of the person is displayed on the mobile terminal 103, such as “Mr. Takayanagi of 302 402 is entering the entrance. Yes/No” of the alert 1214 of the screen display example 1200, and the screen is displayed. It is determined whether an instruction to register is accepted.

If it is determined that the instruction to edit is accepted, the process proceeds to step S619. If it is not determined that the instruction to edit is accepted, the process returns to step S602.

In step S619, the management unit 505 adds the location area 808 to the permission area 807.

In step S620, the management unit 505 updates the resident area 808.

In step S621, the determination unit 502 determines whether the person detected in step S604 is the target person to be notified to the mobile terminal 103.

Specifically, the determination is made from the first character of the ID 801, and if the first character of the ID 801 is a numeral, that is, if it is a resident, the process proceeds to step S622.

If the first character of the ID 801 is not a number, that is, if the person is not a resident but a person registered as a staff member or family, the process returns to step S602.

In step S622, the acquisition unit 501 acquires the data related to the person specified in step S606 or step S609 from the sensor 104.

The data to be acquired from the sensor 104 is data relating to items whose conditions are stored in the heart rate 803, the breathing rate 804, and the leaving bed 805 of the resident information table 800.

For example, in the case of Mr. Ichiro Tanaka, data regarding the heart rate 803, the respiratory rate 804, and the leaving bed 805 are acquired from the sensor 104.

The data to be acquired from the sensor 104 may be different for each person, and is not limited to the heart rate 803, the respiration rate 804, and the leaving bed 805.

The sensors used to acquire the heart rate 803, the respiration rate 804, and the leaving bed 805 are shown in the sensor table 1100 shown in FIG.

The sensor table 1100 includes data items 1101 and sensors 1102.

The data item 1101 is an item of data to be acquired from the sensor 104.

The sensor 1102 is a sensor that acquires the item described in the data item 1101.

Since the sensor table 1100 is associated with the network camera 101, even if the person corresponding to the ID 801 moves in the room, data is acquired from the sensor 104 associated with the camera that identifies the person.

That is, when the person moves from the room "201" to the room "202", the sensor 104 for acquiring the person's data also changes from the sensor 104 installed in "201" to the sensor installed in "202".

In step S623, the determination unit 502 determines whether the data acquired in step S622 satisfies the report condition shown in the resident information table 800, that is, whether the report needs to be issued.

If it is determined that the reporting condition is satisfied, the process proceeds to step S624, and if it is not determined that the reporting condition is satisfied, the process returns to step S622.

In step S624, the notification unit 504 notifies the mobile terminal 103 of the location area 808 of the person identified in step S606 or step S609 and the warning information regarding the person.

By notifying the mobile terminal 103 when the notification condition set in advance is satisfied, the staff and the family of the resident can immediately confirm the state of the resident who satisfies the notification condition.

Steps S622 to S624 are processes that are repeated until the person detected by the motion sensor cannot be specified.

In step S625, the determination unit 502 determines whether the temporary storage flag 812 of the person detected in step S604 is ON.

If it is determined that the temporary storage flag 812 is ON, the process proceeds to step S626.

If it is not determined that the temporary storage flag 812 is ON, and if the process has been performed for all the persons detected in step S604, the process proceeds to step S602.

If it is not determined that the temporary storage flag 812 is ON, and if the process has not been performed for all the persons detected in step S604, the process proceeds to step S605.

In step S626, the management unit 505 deletes the information of the person whose temporary storage flag 812 is ON from the resident information table 800.

Therefore, the information of the person temporarily stored in the resident information table 800 is deleted when the person cannot be specified, and thus the load on the resident information table 800 can be reduced.

Steps S605 to S626 are processes that are repeated until the processes are performed for all the persons detected by the sensor 104, and the processes for a plurality of persons may be performed at the same time.

This is the end of the description of the watching process shown in FIGS. 6 and 7.

In the present embodiment, since the person is identified by using face recognition and then the alarm is issued based on the alarm condition provided for each person, the care receiver enters the room of another person instead of the room where he/she lives. Even if it happens, it is possible to issue an appropriate alarm.

<Other embodiments>
In the present embodiment, the watching process is performed by the network camera 101 installed in the main room of the resident. You may perform the watching process shown in FIG.

Further, the network camera 101 may be installed in an area having a plurality of private toilets, and the face image captured when entering the private toilet may be authenticated by using a face image stored in advance.

Then, face authentication may be similarly performed when leaving the toilet, and if a person does not leave the room after a predetermined time has passed from the face authentication time when entering the toilet, an alert regarding the person may be issued.

Alternatively, the wearer may have a wearable sensor that can obtain position information and that includes identification information of a person, and the position information and identification information of the wearable sensor may be obtained.

Then, it may be determined that a person different from the person corresponding to the identification information possesses the wearable sensor based on the acquired position information, identification information, and identification of the person by face authentication.

Further, the network camera 101 may perform the watching process shown in FIGS. 6 and 7 by identifying the wearable sensor that is the imaging target of the network camera 101 based on the position information of the wearable sensor.

The processes shown in FIGS. 6 and 7 may be performed not by the information processing apparatus 100 but by the network camera 101 having the system configuration as shown in FIG. 3, for example.

The present invention can be embodied as, for example, a system, an apparatus, a method, a program, a storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. It may be applied to a device consisting of one device. It should be noted that the present invention includes a software program that realizes the functions of the above-described embodiments, which directly or remotely supplies the system or device with the software program. The present invention also includes a case where the information processing apparatus of the system or the apparatus is achieved by reading and executing the supplied program code.

Therefore, the program code itself installed in the information processing device to implement the functional processing of the present invention in the information processing device also implements the present invention. That is, the present invention also includes the computer program itself for realizing the functional processing of the present invention.

In that case, the program may take any form such as an object code, a program executed by an interpreter, or script data supplied to an OS as long as it has the function of the program.

Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, MO, a CD-ROM, a CD-R, a CD-RW, and the like. There are also magnetic tapes, non-volatile memory cards, ROMs, DVDs (DVD-ROMs, DVD-Rs), and the like.

In addition, as a method of supplying the program, a browser of a client computer is used to connect to a home page on the Internet. The computer program itself of the present invention or a compressed file containing an automatic installation function can be downloaded from the homepage to a recording medium such as a hard disk.

It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from different homepages. In other words, a WWW server that allows a plurality of users to download the program file for implementing the functional processing of the present invention on the information processing apparatus is also included in the present invention.

In addition, the program of the present invention is encrypted and stored in a storage medium such as a CD-ROM and distributed to users, and the key information for decrypting the encryption is downloaded from the homepage via the Internet to users who have satisfied predetermined conditions. Let Then, it is also possible to execute the encrypted program by using the downloaded key information and install the program in the information processing device to realize the program.

Further, the functions of the above-described embodiments are realized by the information processing device executing the read program. In addition, the OS or the like running on the information processing device may perform some or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments may be realized by the processing.

Further, the program read from the recording medium is written in the memory provided in the function expansion board inserted in the information processing device or the function expansion unit connected to the information processing device. After that, based on the instructions of the program, the CPU or the like included in the function expansion board or the function expansion unit performs some or all of the actual processing, and the processing also realizes the functions of the above-described embodiments.

It should be noted that the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

100 Information Processing Device 101 Network Camera 102 Recording Server 103 Mobile Terminal 104 Sensor 110 Network 201 CPU
202 RAM
203 ROM
204 system bus 205 input controller 206 video controller 207 memory controller 208 communication I/F controller 209 keyboard 210 display 211 external memory

Claims (12)

An information processing device capable of communicating with an imaging device for imaging a person and a storage unit for storing a reporting condition for issuing a report corresponding to the person,
Specifying means for specifying a person based on an image obtained by picking up the image by the image pickup device;
An acquisition unit for acquiring information on the person specified by the specifying unit;
An information processing apparatus, comprising: a determination unit that determines whether or not a report is issued based on the information about the person acquired by the acquisition unit and the notification condition stored in the storage unit. The information processing apparatus according to claim 1, further comprising a display control unit that controls to display a determination result by the determination unit. The information processing apparatus according to claim 2, wherein the display control unit controls to display a screen prompting registration of the person when the person cannot be specified by the specifying unit. 4. The display control means controls to display a screen prompting to edit the information concerning the person when the person specified by the specifying means exists in an area where the use is not permitted. Information processing device. An information processing device capable of communicating with a mobile terminal,
The information processing apparatus according to any one of claims 1 to 4, further comprising: a notification unit that notifies the mobile terminal of the determination result of the determination unit. The information on the person includes information indicating the usage area of the person,
The information processing apparatus according to claim 5, wherein the notifying unit notifies the mobile terminal when the use area of the person specified by the specifying unit is an area for which use is not permitted. 7. The information processing apparatus according to claim 5, wherein the notifying unit notifies that the person is not registered when the person cannot be specified by the specifying unit. The information processing apparatus according to any one of claims 1 to 7, further comprising: a registration unit configured to register the information related to the person or the notification condition in the storage unit. A method of controlling an information processing apparatus capable of communicating with an image pickup apparatus for picking up an image of a person, and a storage section for storing a report condition for issuing a report corresponding to the person,
A specifying step of specifying a person based on an image obtained by being picked up by the image pickup device;
An acquisition step of acquiring information concerning the person identified in the identification step,
A control method of the information processing apparatus, comprising: a determination step of determining whether or not a report is issued based on the information about the person acquired in the acquisition step and the report condition stored in the storage unit. A program for controlling a computer capable of communicating with an image pickup device for picking up an image of a person and a storage unit for storing a report condition for issuing a report corresponding to the person,
The computer,
Specifying means for specifying a person based on an image obtained by picking up the image by the image pickup device;
An acquisition unit for acquiring information on the person specified by the specifying unit;
A program that functions as a determination unit that determines the presence or absence of a report based on the information about the person acquired by the acquisition unit and the notification condition stored in the storage unit. An information processing system capable of communicating with an image pickup device for picking up an image of a person and a storage unit for storing a notification condition for issuing an alarm corresponding to the person,
Specifying means for specifying a person based on an image obtained by picking up the image by the image pickup device;
An acquisition unit for acquiring information on the person specified by the specifying unit;
An information processing system, comprising: a determination unit that determines the presence or absence of a report based on the information related to the person acquired by the acquisition unit and the notification condition stored in the storage unit. A method for controlling an information processing system capable of communicating with an image pickup device for picking up an image of a person and a storage unit for storing a notification condition for issuing an alarm corresponding to the person,
A specifying step of specifying a person based on an image obtained by being picked up by the image pickup device;
An acquisition step of acquiring information concerning the person identified in the identification step,
A control method for an information processing system, comprising: a determination step of determining the presence or absence of a report based on the information about the person acquired in the acquisition step and the report condition stored in the storage unit.

Images