JP6828838B1 - Information processing equipment, information processing systems, information provision methods, and programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To notify a precursor of a predetermined operation by a user while suppressing erroneous notification in an information processing device which detects a user's state by using temperature image data of a user who uses a predetermined facility. To do. An information processing device is an information processing device that detects a state of the user by using temperature image data of a user who uses a predetermined facility, and is provided in the facility and in the vicinity of the facility. A judgment information management unit that manages judgment contents indicating the state of the user when a predetermined temperature is detected in a preset first detection area and a second detection area among a plurality of detection areas. And an information providing unit that provides information of the user indicating the determination content when the predetermined temperature is detected in the first detection area and the second detection area in the temperature image data. Have. [Selection diagram] FIG. 7

Description

The present invention relates to an information processing device, an information processing system, an information providing method, and a program.

In facilities such as medical facilities and long-term care facilities, users who use the bed installed in the facility (for example, have left the bed, have fallen from the bed, are likely to fall from the bed, etc.) There is an information providing system that provides information on.

For example, there is known an information processing system that detects a user's condition by using temperature image data of a user who uses a bed provided in a facility and notifies a predetermined notification destination of information indicating the user's condition. (See, for example, Patent Document 1).

In an information processing system as shown in Patent Document 1, it is desired to detect a sign of an action in which a user gets up and tries to get out of bed (hereinafter referred to as a sign of getting up) and notify the notification destination at the earliest possible stage. There is a request.

However, for example, if it is detected that the user has started to move his / her body before getting up and the notification destination is notified as a sign of getting up, the notification will be performed even in an action other than getting up such as turning over, which is an error. There is a problem that the number of notifications increases.

As described above, in the conventional technology, in the information processing system that detects the state of the user by using the temperature image data of the user who uses the bed, the erroneous notification is suppressed and the sign of the user getting up is swiftly detected. It was difficult to notify.

It should be noted that such problems are not limited to getting up from the bed, but also in various facilities on which the user lies like the bed, such as stretchers, operating tables, and examination tables provided in facilities such as medical facilities and nursing care facilities. It is commonly present in information processing systems that detect the user's condition.

An embodiment of the present invention has been made in view of the above problems, and is an erroneous notification in an information processing system that detects a user's state using temperature image data of a user who uses a predetermined facility. It is possible to notify the precursor of a predetermined operation by the user while suppressing the above.

To solve the above problems, an information processing apparatus according to an embodiment of the present invention is an information processing apparatus for detecting a state of the user using the temperature image data of a user who uses the bed, the bed Among the plurality of detection areas provided around the bed , the first detection area, which is the area where the pillows used by the user are arranged, and the fence provided around a part of the bed Among the information indicating the size of the color region corresponding to the predetermined temperature range in the second detection region which is the arranged region, the above-mentioned when the information of the region larger than the predetermined size is simultaneously detected. In the temperature image data, the judgment information management unit that manages the judgment content indicating the state of the user and the information of the area having the predetermined size or larger are simultaneously detected in the first detection area and the second detection area. It has an information providing unit that provides information on the user indicating the determination content when the information is processed.

According to the embodiment of the present invention, in an information processing system that detects a user's state using temperature image data of a user who uses a predetermined facility, a predetermined operation by the user is suppressed while suppressing erroneous notification. You will be able to notify the precursors of.

It is a figure which shows an example of the system structure of the information processing system which concerns on one Embodiment. It is a figure which shows another example of the system structure of the information processing system which concerns on one Embodiment. It is a figure which shows an example of the arrangement of the bed and the camera which concerns on one Embodiment. It is a figure which shows the example of the detection area which concerns on one Embodiment. It is a figure for demonstrating the determination condition which concerns on one Embodiment, and the example of the determination content. It is a figure which shows the example of the hardware configuration of the computer which concerns on one Embodiment. It is a figure which shows the example of the functional structure of the information processing apparatus which concerns on one Embodiment. It is a figure which shows the example of the information which the information processing apparatus which concerns on one Embodiment manages. It is a figure which shows the example of the setting screen which concerns on one Embodiment. It is a flowchart which shows the example of the processing of the information processing apparatus which concerns on one Embodiment. It is a sequence diagram which shows the example of the processing of the information processing system which concerns on one Embodiment. It is a figure which shows the example of the display screen which concerns on one Embodiment. It is a flowchart which shows the example of the determination process which concerns on 1st Embodiment. It is a flowchart which shows the example of the determination process which concerns on 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment. The information processing system 100 is connected to, for example, a plurality of cameras 102a to 102f installed in an inpatient ward 110, which is an example of a facility such as a medical facility or a nursing facility, and a plurality of cameras 102a to 102f via a network 104. Including the information processing device 101.

In the following description, when indicating an arbitrary camera among the plurality of cameras 102a to 102f, "camera 102" is used. Further, the number of cameras 102 shown in FIG. 1 is an example, and the number of cameras 102 may be one or more other numbers.

The camera 102 is a photographing device that captures image data indicating the temperature of an object (hereinafter referred to as temperature image data), and for example, a general infrared thermography camera (hereinafter referred to as a thermo camera) or the like can be applied. it can. A thermo camera is a device that images infrared radiation from an object to be measured, converts it into temperature, and visualizes the temperature distribution by color or the like. The thermo camera may be called a thermography, a thermo vision, a thermo viewer, or the like.

In the example of FIG. 1, the plurality of cameras 102a to 102f are installed on the wall, ceiling, or the like of each hospital room corresponding to each of the plurality of beds 103a to 103f provided in the hospital rooms A to F. For example, the camera 102a is installed on the wall surface corresponding to the bed 103a provided in the hospital room A, and is installed so as to be able to take temperature image data indicating the temperature around the bed 103a and the bed 103a.

Further, the camera 102a captures temperature image data indicating the temperature around the bed 103a and the bed 103a at a predetermined frame rate (for example, about 5 fps to 1 fps), and obtains the acquired temperature image data via the network 104. It is transmitted to the information processing device 101. The image data captured by the camera 102a is not limited to moving image data, and may be still image data captured at predetermined time intervals (for example, 1 second to 60 second intervals).

At this time, the temperature image data transmitted by the camera 102a includes identification information (for example, IP address, camera ID, bed ID, etc.) for identifying the camera 102a, the bed 103a, and the like.

Similarly, the other cameras 102b to 102f capture the temperature image data indicating the temperature of the corresponding bed and the surroundings of the bed, and the captured temperature image data together with the identification information are subjected to the information processing device 101 via the network 104. Send to.

The information processing device 101 is, for example, an information processing device having a computer configuration, or a system including a plurality of information processing devices. The information processing apparatus 101 acquires temperature image data transmitted from the camera 102, and uses the acquired temperature image data to detect the temperature in the bed and one or more predetermined detection regions around the bed. Further, when the information processing apparatus 101 detects a preset temperature, a combination of temperatures, a change in temperature, or the like in one or more detection regions, the information of the user indicating the state of the user who uses the bed. Is notified to a predetermined notification destination.

The users include, for example, patients who are hospitalized in medical facilities, residents who are in nursing care facilities, and the like. Here, as an example, the following description will be given assuming that the user is a patient admitted to the inpatient ward 110.

The nurse call system 121 is, for example, an information processing device having a computer configuration or a system including a plurality of information processing devices, and is an example of a predetermined notification destination to which the information processing device 101 notifies user information. The nurse call system 121 is an information terminal 123 possessed by the display devices 122a to 122c installed at the nurse stations A to C and the staff of the facility such as a nurse or a caregiver for the call information by the user who uses the bed 103. Etc. to display.

Further, as shown in FIG. 1, the nurse call system 121 according to the present embodiment is connected to the information processing device 101 via the network 104 and receives the user information notified from the information processing device 101. In addition, the nurse call system 121 causes the display devices 122a to 122c, the information terminal 123, and the like to display a display screen for notifying the user's status based on the received user information.

The plurality of display devices 122a to 122c are display devices provided in, for example, a nurse station or the like, and display a display screen transmitted from the nurse call system 121 or the information processing device 101.

The information terminal 123 is, for example, an information processing device such as a smartphone, a tablet terminal, or a notebook PC possessed by staff such as nurses and caregivers. The information terminal 123 can communicate with the nurse call system 121 by, for example, wireless communication, and by executing a predetermined application program (hereinafter referred to as an application), the information terminal 123 can be transmitted from the nurse call system 121 or the information processing device 101. Display the display screen to be sent.

The function of the nurse call system 121 may be included in the information processing device 101, for example, as shown in FIG. Alternatively, the function of the information processing device 101 may be included in the nurse call system 121.

The display devices 122a to 122c and the information terminal 123 are another example of a predetermined notification destination to which the information processing device 101 notifies the user's information.

(Example of bed and camera placement)
FIG. 3 is a diagram showing an example of the arrangement of the bed and the camera according to the embodiment. As shown in FIG. 3A, the camera 102 is installed on the wall surface of the hospital room 111 or the like so that the bed 103 corresponding to the camera 102 and the temperature image data around the bed can be taken. The camera 102 may be installed on the ceiling of the hospital room 111.

FIG. 3B shows a state in which the hospital room 111 is viewed from above. The camera 102 is installed so as to be able to acquire temperature image data of a predetermined range (hereinafter, referred to as a detection range 201) including the bed 103 corresponding to the camera 102 and the periphery of the bed 103.

(Example of detection area)
FIG. 4 is a diagram showing an example of a detection region according to an embodiment. The information processing device 101 manages area information which is information of one or more detection areas set in advance within the detection range 201 of the camera 102. Within the detection range 201 of the camera 102, for example, as shown in FIG. 4, a plurality of detection areas are set. In the example of FIG. 4, within the detection range 201, the detection areas A403, the detection areas B404-1, 404-2, the detection area C405, the detection areas D406-1, 406.2, and the detection areas E407-1, 407- 2nd grade is set.

In the following description, when any detection area B is indicated among the detection areas B404-1 and 404-2, "detection area B404" is used. Further, when indicating an arbitrary detection area D among the detection areas D406-1 and 406-2, "detection area D406" is used. Further, when indicating an arbitrary detection area E among the detection areas E407-1 and 407-2, "detection area E407" is used.

In the example of FIG. 4, the detection area A403 is set to include an area in which the pillow 402 used by the user is arranged. Further, the bed 103 includes, for example, fences 401-1 to 401-3 for fall prevention provided around the bed 103, including a part or all of the fences 401-1 and 401-2. As described above, the detection areas B404-1 and 404-2 are set. Further, the detection area C405 is set near the center of the bed 103, and the detection areas D406-1 and 406.2 are set at the ends of the bed 103 where the fences 401-1 to 401-3 are not provided. Has been done. Furthermore, detection areas E407-1 and 407-2 are set in a region (for example, a floor or the like) adjacent to the side of the bed 103.

The information processing device 101 manages the positions of a plurality of detection regions based on, for example, the coordinate information of the temperature image data. Further, as another example, the information processing apparatus 101 divides the temperature image data into a mesh-like small area (mesh), and manages the positions of a plurality of detection areas based on the divided small areas. You may. In this case, the information processing apparatus 101 has information indicating the vertical position of the divided small area (1, 2, 3, ...) And information indicating the horizontal position (A, B, C, ...).・ ・) Can be combined to manage the positions of multiple detection areas.

(About judgment conditions and judgment contents)
The information processing device 101 detects a predetermined temperature in a plurality of detection regions using the temperature image data acquired from the camera 102, and when the detection result satisfies a preset determination condition, the determination content corresponding to the determination condition. The user's information indicating the above is notified to the predetermined notification destination.

FIG. 5 is a diagram for explaining a determination condition and an example of the determination content according to the embodiment. FIG. 5A shows an image of a temperature image of the user 501 when the user 501 is lying on the bed 103. In the example of FIG. 5A, the user 501 wears clothes such as pajamas and loungewear, and the head 502, hands 503, feet 504, etc. are in a temperature range corresponding to the body temperature of the user 501. It shall be displayed in color.

In this way, when a predetermined temperature (for example, 35 ° C. to 39 ° C., etc.) corresponding to the body temperature of the user 501 is detected in the detection area A403, the information processing apparatus 101 changes the state of the user 501 to the bed. It is determined to be "sleeping" while lying on the floor. This is because it is generally considered that there are few cases where only the foot 504 and the hand 503 of the user 501 are located at the position of the pillow 402 of the user 501.

With this method, for example, even if the user 501 is covered with a futon, a blanket, or the like and the temperature of the user 501's hands 503, foot 504, etc. cannot be detected, the information processing device 101 can use the user 501. It can be determined that the state is "sleeping". In the present embodiment, "sleeping" includes a state in which the user 501 is awake and lying on the bed 103.

FIG. 5B shows an image of a temperature image of the user 501 when the user 501, who is in a relatively good physical condition, gets up from the “sleeping” state. As described above, when the predetermined temperature corresponding to the body temperature of the user 501 is not detected in the detection area A403 and the predetermined temperature is detected in the detection area C405, the information processing apparatus 101 uses the user 501. It is determined that the state has risen, that is, "get up".

However, among users 501, there are users who have difficulty getting up by themselves and users who do not want to get up by themselves (for example, patients who have just awakened from anesthesia after surgery). In such a case, it is desirable to detect the "precursor of getting up" and notify the predetermined notification destination at the earliest possible stage before the user 501 is in the "getting up" state.

For example, when the individual movements of the user 501 are recorded, the characteristic movements that the user 501 performs before getting up are specified, and the user 501 performs the movements, the "precursor of getting up" A method of notifying a predetermined notification destination of the existence of the product can be considered.

As a specific example, when the user 501 gets up, depending on the physical condition, he / she grabs the fences 401-1 and 401-2 next to the bed 103 and assists the fences 401-1 and 401-2 by himself / herself. There are cases where you can get up.

In this case, the user 501 grabs the fences 401-1 and 401-2 from the state of lying on the bed 103, so that, for example, as shown in FIG. 5C, the detection area A403 and the detection area B404 A temperature corresponding to the body temperature of the user 501 will be detected. Therefore, for example, when the information processing device 101 detects a temperature corresponding to the body temperature of the user 501 in the detection area A403 and the detection area B404, the information processing device 101 determines that the user 501 has a “precursor of getting up”. Can be done.

Preferably, the information processing device 101 states that the user 501 has a "precursor of getting up" when the state shown in FIG. 5 (C) is detected from the "sleeping" state shown in FIG. 5 (A). to decide. As a result, the information processing device 101 erroneously detects that there is a "precursor of getting up" when the user 501 grabs the fences 401-1 and 401-2 and lies on the bed 103, and the predetermined notification destination. It is possible to suppress the notification to.

As another example, when the user 501 gets up, depending on the physical condition, the fences 401-1 and 401-2 cannot be grasped, and for example, the user tries to get off the bed 103 from the foot 504. ..

In this case, for example, as shown in FIG. 5D, the temperature corresponding to the body temperature of the user 501 is detected in the detection area A403 and the detection area E407. Therefore, the information processing device 101 determines that the user has a "precursor of getting up" when, for example, a temperature corresponding to the body temperature of the user 501 is detected in the detection area A403 and the detection area E407. Is also good.

Preferably, the information processing apparatus 101 determines that the user 501 has a "precursor of getting up" when the state shown in FIG. 5 (D) is detected from the "sleeping" state shown in FIG. 5 (A). To do. As a result, the information processing device 101 prevents the user 501 from erroneously detecting that there is a "precursor of getting up" and notifying the predetermined notification destination when the user 501 returns to the bed 103 and lies down. be able to.

As described above, according to the present embodiment, in the information processing device 101 that detects the state of the user 501 by using the temperature image data of the user 501 that uses the bed 103, the user 501 gives a "precursor of getting up". , It is possible to notify a predetermined notification destination at an earlier stage.

In addition, the information processing device 101 according to the present embodiment erroneously notifies the user that there is a "precursor of getting up" when the user 501 performs an operation other than getting up, such as turning over. It is possible to reduce the amount of waste.

Therefore, according to the present embodiment, in the information processing system 100 that detects the state of the user 501 by using the temperature image data of the user 501 that uses the bed 103, the user 501 "" while suppressing erroneous notification. You will be able to notify "a sign of getting up".

The bed 103 is an example of predetermined equipment. The bed 103 may be, for example, various facilities such as a stretcher, an operating table, an examination table, etc. provided in a facility such as a medical facility or a nursing facility, on which the user 501 lies like the bed.

Further, the "precursor of getting up" is an example of the determination content indicating the state of the user 501. For example, if the individual movements of the user 501 are recorded and the user 501 performs a characteristic movement before performing a movement other than getting up (for example, hydration, urination, defecation, etc.), information is provided. The processing device 101 may be able to determine a precursor of an operation other than the rising operation.

<Hardware configuration>
Subsequently, an example of the hardware configuration of the information processing apparatus 101 will be described. The information processing device 101 has, for example, the hardware configuration of the computer 600 as shown in FIG. Alternatively, the information processing device 101 is composed of a plurality of computers 600.

FIG. 6 is a diagram showing an example of the hardware configuration of the computer according to the embodiment. The computer 600 includes, for example, a CPU (Central Processing Unit) 601, a ROM (Read Only Memory) 602, a RAM (Random Access Memory) 603, an HD (Hard Disk) 604, an HDD (Hard Disk Drive) controller 605, a display 606, and an external device. It includes a device connection I / F (Interface) 607, a network I / F 608, a keyboard 609, a pointing device 610, a DVD-RW (Digital Versatile Disk Rewritable) drive 612, a media I / F 614, a bus line 615, and the like.

Of these, the CPU 601 controls the overall operation of the computer 600. The ROM 602 stores, for example, a program used for starting the computer 600 such as an IPL (Initial Program Loader). The RAM 603 is used, for example, as a work area of the CPU 601 or the like. The HD604 stores various data such as a program, for example. The HDD controller 605 controls reading or writing of various data to the HD 604 according to the control of the CPU 601.

The display 606 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 607 is an interface for connecting various external devices. The network I / F608 is an interface for performing data communication using the network 104.

The keyboard 609 is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 610 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 612 controls reading or writing of various data to the DVD-RW611 as an example of the removable recording medium. The DVD-RW611 is not limited to the DVD-RW611, and may be another recording medium. The media I / F 614 controls reading or writing (storage) of data to the media 613 such as a flash memory or a memory card. The bus line 615 includes an address bus, a data bus, various control signals, and the like for electrically connecting each of the above components.

<Functional configuration>
Subsequently, the functional configuration of the information processing system 100 will be described. FIG. 7 is a diagram showing an example of the functional configuration of the information processing system 100 according to the embodiment. The information processing system 100 includes, for example, an information processing device 101 connected to a network 104 such as a LAN (Local Area Network), and a plurality of cameras 102a, 102b, ..., And the like. Further, the information processing device 101 can communicate with the nurse call system 121, the information terminal 123, and the display devices 122a, 122b, ..., Which are examples of predetermined notification destinations, via, for example, a network 104.

(Functional configuration of information processing device)
The information processing device 101, for example, by executing a predetermined program on the CPU 601 of FIG. 6, acquires acquisition unit 701, detection unit 702, determination information management unit 703, information providing unit 704, area information management unit 705, and image data management. A unit 706, a display control unit 707, a storage unit 708, and the like are realized. Alternatively, the information processing apparatus 101 may realize each of the above-mentioned functional configurations by executing a predetermined program on a plurality of computers 600.

It should be noted that at least a part of each of the above functional configurations may be realized by hardware.

The acquisition unit 701 acquires the temperature image data of the user transmitted via the network 104 from, for example, the camera 102 that captures the temperature image data indicating the temperature around the bed 103 and the bed 103.

The detection unit 702 detects a predetermined temperature in a plurality of detection regions by using the temperature image data of the user acquired by the acquisition unit 701. For example, the detection unit 702 has a predetermined temperature corresponding to, for example, the body temperature of the user 501 in the plurality of detection areas based on the area information 712 as shown in FIG. 8B stored in the storage unit 708. Is detected.

FIG. 8B shows an image of an example of the area information 712. In the example of FIG. 8B, the area information 712 includes information such as "area", "coordinate range", "temperature range", and "number of pixels" as items. The "area" is information indicating a number and a name (for example, detection area A, etc.) that identify the detection area. The "coordinate range" is an example of information indicating the range of each detection area. For example, when each detection area is rectangular, it is represented by coordinates indicating four vertices of the detection area. The information indicating the range of each detection area may be expressed in various formats such as, for example, a combination of columns and rows of a plurality of small areas, coordinates and radii of a center point, and the like.

The “temperature range” is an example of information indicating a predetermined temperature range to be detected. The information indicating the predetermined temperature range may be represented by the detection color of the temperature image data, for example, as shown in FIG. 8C. In the example of FIG. 8C, as an example, the detected color is represented by a color space of hue (H), luminance (L), and saturation (S). The color space is not limited to hue (H), brightness (L), and saturation (S), and is, for example, red (R), green (G), blue (B), brightness (Y), and blue. It may be represented by another color space such as the color difference (U) of (U) and the color difference (V) of red.

The "number of pixels" is information indicating the size of a region to be detected at a predetermined temperature. For example, the detection region of the region “1” in FIG. 8B is a color region corresponding to the temperature range “35 ° C. to 39 ° C.” within the coordinate range of the region “1” and has the number of pixels “30”. When there is a region having a size equal to or larger than that, the detection unit 702 determines that the predetermined temperature has been detected in the region “1”. Further, the detection region of the region "2" is a color region corresponding to the temperature range "35 ° C. to 39 ° C." within the coordinate range of the region "2" and has a size of the number of pixels "10" or more. When there is a region, the detection unit 702 determines that the predetermined temperature has been detected in the region "2". Similarly, the detection unit 702 determines whether or not a predetermined temperature has been detected in each of the other regions "3", "4", ....

The determination information management unit 703 indicates the determination content indicating the state of the user 501 when a predetermined temperature is detected in the preset first detection area and the second detection area among the plurality of detection areas. To manage.

For example, the determination information management unit 703 is used when a predetermined temperature is detected in the detection area A403 (first detection area) and the detection area B404 (second detection area) in FIG. 5C. It is stored and managed in the determination information 711 or the like that the state of the person is "a sign of getting up".

Preferably, when the determination information management unit 703 detects a predetermined temperature in the detection area A403 and the detection area B404 as shown in FIG. 5C from the "sleeping" state shown in FIG. 5A. The judgment content "precursor of getting up" is stored and managed in the judgment information 711 or the like.

FIG. 8A shows an image of an example of the determination information 711 stored in the storage unit 708 or the like by the determination information management unit 703. In the example of FIG. 8A, the determination information 711 includes information such as “determination content”, “determination condition”, “notification content”, and “priority” as items.

The "determination content" is information indicating the determination content indicating the state of the user when the detection result by the detection unit 702 satisfies the "determination condition". The "judgment condition" is information indicating the judgment condition for each "judgment content". The "notification content" is information indicating the notification content to be notified to a predetermined notification destination when the detection result by the detection unit 702 satisfies the "determination condition". The "priority" is information indicating the priority of a plurality of "determination contents".

An administrator or the like who manages the information processing apparatus 101 can display, for example, a setting screen as shown in FIG. 9 and set, for example, "determination content", "determination condition", and the like.

FIG. 9 is a diagram showing an example of a setting screen according to an embodiment. In the example of FIG. 9, the setting screen 900 includes a display field 901 for displaying a plurality of detection areas, a detection area setting field 902, a determination content setting field 903, and the like.

In the display field 901, as an example, the detection areas A403, the detection areas B404-1, 404-2, the detection area C405, the detection areas D406-1, 406.2, and the detection areas E407-1, 407-2 shown in FIG. The detection areas 1 to 8 corresponding to are displayed. For example, the detection area 1 corresponds to the detection area A403 of FIG. 4, and the detection areas 2 and 3 correspond to the detection areas B404-1 and 404-2 of FIG. Further, the detection area 4 corresponds to the detection area C405 of FIG. 4, and the detection areas 5 and 6 correspond to the detection areas D406-1 and 406.2 of FIG. Further, the detection areas 7 and 8 correspond to the detection areas E407-1 and 407-2 of FIG.

As a preferred example, the administrator or the like can change the position, size, etc. of each detection area in the display field 901 by a drag operation, a pinch-in / pinch-out operation, or the like.

In the detection area setting field 902, it is possible to create a detection area and set the "detection color", "priority", "number of pixels", etc. of each detection area.

In the determination content setting field 903, it is possible to add the determination content, set the "priority", "determination condition", "notification content", etc. of each determination content. In the "determination condition", "1: ON" indicates a state in which a predetermined temperature is detected in the detection area (detection area A403) of the area "1". Further, "1: OFF" indicates a state in which a predetermined temperature is not detected in the detection region of the region "1". The same applies to the other areas "2", "3", ... Further, in the "judgment condition", the arrow "→" indicates the transition of the state.

For example, in the determination content setting field 903 of FIG. 9, the determination condition of the determination content "sleeping" is that a predetermined temperature is detected in the detection area (detection area A403) of the area "1" and the areas "2" and ". It means that the predetermined temperature is not detected in the detection area of "3".

Further, as for the judgment condition of the judgment content "precursor of rising", the predetermined temperature is detected in the detection area of the area "1", and the predetermined temperature is not detected in the detection areas of the areas "2" and "3". It means that the state has changed from the state to the state in which the predetermined temperature is detected in the detection areas of the areas "1" and "2" (or "3"). By raising the "judgment order" of the "precursor of getting up" from the judgment of "sleeping", when the area "1" is detected and the areas "2" to "3" are detected, "getting up" from "sleeping" The judgment result transitions to "a precursor of".

The judgment information management unit 703 stores and manages the judgment conditions set in the judgment content setting field 903 in the judgment information 711 as shown in FIG. 8A, for example.

When the "judgment condition" stored in the judgment information 711 is detected by the detection unit 702, the information providing unit 704 determines the user information indicating the "judgment content" corresponding to the detected "judgment condition". Provide (notify) to the notification destination of.

For example, the detection unit 702 detects a predetermined temperature in the detection area A403 (area “1”) and the detection area B404 (area “2”) from the “sleeping” state as shown in FIG. 5 (A). And. In this case, in the determination information 711 of FIG. 8A, the determination condition of the determination content "precursor of rising" is satisfied, so the information providing unit 704 indicates that the state of the user is "precursor of rising". Provide (notify) user information to a predetermined notification destination.

The predetermined temperature in the region "1" (hereinafter referred to as the first temperature) and the predetermined temperature in the region "2" (hereinafter referred to as the second temperature) are at the same temperature (temperature range). It may be at different temperatures.

The area information management unit 705 stores and manages the area information 712 as shown in FIGS. 8B and 8C in the storage unit 708 or the like, for example. For example, the area information management unit 705 provides information such as "detection color", "priority", and "number of pixels" of the detection area set in the detection area setting field 902 of the setting screen 900 as shown in FIG. Is stored and managed in the area information 712.

The image data management unit 706 stores and manages the temperature image data acquired by the acquisition unit 701 in the storage unit 708 for each camera 102. For example, the image data management unit 706 stores the image data collected from the camera 102a in the image data 713a of the storage unit 708, and stores the image data collected from the camera 102b in the image data 713b of the storage unit 708. ..

Preferably, the image data management unit 706 deletes the image data stored in the storage unit 708 that has passed a predetermined storage period (for example, 15 minutes or the like).

Preferably, the image data in which the "change in temperature" is detected is not deleted but archived. Also, preferably, the archived image data is saved with administrator authority so that only the administrator can delete it.

For example, the display control unit 707 displays the setting screen 900 as shown in FIG. 9 on the display 606 or the like, and accepts the setting operation by the administrator or the like. Alternatively, the display control unit 707 may function as a web server that provides a web page for displaying the setting screen 900 as shown in FIG. 9, and may accept a setting operation for the setting screen 900.

The storage unit 708 is realized by, for example, a program executed by the CPU 601 of FIG. 6, an HD604, an HDD controller 605, a RAM 603, and the like, and various data and information such as determination information 711, area information 712, image data 713a, and 713b. Remember. The storage unit 708 may be realized by another information processing device (for example, a storage server or the like) different from the information processing device 101.

The functional configuration of the information processing device 101 shown in FIG. 7 is an example. For example, each functional configuration of the information processing device 101 of FIG. 7 may be distributed and arranged in a plurality of information processing devices. Further, at least a part of the functions provided by the information processing apparatus 101 may be provided by the nurse call system 121 or the like.

In the present embodiment, the nurse call system 121, the information terminal 123, the display device 122, etc. are examples of predetermined notification destinations, and are optional as long as the user information notified from the information processing device 101 can be displayed. Since the configuration of is sufficient, the description thereof is omitted here.

<Processing flow>
Subsequently, the processing flow of the information providing method according to the present embodiment will be described.

(Processing of information processing equipment)
FIG. 10 is a flowchart showing an example of processing of the information processing apparatus according to the embodiment.

In step S1001, when the acquisition unit 701 of the information processing apparatus 101 acquires the temperature image data of the user transmitted from the camera 102, for example, at predetermined time intervals, the processes after step S1002 are executed.

In step S1002, the detection unit 702 of the information processing device 101 detects a predetermined temperature from a plurality of detection regions of the temperature image data acquired by the acquisition unit 701. For example, the detection unit 702 detects a predetermined temperature from each of the plurality of detection areas as shown in FIG. 4 with reference to the area information 712 stored in the storage unit 708 as shown in FIG. 8 (B). To do.

In step S1003, the information providing unit 704 of the information processing apparatus 101 determines whether or not the detection result by the detection unit 702 satisfies, for example, the "determination condition" of the determination information 711 as shown in FIG. 8A. ..

When the detection result satisfies the determination condition, the information providing unit 704 shifts the process to step S1004. On the other hand, if the detection result does not satisfy the determination condition, the information providing unit 704 ends the process of FIG.

When the process proceeds to step S1004, the information providing unit 704 provides the determination information 711 as shown in FIG. 8A with the user's information indicating the "determination content" whose detection result satisfies the "determination condition". Notify the notification destination (for example, the nurse call system 121, the information terminal 123, the display device 120, etc.).

When the detection result satisfies a plurality of "judgment conditions", the information providing unit 704 determines that the "judgment" has the highest "priority" among the plurality of "judgment contents" in which the detection result satisfies the "judgment condition". Notify the user's information indicating "content" to the predetermined notification destination.

(Processing of information information system)
FIG. 10 is a sequence diagram showing an example of processing of the information processing system according to the embodiment. This process shows an example of the process of the entire information information system 100 corresponding to the process of the information processing apparatus described with reference to FIG.

In step S1101, when the camera 102 captures the temperature image data (temperature image data of the user) of the bed 103 corresponding to the camera 102 and the surroundings of the bed 103, the processes after step S1102 are executed.

In step S1102, the camera 102 transmits the captured temperature image data to the information processing apparatus 101. The temperature image data includes, for example, identification information for identifying the camera 102, the bed 103 corresponding to the camera 102, and the like. The camera 102 repeatedly executes the processes of steps S1101 and S1102 at predetermined time intervals (for example, at intervals of several seconds to several tens of seconds).

In steps S1103 and S1104, the acquisition unit 701 of the information processing apparatus 101 notifies, for example, the image data management unit 706 and the detection unit 702 of the temperature image data acquired from the camera 102.

In step S1105, the image data management unit 706 of the information processing apparatus 101 stores the image data in a folder or the like corresponding to the camera 102 based on the identification information included in the temperature image data notified from the acquisition unit 701.

Preferably, in step S1106, the image data management unit 706 deletes the image data that has passed the predetermined storage period from the image data stored in the folder or the like corresponding to the camera 102.

In step S1107, the detection unit 702 of the information processing device 101 detects a predetermined temperature from a plurality of detection areas of the temperature image data notified from the acquisition unit 701.

In step S1108, the detection unit 702 notifies the information providing unit 704 of the detection results of predetermined temperatures in the plurality of detection regions. This detection result includes the above-mentioned identification information for identifying the camera 102, the bed 103, and the like.

In step S1109, the information providing unit 704 of the information processing apparatus 101 determines whether or not the detection result by the detection unit 702 satisfies, for example, the "determination condition" of the determination information 711 as shown in FIG. 8A. ..

When the detection result by the detection unit 702 satisfies the determination condition, the processes of steps S1110 and S1111 are executed.

In step S1110, the information providing unit 704 calls the user's information indicating the "judgment content" whose detection result satisfies the "judgment condition" in the judgment information 711 as shown in FIG. 8A, for example. Notify system 121. In addition, above this user, "determination content" indicating the state of the user, identification information for identifying the camera 102, the bed 103, and the like are included.

In step S1111, the nurse call system 121 creates, for example, a display screen 1200 as shown in FIG. 12 based on the user information notified from the information processing device 101, and displays the information terminal 123, the display device 122, and the like. Display it.

FIG. 12 is a diagram showing an example of a display screen according to an embodiment. In the example of FIG. 12, the display screen 1200 includes the determination content 1201, the user information 1202, the user image 1203, and the like.

In the determination content 1201, information indicating the state of the user determined by the information processing device 101 is displayed. In the example of FIG. 12, the determination content 1201 indicates that the user indicates a “precursor of getting up”.

In the user information 1202, information about the user corresponding to the determination content 1201 is displayed. In the example of FIG. 12, the user information 1202 displays the user's hospital room number, bed number, user's name, and the like. The user information 1202 is, for example, associated with the identification information for identifying the camera 102, the bed 103, etc. managed by the information processing device 101 and the patient information notified from the nurse call system 121, and the information processing device 101. Creates.

In the user image 1203, for example, a temperature image of the user 501 based on the temperature image data of the user taken by the camera 102 in step S1011 of FIG. 11 is displayed. Further, in the image 1203 of the user, a display element 1204 indicating the position of the bed may be displayed in order to facilitate grasping the positional relationship of the user 501. Further, the image 1203 of the user may display display elements 1205, 1206, etc. indicating a detection area in which a predetermined temperature is detected.

With such a display screen 1200, for example, the staff of a medical facility or the like can recognize that the user A is showing a sign of getting up earlier.

<Detection of signs of getting up>
In order to display the display screen 1200 showing the sign of the rise of the user 501 as shown in FIG. 12, for example, the determination information 711 as shown in FIG. 8 (A) and FIGS. 8 (B) and 8 (C) ), It is premised that appropriate information is set in the area information 712.

Here, a specific embodiment for detecting a sign of rising of the user 501 will be described.

[First Embodiment]
In the first embodiment, for example, as shown in FIG. 5C, when a predetermined temperature (for example, the body temperature of the user 501) is detected in the detection area A403 and the detection area B404, the user An example will be described in which the state of 501 is determined to be a “precursor of rising”.

(Area information setting)
In the first embodiment, for example, in the area information 712 as shown in FIG. 8 (B) or FIG. 8 (C), at least the detection area A403 and the detection of the plurality of detection areas shown in FIG. Area B404 is set.

The "coordinate range" of the detection area A403 is set to include, for example, an area in which the pillow 402 of the user 501 is arranged, as shown in FIG. Since the position of the pillow 402 and the head 502 of the user 501 are likely to move, for example, as shown in FIG. 4, a sufficiently wide coordinate range is set with respect to the size of the pillow 402.

The "coordinate range" of the detection area B404 is, for example, as shown in FIG. 4, the user 501 lying on the bed 103 among the plurality of fences 401-1 to 401-3 provided around the bed 103. It is set to include a part or all of the fences 401-1 and 401-2 provided within reach.

For the temperature range of the detection area A403 (first temperature range) and the temperature range of the detection area B404 (second temperature range), for example, a temperature corresponding to the body temperature of the user 501 is set. The detection area B404 for detecting the hand 503 of the user 501 may be set to a wider temperature range in consideration of the case where the hand 503 of the user 501 is cold.

The "pixel count" of the detection area A403 is used to detect the size of a human head so as not to erroneously detect electronic devices such as mobile phones, smartphones, music players, etc. used by user 501, disposable body warmers, etc. It is desirable to determine the optimum size in advance by, for example, an experiment. On the other hand, the "pixel count" of the detection area B404 is smaller than the "pixel count" of the detection region A403 because it detects the hand 503 smaller than the head 502 and is not easily affected by electronic devices or disposable body warmers. It is desirable to set it to a value.

(Setting of judgment information)
In the first embodiment, for example, as in the determination information 711 shown in FIG. 8A, "determination conditions" corresponding to the determination content "precursor of rising" are set. In the example of FIG. 8A, two determination conditions are set for the "determination condition" corresponding to the determination content "precursor of rising".

The first determination condition is "1: ON → 1: ON and 2: ON". This is because, for example, in FIG. 4, from the "sleeping" state in which the first temperature is detected in the detection area A403, the first temperature is detected in the detection area A403 and the second temperature is detected in the detection area B404-1. It indicates that when the temperature is detected, it is judged as "a sign of getting up".

The second determination condition is "1: ON → 1: ON and 3: ON". This is because, for example, in FIG. 4, from the "sleeping" state in which the first temperature is detected in the detection area A403, the first temperature is detected in the detection area A403 and the second temperature is detected in the detection area B404-2. It indicates that when the temperature is detected, it is judged as "a sign of getting up".

Depending on the state of the bed 103 and the user 501, only one of the first determination condition and the second determination condition may be set in the determination information 711.

<Processing flow>
Subsequently, the flow of the determination process according to the first embodiment will be described.

FIG. 13 is a flowchart showing an example of the determination process according to the first embodiment. This process shows an example of a determination process for determining the "precursor of getting up" of the user 501, which is executed by the information providing unit 704 of the information processing apparatus 101 when the state of the user 501 is "sleeping". There is.

In step S1301, the information providing unit 704 of the information processing apparatus 101 executes the process of step S1302 when it acquires the detection results of the predetermined temperatures in the plurality of detection regions from the detection unit 702.

In step S1302, the information providing unit 704 is a detection area A403, and is a “temperature range” (hereinafter, referred to as a first temperature) corresponding to the detection area A403 (for example, the area “1”) set in the area information 712. Is detected or not.

When the first temperature is detected in the detection area A403, the information providing unit 704 shifts the process to step S1303. On the other hand, when the first temperature is not detected in the detection area A403, the information providing unit 704 ends the process of FIG.

When the process proceeds to step S1303, the information providing unit 704 sets the detection area B404 to have a “temperature range” (hereinafter, the first) corresponding to the detection area B404 (for example, the areas “2” and “3”) set in the area information 712. It is determined whether or not (called the temperature of 2) is detected.

When the second temperature is detected in the detection area B404, the information providing unit 704 shifts the process to step S1304. On the other hand, when the second temperature is not detected in the detection area B404, the information providing unit 704 shifts the process to step S1305.

When the process proceeds to step S1304, the information providing unit 704 determines that the state of the user 501 is a "precursor of getting up".

On the other hand, when the process proceeds to step S1305, the information providing unit 704 determines that the user's state is "sleeping" and returns the process to step S1301.

By the above processing, in the information processing device 101 that detects the state of the user 501 by using the temperature image data of the user 501 that uses the bed 103, the "precursor of getting up" by the user 501 while suppressing the false detection. Will be able to detect.

[Second Embodiment]
In the second embodiment, for example, as shown in FIG. 5D, when a predetermined temperature (for example, the body temperature of the user 501) is detected in the detection area A403 and the detection area E407, the user 501 An example will be described in which the state of is judged to be a “precursor of rising”.

(Area information setting)
In the second embodiment, for example, in the area information 712 as shown in FIG. 8 (B) or FIG. 8 (C), at least the detection area A403 and the detection area among the plurality of detection areas shown in FIG. E407 is set.

The “coordinate range” of the detection area A403 may be the same as that of the first embodiment. The "coordinate range" of the detection area E407 is set so that, for example, as shown in FIG. 5D, the foot 504 enters the detection area E407 when the foot 504 of the user 501 comes out next to the bed 103. An area adjacent to the side of the bed 103 is set.

For the temperature range of the detection area A403 (first temperature range) and the temperature range of the detection area E407 (second temperature range), for example, a temperature corresponding to the body temperature of the user 501 is set. The detection area E407 that detects the foot 504 of the user 501 may be set to a wider temperature range in consideration of the case where the foot 504 of the user 501 is cold.

The “number of pixels” of the detection area A403 may be the same as that of the first embodiment. On the other hand, the "number of pixels" of the detection area E407 is preferably set to a value smaller than the "number of pixels" of the detection area A403 because it detects the foot 504 smaller than the head 502.

(Setting of judgment information)
In the second embodiment, for example, the following two determination conditions are set in the "determination condition" corresponding to the determination content "precursor of rising" in the determination information 711 as shown in FIG. 8A.

The first determination condition is "1: ON → 1: ON and 7: ON". For example, in FIG. 4, from the "sleeping" state in which the first temperature is detected in the detection area A403, the first temperature is detected in the detection area A403 and the second temperature is detected in the detection area E407-1. It indicates that when the temperature is detected, it is judged as "a sign of getting up".

The second determination condition is "1: ON → 1: ON and 8: ON". This is because, for example, in FIG. 4, from the "sleeping" state in which the first temperature is detected in the detection area A403, the first temperature is detected in the detection area A403 and the second temperature is detected in the detection area E407-2. It indicates that when the temperature is detected, it is judged as "a sign of getting up".

Depending on the state of the bed 103 and the user 501, one of the first determination condition and the second determination condition may be set in the determination information 711.

<Processing flow>
FIG. 14 is a flowchart showing an example of the determination process according to the second embodiment. This process is another example of a determination process for determining the "precursor of getting up" of the user 501, which is executed by the information providing unit 704 of the information processing device 101 when the state of the user 501 is "sleeping". Shown.

In step S1401, the information providing unit 704 of the information processing device 101 executes the process of step S1402 when it acquires the detection results of predetermined temperatures in the plurality of detection regions from the detection unit 702.

In step S1402, in the detection area A403, the information providing unit 704 has detected a "temperature range" (first temperature) corresponding to the detection area A403 (for example, the area "1") set in the area information 712. Judge whether or not.

When the first temperature is detected in the detection area A403, the information providing unit 704 shifts the process to step S1403. On the other hand, when the first temperature is not detected in the detection area A403, the information providing unit 704 ends the process of FIG.

When the process proceeds to step S1403, the information providing unit 704 determines whether or not the "temperature range" (second temperature) corresponding to the detection area E407 set in the area information 712 is detected in the detection area E407.

When the second temperature is detected in the detection area E407, the information providing unit 704 shifts the process to step S1404. On the other hand, when the second temperature is not detected in the detection area E407, the information providing unit 704 shifts the process to step S1405.

When the process proceeds to step S1404, the information providing unit 704 determines that the state of the user 501 is a “precursor of getting up”.

On the other hand, when the process proceeds to step S1405, the information providing unit 704 determines that the user's state is "sleeping" and returns the process to step S1401.

By the above processing, in the information processing device 101 that detects the state of the user 501 by using the temperature image data of the user 501 that uses the bed 103, the "precursor of getting up" by the user 501 while suppressing the false detection. Will be able to detect.

The information processing device 101 can apply the second embodiment in combination with the first embodiment.

As described above, according to each embodiment of the present invention, the information processing device 101 that detects the state of the user 501 using the temperature image data of the user 501 that uses the bed 103 detects a sign of getting up by the user. , It becomes possible to notify a predetermined notification destination.

In addition, the information processing device 101 according to the present embodiment mistakenly determines that the user has a "precursor of getting up" when the user 501 performs an operation other than getting up, such as turning over. It is possible to reduce the amount of storage.

Therefore, according to the present embodiment, in the information processing device 101 that detects the state of the user 501 by using the temperature image data of the user 501 that uses the bed 103, the user 501 "" while suppressing erroneous notification. You will be able to notify "a sign of getting up".

The bed 103 is an example of a predetermined facility, and the “precursor of getting up” is an example of a precursor of a predetermined operation.

Further, in each embodiment of the present invention, the case where the temperature is detected in a plurality of predetermined detection regions is described as having a "precursor of rising", but when the temperature is detected at the same time, the "precursor of rising" is described. May be present.

<Supplement>
Each function of each embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

In addition, the device group described in the examples is only one of a plurality of computing environments for implementing the embodiments disclosed in the present specification. In certain embodiments, the information processing apparatus 101 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including networks, shared memory, and the like, and perform the processes disclosed herein. Similarly, the nurse call system 121 can include a plurality of computing devices configured to communicate with each other.

Further, the information processing apparatus 101 and the nurse call system 121 can be configured to share the disclosed processing steps, for example, FIGS. 10, 11, 13, 14 and the like in various combinations. For example, a process executed by a predetermined unit may be executed by the information processing apparatus 101. Similarly, the function of a given unit can be performed by the nurse call system 121. Further, the functions of the information processing device 101 and the nurse call system 121 may be integrated into one server device or may be divided into a plurality of devices.

100 Information processing system 101 Information processing equipment 102 Camera 103 Bed (example of equipment)
121 Nurse call system (an example of a predetermined notification destination)
401, 401-1 to 401-3 Fence 402 Pillow 403 Detection area A (first detection area)
404, 404-1, 404-2 Detection area B (example of second detection area)
407, 407-1, 407-2 Detection area E (another example of the second detection area)
501 User 703 Judgment Information Management Department 704 Information Providing Department

Japanese Unexamined Patent Publication No. 2019-03628

Claims (8)

An information processing device that detects the state of the user using temperature image data of the user who uses the bed .
Of the bed and the plurality of detection areas provided around the bed , the first detection area, which is the area where the pillow used by the user is arranged, and a part around the bed are provided. When the information indicating the size of the color region corresponding to the predetermined temperature range is simultaneously detected in the second detection region, which is the region where the fence is arranged, in the region larger than the predetermined size. Judgment information management unit that manages the judgment contents indicating the state of the user
In the temperature image data, when information in a region having a predetermined size or larger is simultaneously detected in the first detection region and the second detection region, the information of the user indicating the determination content is displayed. Information provision department to provide and
An information processing device that has. The bed comprises a plurality of fences provided around the bed.
The second detection area includes a part or all of the area in which one or more preset fences are arranged among the plurality of fences.
The information processing device according to claim 1 . In the temperature image data, the information providing unit detects information in a region having a predetermined size or larger corresponding to a first temperature range preset in the first detection region, and the second detection region . Information of the user indicating that the user has a sign of getting up when information in a region having a predetermined size or larger corresponding to a second temperature range set in advance is detected in the detection region of The information processing apparatus according to claim 1 or 2 , which notifies a predetermined notification destination. The information providing department
In the temperature image data, after information in a region having a predetermined size or larger corresponding to a first temperature range preset in the first detection region is detected, the temperature image data is described in the first detection region. Information in a region of the predetermined size or larger corresponding to the first temperature range is detected, and the predetermined size or more corresponding to the second temperature range preset in the second detection region. The information processing apparatus according to claim 1 or 2 , wherein when the information in the area of is detected, the information of the user indicating that the user has a sign of getting up is notified to a predetermined notification destination. In the temperature image data, the information providing unit has information on a region having a predetermined size or more corresponding to the first temperature range and information having a size equal to or larger than the predetermined size corresponding to the second temperature range. The information processing device according to claim 3 or 4 , wherein when the information of the area is detected at the same time, the information of the user indicating that the user has a sign of getting up is notified to a predetermined notification destination. An information processing system that detects the state of the user using temperature image data of the user who uses the bed .
Of the bed and the plurality of detection areas provided around the bed , the first detection area, which is the area where the pillow used by the user is arranged, and a part around the bed are provided. When the information indicating the size of the color region corresponding to the predetermined temperature range is simultaneously detected in the second detection region, which is the region where the fence is arranged, in the region larger than the predetermined size. Judgment information management unit that manages the judgment contents indicating the state of the user
In the temperature image data, when information in a region having a predetermined size or larger is simultaneously detected in the first detection region and the second detection region, information on the user indicating the determination content is provided. Information provision department and
An information processing system that has. An information processing device that detects the state of the user using the temperature image data of the user who uses the bed
Of the bed and the plurality of detection areas provided around the bed , the first detection area, which is the area where the pillow used by the user is arranged, and a part around the bed are provided. When the information indicating the size of the color region corresponding to the predetermined temperature range is simultaneously detected in the second detection region, which is the region where the fence is arranged, in the region larger than the predetermined size. The process of managing the judgment content indicating the state of the user and
In the temperature image data, when information in a region having a predetermined size or larger is simultaneously detected in the first detection region and the second detection region, information on the user indicating the determination content is provided. Processing to do and
How to provide information. A program for causing an information processing apparatus to execute the information providing method according to claim 7 .

Images