JP2006055388A - Remote support system for nursing care - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a remote support system for nursing care which reduces a burden on patients suffering from total body paralysis and aphasia, other challenging persons and care givers, and enables smooth care and communication. <P>SOLUTION: The remote support system for nursing care has a camera 16 taking moving images and a client computer 18 processing images taken by the camera 16. The client computer 18 includes an eye position recognizing means for detecting the position of eyes 44 of the person in need of nursing care and constitutes an entry system 4 for the person in need of nursing care, and being connected to a display 14 on the side of the person in need of nursing care with an operational icon manipulated by the person in need of nursing care and a display part for a reply from care givers and to an infrared remote control 22 connected to the client computer 18. A display 30 on the side of care givers and a server computer 28 replying to messages from the person in need of nursing care constitute a confirmation system 6 for care givers. The entry system 4 and the confirmation system 6 are connected by a communication network. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a remote care support system that is used for nurse call systems, home care, etc. of hospitals and special elderly nursing facilities, and for providing care for severely disabled persons and the elderly.

  Conventionally, in hospitals and special elderly nursing facilities, etc., when a cared person wants to ask for a service or when he / she is not feeling well, a nurse call is made by a push button provided on the side of the bed. In response to this call, a nurse or the like generally checks the person who is calling with a display board or the like at a nurse station, and listens to the request through an intercom to deal with the call. In such a nurse call system, verbal communication is generally used as a method for listening to a message from a calling person.

  However, when communicating with patients and persons with disabilities, it is difficult to communicate with persons with disabilities such as myasthenia gravis, amyotrophic lateral sclerosis, general paralysis due to stroke, and aphasia. It is also difficult to use communication devices that use limbs and mouth instead of words.

In order to solve this problem, a method of communicating with eyes is known. As a communication method by eyes, there is a blinking switch in which infrared rays are applied to an eyeball and the switch is turned on / off depending on the difference in the amount of infrared reflection when the eye is opened and when the eye is closed. There is also a computer operation tool that detects the line of sight and inputs to the computer. In addition, there is a transparent dial in which the cared person and the caregiver face each other across a transparent dial, and the carer reads the characters on the dial from the caregiver's line of sight.
JP 2003-150942 A JP-A-11-332831

  The above-mentioned conventional blink switch is troublesome for the user because a blink detection device is mounted in front of the eyes, and the line-of-sight input device requires relative correction of the corneal reflection point and the pupil center. Therefore, it is difficult to obtain high accuracy, and there is a problem of erroneous input. In addition, since the eyeball is irradiated with infrared rays, there is a problem with the eyes when used for a long time. Furthermore, there is also the trouble of fixing the head to the headrest. In addition, the computer operation support tool for persons with disabilities needs to prepare operation conditions and the like separately for each individual. In addition, in the method of attaching any device to the eyes or head, even a person with limited limbs can operate the computer, but it is bothersome to wear the device every time the computer is operated. There was something wrong with wearing it. In addition, there is a drawback that stress tends to accumulate when used for a long time.

  In addition, the transparent dial that reads characters from the caregiver's line of sight has a problem that the caregiver must be present when the caregiver wants to communicate with the caregiver.

  Furthermore, Patent Document 2 discloses a self-supporting care system based on image processing. In this case, the caregiver is only communicated to the caregiver who is accompanied by the cared person, and the caregiver is close to the cared person. It functions only when it is present, and is not applicable when intensive care for a large number of people in a hospital or a special nursing home for the elderly.

  The present invention has been made in view of the problems of the prior art described above, and is a remote care support system that can reduce the burden on patients with general paralysis and aphasia, other disabled persons and caregivers, and enables smoother care and communication. The purpose is to provide.

  The present invention relates to an input system for one or more care recipients having a client computer capable of image processing installed on the care receiver side, and a caregiver confirmation system including a server computer to which the client computer is connected. The care receiver input system includes a camera capable of capturing moving images, an operation icon connected to the client computer and operated by the care receiver, and a response display from the caregiver. And an infrared remote controller connected to the client computer, an image captured by the camera is input to the client computer, and the client computer captures image information of the face of the cared person captured by the camera. To a template that falls within the iris size of the human eye The caregiver confirmation system comprises eye position recognition means for detecting the position of the eye of the cared person by scanning the image while detecting the shade of the image of the iris part of the eye and the other part of the face. A caregiver-side display and an input device for displaying a message from the care recipient are connected to the server computer, and a response to the message from the caregiver displayed on the caregiver-side display is performed by the input device. And the response is displayed on the care receiver display, the care receiver input system and the caregiver confirmation system are connected via a communication network, and the caregiver confirmation system is connected to the care receiver. A remote care support system having a communication line connection function for calling a registered contact in response to a request from an input system for a person.

  The care receiver input system uses the eye position information captured by the eye position recognizing means, detects that the eye has stopped in a predetermined range, and is displayed on the care receiver display. Is provided with an input processing function. Further, the infrared remote controller is operated by operating icons displayed on the care receiver display, for example, turning off and turning on indoor lighting, switching TV channels, switching air conditioning operation, and adjusting temperature, and this operation is performed. The detection by the eye position recognizing means is performed by the processing of the client computer.

  Further, the caregiver confirmation system has an external transfer function for transferring a request from the care recipient to a registered telephone through a public line when there is no response from the input device on the caregiver side. It is.

  According to the remote care support system of the present invention, even when a cared person such as a patient or a disabled person and a caregiver are separated from each other, the two-way communication between them can be easily and appropriately performed, and smooth and detailed care can be performed. Is something that can be done.

  Hereinafter, an embodiment of a remote care support system according to the present invention will be described with reference to FIGS. The remote care support system 2 of this embodiment is used as a system for supporting care for a handicapped person with a disability such as myasthenia gravis, amyotrophic lateral sclerosis, general paralysis due to stroke, aphasia and the like. The

  The configuration of the remote care support system 2 of this embodiment will be described. The remote care support system 2 includes a care receiver input system 4 and a caregiver confirmation system 6. First, the configuration of the care receiver input system 4 in FIG. 1 will be described. In the care receiver input system 4, a metal display support 12 is attached to the center part of the handrail 10 on the head side of the bed 8 on which the care recipient who is a patient or a disabled person is sleeping by a fixture (not shown). It has been. The support 12 can be adjusted in refraction, and a display 14 is attached to one end by a fixture (not shown). A video camera 16 is provided above the display 14. The orientation of the video camera 16 is adjusted so as to image the vicinity of the eyes of the cared person sleeping on the bed. A connecting line between the video camera 16 and the display 14 is provided from an attachment portion of the support 12 attached to the handrail 10 of the bed 8, and this connecting line is connected to a client computer 18 composed of a personal computer placed beside the handrail 10. Has been. In addition, an infrared remote controller 22 is placed on the bed 20 side, and the infrared remote controller 22 and the client computer 18 are also connected by a connection line.

  Furthermore, the client computer 18 is connected to a LAN (Local Area Network). A normal screen 24 in FIG. 2 and a local screen 26 in FIG. 3 can be displayed on the display 14 connected to the client computer 18. The normal screen 24 of FIG. 2 includes various request operation icons 24a for caregivers such as nurses, and further includes a response display field 24b for displaying a response to the request. In the local screen 26 of FIG. 3, an illumination control operation icon 26a, an air conditioning setting operation icon 26b, and a television operation icon 26c are prepared. These operation icons are the same as well-known icons that function when the cursor on the screen is moved and a predetermined operation such as a click is performed.

  Next, the configuration of the caregiver confirmation system 6 in FIG. 1 will be described. The caregiver confirmation system 6 includes a server computer 28, a display 30, and a keyboard and mouse (not shown) that are input devices. The server computer 28 is connected to a LAN and also connected to a telephone line. A telephone line connected to the server computer 28 is connected to a telephone network of a mobile phone 34, for example, via an exchange 32. A base station 36 of a mobile phone 34 and the like are connected to the telephone network so that a call can be made with the mobile phone 34.

  A server screen 38 of FIG. 4 is displayed on the display 30 connected to the server computer 28. The server screen 38 in FIG. 4 includes a status column 38a, a name column 38b, a request column 38c for contents for requesting assistance, a confirmation operation icon 38d for a request, and a history column 38e indicating the connection status for each care recipient's room. Have.

  The client computer 18 of the care receiver input system 4 and the server computer 28 of the caregiver confirmation system 6 are respectively connected on the LAN. The LAN is constructed by, for example, Ethernet (registered trademark), and uses TCP / IP as a network communication procedure. For data transmission / reception, application communication on the client computer 18 side is used as a client, and application software on the server computer 28 side is used as a server, and socket communication is performed by a server / client that is currently widely used.

  Next, an eye tracking method and an eye retention method used in the remote care support system 2 of the present embodiment will be described. First, an eye position detection method and a tracking method will be described. The face grayscale image acquired by the video camera 16 is binarized by threshold processing, and template matching is performed on the binarized image using the one-eye partial template 40 shown in FIG. For this template matching, a residual successive detection method is used.

In the residual successive detection method, the difference from the template data is sequentially obtained for image information having the number of pixels of L × L pixels. As shown in FIG. 6, the matching evaluation value E (p, q) between the template t (x, y) and the partial input image f (x, y) on the captured image template is calculated by the following equation (1). In this method, the processing is aborted assuming that there is no correlation when a predetermined threshold value is exceeded, and the same position is calculated at the next position to find a matching position.
E (p, q) = Σ ^M−1 _{x = 0} Σ ^M−1 _{y = 0} | f (x + p, y + q) −t (x, y) | (1)
However, 0 ≦ p ≦ LM + 1, 0 ≦ q ≦ LM + 1

  In general, as shown in FIG. 5, the diameter of the cornea of the adult eye 44 is about 11 mm (length: 9.3 to 11 mm, width: 10.6 to 12 mm). The iris 42, which is a black eye, is located under the cornea, and its size substantially matches the cornea size. Therefore, in this embodiment, the size of the iris 42 is set to 11 mm in diameter. The one-eye portion template 40 is configured by a cross-shaped straight line, has a horizontal width of 11 mm, a vertical width of 6 mm, and a line width of 1 pixel on the screen of the display 14. The number of pixels per mm is calculated based on the number of pixels by drawing a straight line of a certain length on the display 14. In this embodiment, 1 mm is 4.2 pixels, 11 mm is about 46 pixels, and 6 mm is about 25 pixels. By using such a cross-shaped template 40, even if the eyes 44 are inclined, the circular iris 42 is not affected, and accurate matching with the iris 42 can be obtained.

  In this embodiment, template matching is performed by moving the one-eye portion template 40 by four pixels for speeding up. When the matching degree of 90% or more of the horizontal width and 40% or more of the vertical width is satisfied at the same time, the matching degree of the horizontal width of the template 40 of one eye part is compared pixel by pixel and the matching degree of the vertical width is compared with skipping one pixel. Judgment is likely.

  Further, the template matching by the one-eye partial template 40 may erroneously detect some prominent regions on the image. As a countermeasure against the mistake, as shown in FIG. 7, the features around the eyes 44 were used as check items. In this embodiment, the reference diameter of the iris 42 is 11 mm. Here, as shown in FIG. 7, the presence or absence of the eyebrows 48 above and below the template 40 is used as a determination material. For example, when the iris 42 is 11 mm, the check point a at the position of the eyebrow 48 is between 25 to 50 mm above the center of the template 40 and the eyebrow 48 and the eye 44 in consideration of differences due to individual differences in human face. The check point b between 15 and 27 is also 15 to 27 mm, and the check point c below the eye 44 is also 15 to 22 mm. The check point b is determined by comparing with the threshold value based on the binarized pixel values. As a result, the determination of eye detection from the binarized image data indicates that there is no black portion immediately above the eye 44, the black portion by the eyebrow 48 is detected above, and the black portion is not below the eye 44. Can be a material. Further, as shown by a two-dot chain line in FIG. 7, even when the face image is enlarged, the eye position can be accurately determined by detecting the eyebrows 48 and the portion below the eyebrows 48 within a certain range. .

  The degree of matching of these check items is 10% or more at the position of the eyebrow 48, 20% or more at the position between the eyebrow 48 and the eye 44, and 20% at the position below the eye 44 in the portion above the eye 44. As described above, when the matching of the check items is obtained, it is determined that the condition suitability of the eyes 44 is satisfied. Then, the final determination that the eye is 44 is made by template matching and feature check items around the eye 44. Thus, even when the face image acquired by the camera 16 is enlarged or reduced, the position of the eye 44 can be accurately determined by detecting the eyebrow 48 and the portion below the eyebrow 48 within a certain range.

  Next, a description will be given of a stopping method in which the eye position detected by the eye tracking system is made to correspond to the cursor and it is determined that the operation icon on the screen has been clicked by stopping the eye. FIG. 8 shows a flowchart up to the stop determination of the stop method. Further, FIG. 9 shows a state of click determination based on eye stoppage. First, as described above, the face image captured by the camera 16 is binarized, and eyes are detected by the above-described eye tracking. Then, when the eyes are caught by tracking the eyes, as shown in FIG. 9A, the eye position 54 within a predetermined time is detected using the detected eye position as a reference point 52, and the stop position is within a predetermined setting range. The range s is measured. As shown in FIG. 9B, when the eye position 54 is out of range 56, the stationary measurement is reset. If the eye position 54 is stopped within the predetermined range s and the stop time of the eye position 54 is shorter than the threshold Th, a new image frame is captured. Further, as shown in FIG. 9A, when the eye retention time exceeds the threshold value Th and the eye position 58 is within the range s, it is determined that the client computer 18 is clicked by the input device.

  Next, the operation of this embodiment will be described. First, with the care receiver input system 4 on the left in FIG. 1, the video camera 16 captures an image of the face of the care receiver looking at the display 14. Eyes are detected from the captured face image, and the cursor on the screen of the display 14 is moved in accordance with the eye position from the detected eye position information. Next, when it is confirmed that the eye has stopped for a certain time or more as described above, the operation icon on the display 14 screen on which the cursor is overlapped is processed. Hereinafter, this process will be described.

  First, the case where the display 14 screen is the normal screen 24 shown in FIG. 2 will be described. The processing data is transmitted to the server computer 28 through the LAN, and the server computer 28 that has received the data reflects the care receiver's request on the display 30 screen of the server computer 28. As shown in FIG. 4, the nurse confirms the content displayed on the display 30 connected to the server computer 28, and clicks the confirmation operation icon 38d on the display 30 screen by operating the mouse. Then, the data is transmitted to the client computer 18 of the care receiver input system 4 and the response is displayed on the display 14 connected to the client computer 18. At this time, if the nurse is absent, the phone number such as a mobile phone or PHS that is registered in advance is automatically called via the telephone network, and in the order of the registered phone numbers, Call until you get a response. If all the registered telephone numbers do not appear or if the telephone number is cut off immediately, the care recipient's request is displayed again on the display 30 screen connected to the server computer 28. This operation is repeated until the confirmation operation icon is pressed by the nurse or the like.

  Next, the case where the display 14 screen of the care receiver input system 4 is the local screen 26 shown in FIG. 3 will be described. In this case, the functions of the various operation icons 26a, 26b, and 26c selected by stopping the eyes are processed, and the client computer 18 controls the infrared remote controller 22 to perform remote operations such as lighting, air conditioning, and television. .

  According to the remote care support system 2 of this embodiment, the burden on a patient with general palsy and aphasia, a disabled person, and a caregiver is reduced, and the caregiver and the cared person can perform smoother care and communication. is there. Moreover, since the network by the existing socket communication is used for the data communication between the care receiver input system 4 and the caregiver confirmation system 6, two-way communication between the care receiver and the caregiver is easily constructed. It is possible to respond even if the caregiver is away.

  In addition, since the click operation can be performed by stopping the eyes, various operations can be performed with a slight movement of the eyes and head, and communication such as conversation using the limbs is difficult due to symptoms such as general paralysis and aphasia. It can also be used by people with severe disabilities. In addition, it is non-contact, it can reduce fatigue even when it is used for a long time without wearing the head or eyes, and it does not irradiate the eyes with infrared rays, etc., so the burden on the eyes is small. .

  In addition, since the one-eye partial template method is used for the eye position detection method, assuming that the iris 42 is a circle with respect to the rotation change in the same plane, the template matching by the one-eye partial template 40 is performed even if it rotates 360 degrees. There is no mismatch. Further, in the measured scale enlargement change of the eye, the diameter of the iris on the image is greater than or equal to a predetermined value, so there is no mismatch. Furthermore, the scale reduction change corresponds to 90%. In this embodiment, template matching is performed by moving the one-eye portion template by four pixels for speeding up. The matching degree of the horizontal width of the one-eye part template is checked pixel by pixel, and the matching degree of the vertical width is checked by skipping one pixel. When the matching degree of 90% or more of the horizontal width and 40% or more of the vertical width is satisfied at the same time, it is determined to be prominent.

  The remote care system of the present invention is not limited to the above embodiment, and the client computer of the care receiver input system may be integrated with a display. In this case, it is assumed that the support for attaching the display has sufficient strength. Further, the display may be placed on a shelf or a stand at a position where the eyes of the cared person can reach instead of using a support as shown in FIG. At this time, a video camera is installed so that the face of the care recipient can be imaged.

  In addition, since the application software can be ported, the network medium constituting the LAN can be selected as appropriate. In this case, at the same time, a network communication procedure instead of TCP / IP and a socket interface and a network API (Application Programming Interface) may be used as appropriate.

It is a system configuration schematic diagram of a remote care support system of one embodiment of this invention. It is the schematic which shows the normal screen of the care receiver input system of this embodiment. It is the schematic which shows the local screen of the care receiver input system of this embodiment. It is the schematic which shows the server screen of the confirmation system for caregivers of this embodiment. It is the schematic which shows the one-eye part template of this embodiment. It is the schematic which shows the relationship between the image of a residual successive detection method of this embodiment, and a template. It is the schematic which shows the position of the check point of the peripheral region feature of the eye of this embodiment. It is a schematic sectional drawing which shows the flowchart of the retention system of this embodiment. It is a schematic explanatory drawing which shows the mode of the eye stop state of this embodiment.

Explanation of symbols

2 care support system 4 care receiver input system 6 caregiver confirmation system 12 support 14, 30 display 16 video camera 18 client computer 22 infrared remote control 28 server computer 32 exchange 34 mobile phone 40 template 42 iris 44 eyes

Claims (4)

An input system for one or more care recipients having a client computer capable of image processing installed on the care receiver side, and a confirmation system for a caregiver provided with a server computer to which the client computer is connected,
The care receiver input system includes a camera capable of capturing moving images, a care receiver side display having an operation icon connected to the client computer and operated by the care receiver, and a response display from the caregiver. An infrared remote controller connected to the client computer, an image captured by the camera is input to the client computer, and the client computer captures image information of the face of the cared person captured by the camera. Eye position that detects the position of the cared person's eye by scanning the image while comparing it with a template that falls within the range of eye iris size, and detecting the shade of the image of the iris part of the eye and other parts of the face With recognition means,
In the caregiver confirmation system, a caregiver-side display and an input device for displaying communication items from the caregiver are connected to the server computer, and communication from the caregiver displayed on the caregiver-side display. A response to the matter can be made by the input device, and the response is displayed on the care receiver side display,
The care receiver input system and the caregiver confirmation system are connected via a communication network, and the caregiver confirmation system is a communication for calling a registered contact in response to a request from the care receiver input system. A remote care support system characterized by a line connection function.   The care receiver input system uses the eye position information captured by the eye position recognizing means, detects that the eye has stopped in a predetermined range, and is displayed on the care receiver display. The remote care support system according to claim 1, further comprising an input processing function according to claim 1.   The infrared remote controller is operated by an operation icon displayed on the care receiver display, and this operation is performed by the processing of the client computer by detecting the stop of the eye position by the eye position recognition means. The remote care support system according to claim 2, wherein: The caregiver confirmation system includes an external transfer function for transferring a request from the care recipient to a registered telephone through a public line when there is no response from the caregiver's input device. The remote care support system according to claim 1.

Images