JP7113045B2 - display controller - Google Patents

Description

The present invention relates to a display control device.

2. Description of the Related Art Devices that provide various types of information are conventionally known. These devices are provided at the bedside, for example, so that patient information can be conveyed appropriately. In addition, there are known inventions that display various types of information to the user during rehabilitation (see, for example, Patent Documents 1 and 2).

In addition, as in Patent Document 3, in the medical field, an invention that provides information to medical staff by displaying prohibited medical practices, and in the case of the nursing care field, provides information to nursing care staff. Inventions are known that provide appropriate nursing care and reduce risks to the person being assisted.

Japanese Patent Application Laid-Open No. 2003-260150 Japanese Patent Publication No. 2010-517731 JP 2009-271574 A

However, if information is given indiscriminately, information will be displayed without distinguishing between patients and staff, and information should not be given to those who do not need it. There was also the problem of giving information to the target audience.

In particular, in the case of medical information, there is information that should be communicated only to medical staff or nursing care staff, but there are cases where this is communicated to patients and their families, causing unnecessary anxiety. In addition, in the field of nursing care, there is a problem that providing too much information to the elderly, etc., leads to unexpected behavior and risks due to confusion and misunderstanding.

In view of the above-described problems, an object of the present invention is to provide a display control device capable of appropriately providing information to a target person to whom information is to be provided by appropriately determining a target person to whom information is to be provided by sensing. etc. is to be provided.

In view of the problems described above, the display control device of the present invention
display means capable of displaying a message;
Characteristic determination means for determining characteristics of a target person existing in the recognition area by sensing;
When the characteristic determination means determines that the target person is a display target person, control is performed to display a message on the display means, and when it is determined that the target person is other than the display target person a display control means for controlling not to display a message on the display means;
characterized by comprising

In addition, the program of the present invention can be stored in a computer by
A display function that can display messages,
A characteristic determination function that determines the characteristics of the target person existing in the recognition area by sensing;
If the characteristic determination function determines that the target person is a display target person, control is performed to display a message on the display function, and if the target person is determined to be other than the display target person has a display control function that controls not to display a message on the display function;
It is characterized by realizing

A display control method of the present invention includes:
a display step capable of displaying a message;
a characteristic determination step of determining, by sensing, a characteristic of a target person existing in the recognition area;
When the characteristics determination step determines that the target person is a display target person, control is performed to display a message in the display step, and when it is determined that the target person is other than the display target person a display control step for controlling not to display a message in the display step;
characterized by having

The display system of the present invention is
A display system including a recognition device that senses and recognizes a target person, a display device that can display a message, and a control device,
The control device is
a characteristic determination means for recognizing a target person existing in a recognition area by the recognition device and determining a characteristic of the target person;
When the characteristic determination means determines that the target person is a display target person, control is performed to display a message on the display device, and when the target person is determined to be other than the display target person display control means for controlling not to display a message on the display device;
characterized by comprising

According to the present invention, the characteristics of the target person present in the recognition area are determined by sensing, and when it is determined that the target person is a display target person, control is performed to display a message on the display means, and the target person is When it is determined that the person is not the person to be displayed, control can be performed so that the message is not displayed on the display means. Therefore, by sensing, it is possible to give information to a target person who is determined to be a staff, for example, as a display target person. In addition, since the information is not communicated to anyone other than the person to whom the information is to be displayed, it is possible to avoid unnecessary confusion and to prevent unnecessary anxiety.

It is a figure for demonstrating the whole in 1st Embodiment. 3 is a diagram for explaining a functional configuration in the first embodiment; FIG. It is a figure for demonstrating the recognition method of the human body in 1st Embodiment. 4 is a diagram for explaining a display area in the first embodiment; FIG. It is an example of the data configuration of a characteristic determination table in the first embodiment. It is an example of a data configuration of a display position table in the first embodiment. 4 is a flowchart for explaining an example of main processing in the first embodiment; 6 is a flowchart for explaining an example of display determination processing in the first embodiment; It is a figure for demonstrating the application example in 1st Embodiment. FIG. 11 is a flowchart for explaining an example of display determination processing in the second embodiment; FIG. It is a figure for demonstrating the application example in 2nd Embodiment. FIG. 14 is a flowchart for explaining an example of display determination processing in the third embodiment; FIG. It is a figure for demonstrating the application example in 3rd Embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION The best mode for carrying out the present invention will be described below with reference to the drawings. The following embodiments are examples to which the display control device (display system) of the present invention is applied, and the content of the present invention is not limited to these embodiments.

[1. First Embodiment]
[1.1 System Appearance]
The entire display system 1 according to the first embodiment will be described with reference to FIG. As shown in FIG. 1, the display system 1 has a recognition device 20 that recognizes a subject P and an image display device 30 that displays an image. It is connected.

In this embodiment, an image display device (display) 32 and an image display device (projector) 34 are provided as an example of the image display device 30 . Then, the image display device (projector) 34 displays the image in the display area R34 as the display position. Here, the number of image display devices 30 may be one or plural. In other words, it is sufficient that the image can be displayed at a plurality of display positions. Details of each device will be described in the following functional configuration.

Furthermore, the image display device 30 may be configured to be movable. For example, in the case of a projector, the projector mounting portion may be configured to be movable, and the lens direction may be freely moved in the direction in which the projector emits light. Moreover, in the case of a display, the attachment portion is movable, and the configuration may be such that the display can be moved to a desired location.

In this embodiment, the recognition device 20, the image display device 30, and the control device 10 are connected in the same premises (for example, they are connected to the same network, or they are locally connected such as USB connection). ), but may be connected via an external network.

That is, the control device 10 is a server connected to the Internet and executes processing on the server. Then, the recognition device 20 and the image display device 30 may be connected via the Internet. In this case, the administrator on the server side can perform multiple controls even from a remote location.

[1.2 Functional configuration]
Next, the functional configuration of each device constituting the display system 1 in this embodiment will be described with reference to FIG.

As shown in FIG. 2 , a recognition device 20 and one or a plurality of image display devices 30 are connected to the control device 10 .

The control device 10 also includes a control section 110 , a recognition section 120 , an image display control section 130 and a storage section 140 .

The control unit 110 is a functional unit for controlling the control device 10 as a whole. The control unit 110 implements various functions by reading and executing various programs stored in the storage unit 140, and is configured by, for example, a CPU (Central Process Unit).

The recognition unit 120 is a functional unit for inputting and processing information obtained by recognizing the human body of the target person by the recognition device 20 . That is, it is a functional unit that recognizes the characteristics of the target person by using the technology (image processing, thermography, position sensor, pressure sensor) for sensing the movement of a person in space in the recognition device 20 .

Here, the characteristics of a target person (person) are determined from the attributes and actions of the target person. Specifically, it indicates whether the subject is a "patient" or a "staff". Other characteristics may be "others" (eg, family members, etc.) who have visited the hospital or facility.

In addition, the “staff” indicates persons such as medical staff such as medical staff and care staff such as nursing care facilities. That is, since the word corresponds to "patient", for example, in home care, "staff" may include family members.

The target person's attributes are those that can determine the target person by the recognition device 20, such as height and physique. Furthermore, by recognizing the size, facial expression, etc. of a person, it is possible to recognize whether the person is male or female, an adult, a child, a healthy person, or the like. It also includes information about what the subject wears, such as the type of clothing the subject wears, the presence or absence of a nameplate, and the presence or absence of a cane. It is also possible to determine the attributes of the subject based on the position of the subject. For example, it is possible to recognize an attribute of a subject on a bed apparatus as a patient, or recognize a subject behind a subject who is determined to be a moving patient as a staff member.

In addition, the target person's motion is to determine the target person's movement, such as the direction and speed of the target person's movement, the posture of the target person such as whether the target person is standing, You can perceive things like movement.

Here, a mechanism by which the recognition device 20 and the recognition unit 120 recognize a human body in space will be described with reference to FIG. Various devices can be used as the recognition device 20. As an example, a method using Kinect (registered trademark) manufactured by Microsoft Corporation will be described.

By using Kinect, it becomes possible to grasp the human skeleton as a stick picture (human body model) (Fig. 3(a)). That is, feature points of each joint of the human body and lines connecting the feature points are detected.

By grasping the human skeleton as a human body model, it is possible to easily recognize the positions of the subject's hands, feet, joints, and the like.

Furthermore, based on these feature points, it is possible to recognize the orientation of a person's face and body. For example, in order to obtain the orientation of the face and the orientation of the body, as shown in FIG. 3B, the characteristic points of each joint are used for recognition. In FIG. 3B, the (x, y, z) coordinates of the feature point P01 and the feature point P02 are respectively detected. The orientation of the face is recognized by calculating the angle between the straight line connecting the two coordinates and the reference plane.

Also, the (x, y, z) coordinates of the feature point P03 and the feature point P04 are respectively detected, and the body direction (orientation and inclination ).

Thereby, for example, the area in which the image is to be displayed can be specified from the direction in which the subject is facing. For example, as shown in FIG. 3C, the angle with respect to the central axis is calculated from the feature point P01 and the feature point P02, thereby calculating the angle of the face (the angle at which the subject is facing) θ1. be done.

Further, it is possible to determine the position where the line of sight reaches the ground line L04 from the height L02 of the viewpoint of the human body (for example, approximately the height of the feature point P02) and the normal direction L03 of the angle of the face. (for example, height L02×arctan θ1). An image can be displayed using this position as a display region R01.

Note that other methods may be used as the method of recognizing the human body. For example, image processing technology using a camera, recognition technology using thermography or infrared rays may be used, or a pressure sensor may be placed on the floor and a person may be recognized based on the direction of the foot and the direction of the load. Also, the line of sight may be detected using a line of sight detection technique.

Here, for recognition of the orientation of the target person P (human body), priority is given to the direction of the line of sight, the orientation of the face, and the orientation of the body in this order. That is, the direction of the line of sight is recognized if the direction of the line of sight can be recognized, the direction of the face is recognized if the direction of the face can be detected, and the body direction is otherwise recognized. A plurality of these may be recognized, or a single direction may be recognized.

Next, the image display control unit 130 is a functional unit for controlling image display on one or more connected image display devices 30 . Here, the displayed image can be displayed at a plurality of display positions. The display position is determined by the switchable display area of one image display device 30, or is determined by switching the image display device to be displayed. That is, a plurality of display positions may be configured by combining the display area and the image display device.

Here, the image display device 30 is a device capable of displaying an image, such as a display device such as a liquid crystal display or an organic EL display, or a projection device such as a projector. Note that the image in this embodiment is a concept that includes not only still images such as characters, figures, and photographs, but also moving images. In FIG. 2, an image display device 32 and an image display device 34 are connected.

FIG. 4 shows an example of setting a display position by combining a plurality of image display devices and display areas. For example, as the image display device 30, an image display device 32 that is a liquid crystal display and image display devices 34 and 36 that are projection devices (projectors) are connected.

The image display device 32 displays an image on the device itself as a display area R32. Also, the image display device 34, which is a projection device, projects an image at a distant position (display area R34). Further, the image display device 36 can display images in a plurality of display areas such as a display area R36a and a display area R36b.

That is, in the case of FIG. 4, images can be displayed at four display positions of the display regions R32, R34, R36a, and R36b. This makes it possible to display the image at an appropriate position. Note that FIG. 4 is a diagram for explaining an example of the image display device 30 and display areas, and in the present embodiment, the number of display areas may be one.

The storage unit 140 is a functional unit that stores various programs and data necessary for the operation of the control device 10 . The storage unit 140 is configured by, for example, a semiconductor memory, an HDD (Hard Disk Drive), or the like. The storage unit 140 also stores a specific determination table 142 and a display position table 144 .

The specific determination table 142 is a table that stores information for determining characteristics of a subject. For example, as shown in FIG. 5, attributes (for example, "white coat") and characteristics (for example, "staff") are stored.

Here, the property is for determining the position for displaying a display message, which will be described later. In this embodiment, "staff", "patient", etc. are stored, but for example, "adult", "child", etc. may be stored.

The display position table 144 is a table that stores characteristics of a subject and display positions in association with each other. For example, the display position for the characteristic "staff" is stored in the display area R32, and the display position for the characteristic "patient" is stored in the display area R34.

In addition, the display position table 142 may also store display positions. For example, the coordinates in the space are set, and where the display position in the space is set is stored as the coordinates. Thereby, for example, it is possible to determine the display position corresponding to the direction in which the subject P is facing, and it is possible to specify the image display device 30 corresponding to the display position.

[1.3 Flow of processing]
Next, the flow of processing in this embodiment will be described. First, when a human body is detected (step S102; Yes), human body recognition processing is executed (step S104). In this embodiment, feature points corresponding to joints and the like are detected from the results of sensing performed by the recognition device 20, and the orientation of the human body and the like are recognized. Attributes such as clothing patterns and the presence or absence of a name tag are also detected by sensing.

Also, an area for detecting and recognizing a human body is a predetermined recognition area. For example, it may be near a bed apparatus, a rehabilitation facility, or a hallway in the facility. Normally, a fixed area is recognized, but by providing the recognition device 20, for example, the recognition area can be anywhere. For example, in the case of home care, a recognition area may be provided in the home.

Subsequently, a characteristic determination process is executed based on the recognition result of the human body recognition process (step S106). In this embodiment, the characteristics are determined from the attributes detected in step S104.

Subsequently, display determination processing is executed to determine the display position/message based on the characteristics determined by the characteristics determination processing (step S108). A message or an image is displayed in the display area determined by the display determination process (step S110).

Here, the display determination processing will be explained using FIG. First, it is determined whether a patient exists in the recognition area (step S152). Here, if there is no patient, that is, if there is no subject whose characteristic is "patient", this process is terminated without doing anything (step S152; No). In this case, for example, in step S110, default processing (for example, displaying a predetermined message on a predetermined image display device 30) may be performed, or processing for not displaying a message may be performed. Also good.

Next, it is determined whether or not there is a staff member near the patient (step S154). Here, if there is no staff, it is determined to display a patient message, which is a message intended for the patient, at a patient display position that is easy for the patient to see (step S156).

Also, if the staff is present near the patient, it is determined to display a staff message, which is a message intended for the staff, at the staff display position where the staff can easily see it (step S158).

Then, in step S110, the content of the display message is displayed at the display position determined in step S156 or step S158.

In the above-described embodiment, it is assumed that a patient exists in the recognition area. Also good. That is, even if step S152 is No, if the staff is detected, step S158 may be executed.

[1.4 Application examples and effects]
Next, an application example of this embodiment will be described with reference to FIG. FIG. 9 shows a state in which messages and images can be displayed in the display area R32 and the display area R34 by the image display device.

FIG. 9(a) is a diagram when the target person PP is in the recognition area. First, the target person PP is recognized by human body recognition processing. Then, it is determined that the characteristic of the subject PP is that of a patient. Here, determination of the subject as a patient can be made by sensing, for example, the posture/walking pattern (step length, trunk posture during walking, etc.).

For example, if the patient PP is an elderly person, the elderly often have a downward posture, so if you want to disclose information to the patient (for example, when a dementia patient is going to an unexpected place) displays a patient message on the display area R32 on the floor.

FIG. 9(b) is a diagram when the target person PS exists on the target person PP side. The subject PP is determined as a patient and the subject PS as a staff. Since the staff is present on the patient's side, the staff message is displayed in the display area R34 located near the ceiling. By displaying information such as a message near the ceiling where it is difficult for other patients to notice it, it is possible to provide useful information only to the staff without giving patients too much information.

[2. Second Embodiment]
Next, a second embodiment will be described. In the second embodiment, a case will be described in which a message is displayed by determining the display position according to the movement (direction) or action of the subject.

The functional configuration of this embodiment is the same as that of the first embodiment, and the display determination process of FIG. 8 is replaced with the display determination process of FIG.

First, it is determined whether or not the staff exists (step S202). That is, the perceived property is whether the subject of the staff is present, and uses the movement of the staff to ascertain this presence. For example, in the case of FIG. 10, first, the presence of a patient whose motion is characteristic is confirmed. Then, the presence of a target person near the patient or a target person moving in the opposite direction to the patient is confirmed as a staff member (step S202: Yes).

Next, the movement of the staff is detected. For example, in the case of the processing of FIG. 10, the display area has a first position in the first direction and a second position in the opposite second direction. Therefore, it is determined whether or not it is the first direction (step S204). If the movement (orientation) of the staff is in the first direction, it is determined to display the staff message at the first position (step S204; Yes→step S206). It is decided to display a message (step S204; No→step S208).

Since it is sufficient to detect whether or not the staff is facing a displayable position, a plurality of directions may be provided. It may also decide not to display the staff message if it is not facing in any direction. Furthermore, in the present embodiment, the direction is detected, but the display position may be specified by detecting the orientation of the face, for example.

FIG. 11 is a diagram for explaining an application example of this embodiment. Display areas R32 and R34 are provided as display positions. Here, to explain using FIG. 11, first, there are the subject PP and the subject PS.

Then, the characteristics of the subject PP are determined to be that of a patient based on the posture, motion, and the like. Here, patient PP is moving in direction V1. In this case, the target person PS facing the direction V1 side is determined to be a staff member (for example, a therapist or a medical worker). Therefore, the staff message is displayed in the display area R34 in the direction the target person PS is facing.

In this case, since the patient is recognized, for example, the patient's walking motion, speed, time, stride length, etc. may be measured and provided to indicate the rehabilitation result.

Also, in FIG. 11(b), first, the subject PP is determined to be a patient. In the case of FIG. 11(b), the subject PP and the subject PS are moving in the same direction V1. In this case, the subject PP in front is determined to be the patient, and the subject PS in the back is determined to be the staff. It should be noted that the patient/staff may be determined from the motion, posture, etc. of the subject PP and the subject PS.

In the case of FIG. 11(b), since the target person PS who is the staff is facing in the direction V1, the staff message is displayed in the display area R32.

Also, in this case, display/non-display of the screen may be controlled according to the line of sight of the subject. For example, even if the direction S1 in which the subject PS (staff) is facing is the direction of the display area R32, if the direction S2 in which the subject PP (patient) is facing is the direction of the display area R32, the message is hidden. Then, as shown in FIG. 11B, when the direction S2 in which the target person PP is facing is out of the display area R32, the staff message is displayed in the display area R32.

Thus, according to this embodiment, the message is displayed appropriately in the moving direction of the staff. For example, in the rehabilitation field, it is necessary to select and recognize the patient and the therapist when it is desired to sense and measure the patient's gait, etc., while a therapist sits in front of and behind the patient during walking training. In that case, it is possible to identify the patient by sensing the directions and positions of the patient and the therapist. Then, it becomes possible to display an appropriate message to the staff accompanying the patient.

In addition, by detecting the direction in which the target person is facing, it is possible to display a message only to the appropriate target person (staff) (hide information that the target person (patient) does not want to see). Become.

[3. Third Embodiment]
Next, a third embodiment will be described. The third embodiment is processing for determining whether or not to display a message based on the determined characteristics.

The functional configuration of this embodiment is the same as that of the first embodiment, and the display determination process of FIG. 8 is replaced with the display determination process of FIG.

This embodiment is an embodiment in which a necessary staff message is displayed to the staff.

First, when the staff does not exist for the target person in the recognition area, the message is not displayed (step S302; No→step S308). Here, even if there is a staff member, the message is not displayed even if another person is present in the vicinity of the staff member (step S302; Yes→step S304; Yes→step S308). That is, the staff message is displayed only when there is only the target person whose characteristics are determined to be staff (step S302; Yes→step S304; No→step S306).

An application example of this embodiment will be described with reference to FIG. The recognition area in FIG. 13 is the bedside. The display area R32 is arranged at a position that cannot be seen from the position of the patient PP.

First, as shown in FIG. 13(a), when the target person PS is determined to be a staff member, a staff message is displayed in the display area R32. In this case, the staff message will be displayed when detected near the patient PP, ie, at the bedside or the like.

Also, in FIG. 13(b), the target person PF exists in the recognition area, but the characteristic is not staff (for example, "family"). In this way, the target person PF is present in the recognition area, but the message is not displayed because the characteristic is determined to be other than staff.

Also, in FIG. 13(c), there is a target person PS who is determined to be a staff member in the recognition area. However, a target person PF other than the staff exists in the vicinity. In this case, it is also possible not to display it. In this case, according to the importance of the message, the message may be displayed even if there is a target person other than the staff.

[4. Modification]
Although the embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and designs and the like within the scope of the scope of the claims are also possible without departing from the gist of the present invention. include.

Further, in the above-described embodiments, the display device is mainly used as means for providing information to staff, patients, etc., but other notification devices may be used. For example, it may be light, sound, or vibration.

In the above-described embodiment, the characteristics of the display target are staff (medical staff, nursing care staff, etc.), and the characteristics other than the display target are patients, care recipients, elderly people, etc., but the present invention is not limited to this. do not have. For example, the display target may be a patient or the like, and the non-display target may be reversed to the staff, or the display target may have a characteristic such as a family member or a trader who has come to visit the display target.

In each of the above-described embodiments, the processing is described separately for convenience of explanation, but it is of course possible to operate in combination within a possible range.

In addition, the program that operates in each device in the embodiment is a program that controls the CPU and the like (a program that causes the computer to function) so as to implement the functions of the above-described embodiments. Information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) during processing, then stored in various ROM and HDD storage devices, and read and corrected by the CPU as necessary. • Writing is performed.

Here, as recording media for storing programs, semiconductor media (eg, ROM, non-volatile memory cards, etc.), optical recording media/magneto-optical recording media (eg, DVD (Digital Versatile Disc), MO (Magneto Optical Disc), CD (Compact Disc), BD, etc.), magnetic recording media (for example, magnetic tape, flexible disc, etc.). By executing the loaded program, the functions of the above-described embodiments are realized. In some cases, inventive features are realized.

When distributed to the market, the program can be stored in a portable recording medium for distribution, or transferred to a server computer connected via a network such as the Internet. In this case, of course, the storage device of the server is also included in the present invention.

Also, part or all of the devices in the above-described embodiments may typically be realized as an LSI (Large Scale Integration), which is an integrated circuit. Each functional block of each device may be individually chipped, or part or all may be integrated and chipped. Also, the method of circuit integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. Further, when a technology for integrating circuits to replace LSIs emerges due to advances in semiconductor technology, it is of course possible to use an integrated circuit based on this technology.

1: Display system 10: Control device 110: Control unit 120: Recognition unit 130: Image display control unit 140: Storage unit 142: Characteristic determination table 144: Display position table 20: Recognition devices 30 to 34: Image display device

Claims (4)

display control means capable of displaying information at a first position or at a second position different from the first position;
Determination means for determining whether or not a target person present in the area is a display target person, and determining the orientation of the target person's face;
with
The determination means determines the characteristics of the subject,
The display control means is
If the target person is determined to be a display target person , specifying either a first position or a second position based on the orientation of the face facing the target person ,
performing control to display information corresponding to the characteristics of the subject at the specified display position ;
When it is determined that the target person is not a display target person, performing control not to display the information,
Control to display the information when the characteristic of the display target person is a medical worker or a care staff in the area, and not to display the information when the area includes other people. conduct
A display control device characterized by: display control means capable of displaying information at a first position or at a second position different from the first position;
Determination means for determining whether or not a target person present in the area is a display target person and determining the direction of the face of the target person;
with
The first position is a position on the floor, the second position is a position other than the floor,
The determination means determines the characteristics of the subject,
The display control means is
If the target person is determined to be a display target person, specifying either a first position or a second position based on the orientation of the face facing the target person,
performing control to display information corresponding to the characteristics of the subject at the specified display position;
When it is determined that the target person is not a display target person, performing control not to display the information,
If the characteristic of the subject is elderly, control is performed to display the information at the first position and at the position of the face orientation to which the subject is facing.
A display control device characterized by: 3. The display control device according to claim 1, wherein the display control means performs control not to display the information when it is determined that a person who is not a person to be displayed exists in the area. The display target includes a first display target and a second display target,
The display control means is
performing control to display the first information when it is determined that the first display target person exists in the area;
4. The method according to any one of claims 1 to 3 , wherein control is performed to display second information different from the first information when it is determined that a second display target person exists in the area. Display controller.

Images