JP2022175504A - Information processing apparatus, program, and information processing method - Google Patents

Abstract

To provide technology that enables more appropriate mental care.SOLUTION: According to one aspect of the present invention, an information processing apparatus is provided. The information processing apparatus is configured to execute a determination step, a reception step, and a display control step. The determination step determines an emotion of a first user based on a facial image of the first user. The reception step receives first emotion information. The first emotion information includes a self-evaluation result of self-evaluation by the user. The display control step displays notification information 7 if the discrepancy between the determination result in the determination step and the self-evaluation result is equal to or greater than a threshold.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, a program, and an information processing method.

2. Description of the Related Art Conventionally, there has been known a technique for judging the health condition of a worker by capturing an image of the worker's facial expression or the like. Patent Literature 1 discloses a system that determines the state of a worker by capturing an image of the worker's face.

JP 2014-228957 A

However, even if they are judged to be in good health from the outside, they may actually be in poor health, and if there is a large gap between the external and internal aspects, the manager may not be able to provide appropriate mental care.

In view of the above circumstances, the present invention provides a technique that enables more appropriate mental care.

According to one aspect of the present invention, an information processing device is provided. This information processing device is configured to execute a determination step, a reception step, and a display control step. In the determination step, the emotion of the first user is determined based on the face image of the first user. The receiving step receives the first emotion information. The first emotion information includes self-evaluation results of self-evaluation by the user. In the display control step, notification information 7 is displayed when the discrepancy between the determination result in the determination step and the self-evaluation result is equal to or greater than a threshold.

This makes it possible to provide a technology that enables more appropriate mental care.

1 is a diagram showing an overview of the configuration of an information processing system 1 according to this embodiment; FIG. 3 is a block diagram showing the hardware configuration of the information processing device 3; FIG. 3 is a functional block diagram showing functions of the information processing device 3; FIG. 4 is an activity diagram showing an example of information processing by the information processing device 3; FIG. 4A and 4B are diagrams showing examples of screens displayed on the first user terminal 2 and the second user terminal 4; FIG. 4A and 4B are diagrams showing examples of screens displayed on the first user terminal 2 and the second user terminal 4; FIG. It is a figure which shows an example of the element of emotion. FIG. 11 is a diagram showing an example of a determination result by a determination unit 331; FIG. FIG. 4 is a diagram showing an example of a database in which emotions and scores are associated; 3 is a diagram showing an example of a screen displayed on the first user terminal 2; FIG. 7 is a diagram showing an example of notification information 7; FIG. 4 is a diagram showing an example of a screen displayed on the second user terminal 4; FIG. It is a figure which shows an example of the degree of coincidence of a self-evaluation result and a peer's evaluation result.

Embodiments of the present invention will be described below with reference to the drawings. Various features shown in the embodiments shown below can be combined with each other.

By the way, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium (Non-Transitory Computer-Readable Medium), or may be downloaded from an external server. It may be provided as possible, or may be provided so that the program is activated on an external computer and the function is realized on the client terminal (so-called cloud computing).

Further, in the present embodiment, the term “unit” may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be specifically realized by these hardware resources. . In addition, various information is handled in the present embodiment, and these information are, for example, physical values of signal values representing voltage and current, and signal values as binary bit aggregates composed of 0 or 1. It is represented by high and low, or quantum superposition (so-called quantum bit), and communication and operation can be performed on a circuit in a broad sense.

A circuit in a broad sense is a circuit implemented by appropriately combining at least circuits, circuits, processors, memories, and the like. Application Specific Integrated Circuits (ASICs), programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and field It includes a programmable gate array (Field Programmable Gate Array: FPGA).

1. Hardware Configuration This section describes the hardware configuration of this embodiment. FIG. 1 is a diagram showing an overview of the configuration of an information processing system 1 according to this embodiment.

1.1 Information processing system 1
The information processing system 1 includes a first user terminal 2, an information processing device 3, and a plurality of second user terminals 4 (second user terminals 4-1 to 4-N), which are electrically connected. It is configured to be communicable through a communication line. The information processing device 3 can communicate with the first user terminal 2 and each of the second user terminals 4-1 to 4-N via a network such as the Internet. These components are further described.

1.2 First User Terminal 2
The first user terminal 2 is operated by the first user who performs self-evaluation of emotions, and is capable of accessing the information processing device 3 and the second user terminal 4 via a smartphone, tablet terminal, computer, or other telecommunication line. If so, its form does not matter.

The first user terminal 2 has a display unit, an input unit, a communication unit, a storage unit, a control unit, and an imaging unit. are electrically connected via

For example, the display unit and the input unit may be included in the housing of the first user terminal 2, or may be externally attached. The display unit displays a screen of a graphical user interface (GUI) that can be operated by the user. The input unit may be implemented as a touch panel integrated with the display unit. With a touch panel, the user can input a tap operation, a swipe operation, or the like. Of course, a switch button, a mouse, a QWERTY keyboard, or the like may be employed instead of the touch panel.

The imaging unit includes a camera or the like that combines a CCD (Charge Couple Devices) image sensor or a CMOS (Completely Metal Oxide Semiconductor) image sensor as an imaging element with an optical element such as a lens. Furthermore, the imaging unit includes an image recognition processor using a convolutional neural network, etc., and is capable of recognizing the user's face. The imaging unit can also use a plurality of cameras. An imaging device is an electronic component that converts light entering from a lens of an imaging unit into an electrical signal. Technical specifications such as the number and size of the imaging elements of the imaging unit are not limited.

For specific descriptions of the communication unit, the storage unit, and the control unit, refer to the description of the communication unit 31, the storage unit 32, and the control unit 33 in the information processing device 3 described below.

1.3 Information processing device 3
FIG. 2 is a block diagram showing the hardware configuration of the information processing device 3. As shown in FIG. The information processing device 3 has a communication section 31 , a storage section 32 and a control section 33 , and these components are electrically connected via a communication bus 30 inside the information processing device 3 . Each component will be further described.

(Communication unit 31)
The communication unit 31 preferably uses wired communication means such as USB, IEEE1394, Thunderbolt, wired LAN network communication, etc., but wireless LAN network communication, mobile communication such as LTE/3G, Bluetooth (registered trademark) communication, etc., can be used as necessary. may be included. That is, it is more preferable to implement as a set of these communication means.

(storage unit 32)
The storage unit 32 stores various information defined by the above description. For example, it can be used as a storage device such as a solid state drive (SSD) for storing various programs related to the information processing device 3 executed by the control unit 33, or as a temporary storage device related to program calculation. It can be implemented as a memory such as a Random Access Memory (RAM) that stores information (arguments, arrays, etc.) required for the . A combination of these may also be used.

(control unit 33)
The control unit 33 processes and controls overall operations related to the information processing device 3 . The control unit 33 is, for example, a central processing unit (CPU) (not shown). The control unit 33 implements various functions related to the information processing device 3 by reading a predetermined program stored in the storage unit 32 . That is, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33), and each function unit (see FIG. 3) included in the control unit 33 executes can be These are further detailed in the next section. Note that the control unit 33 is not limited to a single unit, and may be implemented to have a plurality of control units 33 for each function. A combination thereof may also be used.

1.4 Second user terminal 4
The second user terminal 4 is operated by a second user who is a worker belonging to the same organization. The form of the second user terminal 4 is not limited as long as it can access the first user terminal 2 and the information processing device 3 through a smart phone, a tablet terminal, a computer, or any other telecommunication line.

The second user terminal 4 has a display unit, an input unit, a communication unit, a storage unit, a control unit, and an imaging unit. electrically connected via a bus. Please refer to the description of the first user terminal 2 and the information processing device 3 for the description of each component.

2. Functional Configuration This section describes the functional configuration of this embodiment. FIG. 3 is a functional block diagram showing functions of the information processing device 3. As shown in FIG. As described above, information processing by software (stored in the storage unit 32 ) is specifically realized by hardware (control unit 33 ), and can be executed as each functional unit included in the control unit 33 .

Specifically, the information processing device 3 (control unit 33) includes a determination unit 331, a reception unit 332, a display control unit 333, and a selection unit 334 as functional units.

(Determination unit 331)
The determination unit 331 is configured to be able to execute the determination step. In the determination step, the determination unit 331 compares various pieces of information acquired by the reception unit 332 and determines whether or not a predetermined condition is satisfied. For example, the determination unit 331 can compare various data acquired by the reception unit 332 with conditions stored in advance in the storage unit 32 and determine whether or not the conditions are satisfied. The information processing device 3 can execute different processes based on the determination result of the determination unit 331 . Specific examples are further detailed in the next section.

(Reception unit 332)
The reception unit 332 is configured to be able to execute the reception step. The reception unit 332 is configured to receive information via the communication unit 31 or the storage unit 32 and read it into the working memory. In particular, the reception unit 332 is configured to receive various information from the first user terminal 2 via the network and the communication unit 31 . In this embodiment, various information received by the receiving unit 332 is described as being stored in the storage unit 32 .

In particular, the reception unit 332 receives the face image of the first user, the first emotion information 51 and the second emotion information 52 .

(Display control unit 333)
The display control unit 333 is configured to be able to execute the display control step. In the display control step, the display control unit 333 displays various information stored in the storage unit 32 or a screen including these in a form that can be viewed on the first user terminal 2 . Specifically, the display control unit 333 controls the display unit of the first user terminal 2 to display visual information such as screens, images, icons, and messages. The display control unit 333 may generate only rendering information for displaying visual information on the first user terminal 2 .

In particular, the display control unit 333 causes the notification information 7 to be displayed. Details will be described later.

(Selection unit 334)
The selection unit 334 executes a selection step. In the selection step, the selection unit 334 selects a second user whose self-evaluation result and each other's evaluation result by a plurality of second users have a degree of matching equal to or greater than a threshold. Details will be described later.

3. Details of Information Processing In Section 3, the information processing method executed by the information processing device 3 will be described with reference to activity diagrams. As shown below, the information processing method includes steps of the information processing system 1 . In the following, an example will be described in which the emotion of the first user is determined at the time of going to and from work.

FIG. 4 is an activity diagram showing an example of information processing by the information processing device 3. As shown in FIG. Note that the order of the following processes can be changed as appropriate, and a plurality of processes may be executed simultaneously, or not all of them need to be executed.

5 and 6 are diagrams showing examples of screens displayed on the first user terminal 2 and the second user terminal 4. FIG. When the first user goes to work, the first user can transmit the time of coming to work to the information processing device 3 by operating the screen shown in FIG. When the first user presses the go-to-work button 61, the screen shown in FIG. 6 is displayed, and the imaging unit of the first user terminal 2 captures and recognizes the user's face. The photographed face image is transmitted to the information processing device 3 via the communication unit of the first user terminal 2 .

Subsequently, the reception unit 332 receives the face image via the network and the communication unit 31 (A101). Also, the determination unit 331 determines the emotion of the first user based on the face image of the first user (A102). Specifically, for example, the determination unit 331 recognizes an emotion consisting of a plurality of elements based on a face image, and then determines the emotion by replacing it with a one-dimensional numerical value based on a predetermined evaluation criterion.

FIG. 7 is a diagram showing an example of emotional elements. As shown in FIG. 7, emotions that can be recognized based on facial images are roughly classified into either “positive” or “negative”, and “positive” emotions include “happy” and “calm”. ', and 'negative' emotions include the elements 'angry', 'fear', 'sad', 'disgust', 'confusion' and 'surprise'.

FIG. 8 is a diagram showing an example of the determination result by the determination unit 331. As shown in FIG. The determination unit 331 classifies the face image into predetermined elements according to facial expressions included in the face image. At this time, as shown in FIG. 8, the determination unit 331 can classify the emotion of the first user as including multiple elements according to the facial expression of the first user. In the example of FIG. 8, the determining unit 331 determines that the emotional elements recognized from the face image are 15% "happy", 55% "calm", 10% "angry", and 10% "fear". %, 5% "sad", and 5% "confused".

FIG. 9 is a diagram showing an example of a database in which the first user's mood and points are associated with each other. As shown in FIG. 9, the storage unit 32 expresses the emotions of the first user in terms of moods, and stores scores from 1 to 6 in correspondence with the moods. Specifically, the first user's mood is 1 point if it is "very good", 3 points if it is "good", 4 points if it is "somewhat good", 5 points if it is "bad", and 5 points if it is "bad". is associated with 6 points and stored. Further explanation will be continued below, assuming that a lower score indicates a more positive emotion.

Here, for example, the evaluation criteria when the determination unit 331 expresses the first user's emotion as a score are as follows.
(1) A score of 1 is given when the sum of the ratios of elements classified as "positive" as the types of the elements recognized from the face image is 70% or more.
(2) If the ratio of "calm" is 70% or more, it is judged as 2 points.
(3) If the sum of the ratios of elements classified as "negative" is 70% or more, a score of 6 is given.
(4) If the sum of the proportions of elements classified as "negative" is 60% or more, a score of 5 is given.
(5) If the sum of the proportions of elements classified as "negative" is 50% or more, it is judged as 4 points.
(6) If none of the above (1) to (5) apply, a score of 3 is determined.
In the example of FIG. 8, when the determination is made according to the above evaluation criteria, none of the above (1) to (5) apply, so the determination result is 3 points. In this way, the determination unit 331 determines by quantifying the user's emotion based on the user's face image.

After that, the display control unit 333 causes the first user terminal 2 to highlight and display the score determined by the determination unit 331 and the corresponding emotion type. FIG. 10 is a diagram showing an example of a screen displayed on the first user terminal 2. As shown in FIG. In the example of FIG. 10, the emotion determined by the determination unit 331 is "somewhat good", and the display control unit 333 displays the first emotion evaluation button 62 corresponding to each of the plurality of types of emotions. The first emotion evaluation button 62 corresponding to each of the emotions described above is displayed in a manner in which it is highlighted. By pressing the first emotion evaluation button 62, the user can self-evaluate his own emotions. When the first user selects one of the plurality of first emotion evaluation buttons 62 displayed on the first user terminal 2, the selected emotion type is transmitted to the information processing device 3 as the first emotion information 51. do. That is, the first emotion information 51 includes a self-evaluation result obtained by the user's self-evaluation of his/her emotions. Here, the types of emotions corresponding to the first emotion evaluation button 62 and the second emotion evaluation button 63, which will be described later, respectively correspond to the scores shown in FIG. That is, the self-evaluation result is a quantification of emotions.

After that, the reception unit 332 receives the first emotion information 51 (A103). Moreover, the display control unit 333 displays the notification information 7 when the difference between the determination result of the determination unit 331 and the self-evaluation result is equal to or greater than the threshold (A104). For example, if the difference between the determination result of the determination unit 331 and the self-evaluation result by the first user is two points or more, the notification information 7 is displayed. FIG. 11 is a diagram showing an example of notification information 7. As shown in FIG. In this case, if the score of the determination result in the determination unit 331 is 1 point and the score related to the self-evaluation result by the first user is 3 points, the score difference between the two is 2 points. Is displayed. Thus, when the difference between the evaluation result derived based on the facial expression of the first user and the self-evaluation result by the first user is equal to or greater than a threshold, the user can be notified and mental care can be provided. It becomes possible.

4. Others Evaluation Function Section 4 describes an example in which the second user can evaluate the emotion of the first user, who is another user. FIG. 12 is a diagram showing an example of a screen displayed on the second user terminal 4. As shown in FIG. It is assumed that the second user evaluates the feelings of other users during the day at the time of clocking out.

As shown in FIG. 12, at the second user terminal 4, the second user can assess the emotions of the first user. Specifically, the second user selects one of the second emotion evaluation buttons 63 displayed on the second user terminal 4 to evaluate the emotion of the first user. At this time, the second user who can evaluate the emotions of the first user is preferably a person who is considered to have business contact with the first user, such as a person who belongs to the same department as the first user. When the second emotion evaluation button 63 is selected, the selection result is transmitted to the information processing device 3 via the communication section of the second user terminal 4 as the second emotion information 52 . In this case, the second emotion information 52 includes a peer evaluation result obtained by evaluating the emotion of the first user by another second user different from the first user. Further, when there are a plurality of second users who evaluate the first user, the second emotion information 52 is the average of a plurality of peer evaluation results evaluated by a plurality of second users different from the first user. include.

Subsequently, at A<b>103 , the reception unit 332 receives the second emotion information 52 via the network and the communication unit 31 . Thereafter, in A104, the display control unit 333 displays the notification information 7 when the difference between the determination result of the determination unit 331 and the peer evaluation result is equal to or greater than the threshold. When there are a plurality of second users who evaluate the first user, the display control unit 333 displays the notification information 7 when the difference between the determination result of the determination unit 331 and the average of the peer evaluation results is equal to or greater than the threshold. .

5. Second User Selection Process A second user capable of evaluating the emotions of the first user may be selected under certain conditions. Specifically, the selection unit 334 selects a second user whose degree of matching between the first user's self-evaluation result and each other's evaluation result by a plurality of second users is equal to or greater than a threshold. Specifically, for example, the selection unit 334 selects the second user whose average difference between the score of the self-evaluation result by the first user and the score of the evaluation result by the second user is less than one.

FIG. 13 is a diagram showing an example of the degree of matching between self-evaluation results and peer-evaluation results. In the example of FIG. 13, the difference between the score of the self-evaluation result by the first user and the score of the evaluation result by each of the second users, and the average of the differences over a certain period of time are shown. According to the example of FIG. 13, "Eita Taniguchi" and "Tsuyoshi Abe" whose average difference is less than 1 can be evaluated by the first user. In this case, the display control unit 333 displays the notification information 7 when the discrepancy between the determination result of the determination unit 331 and the peer evaluation result by the selected second user is equal to or greater than the threshold.

6. Others Regarding the information processing system 1 according to the present embodiment, the following aspects may be adopted.

(1) The aspect of this embodiment may be a program. The program causes the computer to execute each step of the information processing device 3 .
(2) In the present embodiment, the reference information 5 is assumed to be a database such as a lookup table, and detailed items included therein have been described, but this is merely an example and is not limited to this. Furthermore, the reference information 5 is not limited to a database such as a lookup table in the first place, and may be a mathematical model in which a plurality of pieces of information are mathematically related. It may be a trained model.
(3) In the present embodiment, an example in which emotions are evaluated when the first user and the second user arrive at work has been described. It's okay.

Furthermore, it may be provided in each aspect described below.
In the information processing device, in the determination step, determination is made by quantifying the emotion of the first user based on the face image of the first user, and the self-evaluation result is obtained by quantifying the emotion. ,thing.
In the information processing device, in the receiving step, second emotion information is received, and the second emotion information is obtained by evaluating the emotion of the first user by a second user different from the first user. The evaluation result is included, and in the display control step, notification information 7 is displayed when the discrepancy between the judgment result in the judgment step and the peer evaluation result is equal to or greater than a threshold.
In the information processing device, the second emotion information includes an average of the plurality of peer evaluation results obtained by evaluating the emotion by a plurality of second users different from the first user, and the display control step includes: If the discrepancy between the determination result in the determination step and the average is equal to or greater than a threshold, notification information 7 is displayed.
The information processing device is configured to further execute a selection step, in the selection step, the degree of matching between the self-evaluation result and each of the peer-evaluation results by the plurality of second users is equal to or greater than a threshold value. Select a second user, and in the display control step, if the difference between the judgment result in the judgment step and the peer evaluation result by the selected second user is a threshold or more, display notification information 7; thing.
A program that causes a computer to execute each step of the information processing device.
An information processing method executed by an information processing device, comprising steps of the information processing system.
Of course, this is not the only case.

Finally, while various embodiments of the invention have been described, these have been presented by way of example and are not intended to limit the scope of the invention. The novel embodiment can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. The embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

1: information processing system 2 : first user terminal 3 : information processing device 30 : communication bus 31 : communication unit 32 : storage unit 33 : control unit 331 : determination unit 332 : reception unit 333 : display control unit 334 : selection unit 4 : Second user terminal 5 : Reference information 51 : First emotion information 52 : Second emotion information 61 : Attendance button 62 : First emotion evaluation button 63 : Second emotion evaluation button 7 : Notification information

Claims (7)

An information processing device,
configured to execute a determination step, a reception step, and a display control step;
In the determination step, the emotion of the first user is determined based on the facial image of the first user;
In the receiving step, the first emotion information is received;
The first emotion information includes a self-evaluation result obtained by evaluating the user's own emotion,
In the display control step, if the discrepancy between the determination result in the determination step and the self-evaluation result is equal to or greater than a threshold, notification information 7 is displayed.
thing. In the information processing device according to claim 1,
In the determination step, determination is made by quantifying the emotion of the first user based on the facial image of the first user;
The self-evaluation results are obtained by quantifying the emotions,
thing. In the information processing apparatus according to claim 1 or claim 2,
receiving the second emotion information in the receiving step;
The second emotion information includes a peer evaluation result obtained by evaluating the emotion of the first user by another second user different from the first user,
In the display control step, if the discrepancy between the determination result in the determination step and the peer evaluation result is equal to or greater than a threshold, notification information 7 is displayed.
thing. In the information processing device according to claim 3,
The second emotion information includes an average of a plurality of peer evaluation results obtained by evaluating the emotion by a plurality of second users different from the first user,
In the display control step, when the difference between the determination result in the determination step and the average is equal to or greater than a threshold, notification information 7 is displayed.
thing. In the information processing device according to claim 4,
configured to perform further selection steps,
In the selecting step, the second user whose degree of matching between the self-evaluation result and each of the peer-evaluation results by the plurality of second users is equal to or greater than a threshold is selected;
In the display control step, when the discrepancy between the determination result in the determination step and the peer evaluation result by the selected second user is equal to or greater than a threshold, notification information 7 is displayed.
thing. a program,
causing a computer to execute each step of the information processing apparatus according to any one of claims 1 to 5,
thing. An information processing method executed by an information processing device,
Each step of the information processing system according to any one of claims 1 to 6,
Method.

Images