JP2016062084A - Work management device, feeling analysis terminal, work management program and work management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of correctly evaluating a worker.SOLUTION: A work management device 200 comprises: a communication part 140 for receiving a voice code obtained by encoding a voice related to work with a predetermined method from another device; and a work evaluation part 234 for evaluating a feeling of the voice using the received voice code, and using the evaluation for evaluating the work related to the voice code. When there is information of an image related to the work, the work evaluation part 234 modifies the evaluation of the feeling of the voice using the feeling specified by a facial expression processing part 232, and evaluates the work.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a work management device, an emotion analysis terminal, a work management program, and a work management method.

  Patent Document 1 states that “an explicit feature amount indicating how much the content of meaning is clearly specified in the target text data is estimated using at least one of the morphological analysis result and the syntax analysis result of the target text data. Angry in the target text data using an explicitness estimation unit and a discriminator previously learned from the explicit feature quantity of learning text data with a teacher signal indicating whether or not an angry emotion is expressed It is described that the emotion estimation device includes an emotion identification unit that identifies whether or not the emotion is expressed from the explicit feature amount of the target text data estimated by the explicit estimation unit.

JP 2012-113542 A

  In the above technique, although it is possible to identify emotions based on linguistic information caused by characters, it is difficult to perform emotion evaluation that takes into account the quality of the voice during speech. For example, when the worker performs work under the direction of the worker in charge of visit, when evaluating the worker considering the mind given to the worker, only the language information at the time of utterance is used. Therefore, it is difficult to accurately evaluate workers. In other words, since the mentality of the worker is not necessarily expressed as a change in linguistic information, evaluating the worker using only language information ignoring the quality of conversational speech with the worker is useful. There is a high risk of lack of accuracy.

  An object of the present invention is to provide a technique for more accurately evaluating an operator.

  The present application includes a plurality of means for solving at least a part of the above-described problems. Examples of such means are as follows. In order to solve the above-described problem, the work management device according to the present invention includes a communication unit that receives a voice code obtained by coding a voice related to work from another device by a predetermined method, and uses the received voice code. A work evaluation unit that performs evaluation of emotion and uses the evaluation to evaluate work related to the voice code.

  Further, in the work management apparatus, the work evaluation unit performs the evaluation of the emotion of the voice using the geometric information of the waveform constituted by the temporal change of the quality of the emotion specified using the voice code. It may be a thing.

  Further, in the work management apparatus, when the waveform has a change exceeding a predetermined degree in the quality of the emotion, the sound is reproduced starting from the past for a predetermined time from the time of the change. A cause search unit may be provided.

  Further, in the work management apparatus, the work evaluation unit evaluates the emotion according to the degree of difficulty related to the work, and evaluates the work by adding correction based on the difficulty to the evaluation of the emotion. It may be a thing.

  Further, in the work management device, the work evaluation unit evaluates the emotion according to a person in charge of managing the work, and adds the correction based on the characteristic of the person in charge to the evaluation of the emotion. The work may be evaluated.

  Further, in the work management apparatus, the communication unit receives the voice code, receives image information related to the voice code, and includes an expression processing unit that identifies an emotion represented by a facial expression included in the image information. The work evaluation unit may evaluate the work by adding a correction based on the emotion specified by the facial expression processing unit to the evaluation of the emotion of the voice.

  In addition, the work management apparatus includes a storage unit that stores knowledge information received by the communication unit in association with a voice code related to the work, and a knowledge management unit, and the communication unit identifies the work. The knowledge management unit may identify and output knowledge information associated with the received voice code related to the work.

  Also, the emotion analysis terminal according to the present application includes a voice acquisition unit that acquires voice information related to work, a voice analysis unit that specifies voice code information indicating the quality of emotion according to an attribute included in the voice information, and the voice code An emotion analysis unit for associating information with the work, and a communication unit for outputting the voice code information to another apparatus together with the information on the work associated therewith.

  In the emotion analysis terminal, the image acquisition unit that acquires image information corresponding to the audio information, the facial expression analysis unit that specifies facial expressions included in the image information, and the emotion analysis unit include: Information of the voice code information and the facial expression may be associated with the work.

  Further, the work management program according to the present application is a work management program for causing a computer to execute a procedure for managing work. The work management program causes the computer to function as control means, and causes the control means to be transmitted from another device. A communication step for receiving a voice code obtained by coding a voice related to a work by a predetermined method, and evaluating the emotion of the voice using the received voice code, and using the evaluation to evaluate a work related to the voice code. And an operation evaluation step.

  Further, the work management method according to the present application is a work management method for performing a work management procedure using a computer, wherein the computer includes a control unit, and the control unit is configured to output a sound related to the work from another device. A communication step for accepting a voice code encoded by the method, a work evaluation step for evaluating the emotion of the voice using the received voice code, and using the evaluation to evaluate a work related to the voice code, Execute.

  According to the present invention, it is possible to provide a technique for more accurately evaluating an operator. However, problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

It is a figure showing an example of composition of a work management system concerning an embodiment. It is a figure which shows the example of the data structure stored in an emotion analysis memory | storage part. It is a figure which shows the example of the data structure stored in an emotion reference | standard memory | storage part. It is a figure which shows the example of the data structure stored in a work performance memory | storage part. It is a figure which shows the example of the data structure stored in an evaluation basic information storage part. It is a figure which shows the example of the data structure stored in a personal reception memory | storage part. It is a figure which shows the example of the data structure stored in a work information storage part. It is a figure which shows the hardware constitutions of an emotion analysis terminal. It is a figure which shows the hardware constitutions of a work management apparatus. It is a figure which shows the flow of an emotion evaluation process. It is a figure which shows the flow of a work evaluation process. It is a figure which shows the example of a customer's emotion fluctuation graph. It is a figure which shows the example of the work evaluation result screen displayed in a work evaluation process. It is a figure which shows the flow of a cause search process. It is a figure which shows the example of the cause display screen displayed in a cause search process. It is a figure which shows the structural example of a complaint prevention information storage part. It is a figure which shows the flow of a claim prevention process. It is a figure which shows the example of the prevention information display screen displayed in a claim prevention process. It is a figure which shows another example of the prevention information display screen displayed in a claim prevention process.

  Hereinafter, an embodiment using a work management system 1 to which a first embodiment according to the present invention is applied will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration example of a work management system 1 according to the first embodiment of the present invention. The work management system 1 assumed in the present embodiment includes an emotion analysis terminal 100 and a work management apparatus 200. In this system, the worker 10 attaches the emotion analysis terminal 100 to the work target device 300 while wearing the emotion analysis terminal 100 on the body (for example, the face when the emotion analysis terminal 100 is a glasses-type device, so-called smart glasses). On the other hand, a predetermined work is performed. For example, maintenance work, installation work, inventory work, and the like of the work target device 300 are performed.

  The emotion analysis terminal 100 is an apparatus called a so-called wearable device including an entire computer that can be detachably attached to a part of the body of the worker 10. Desirably, the emotion analysis terminal 100 is worn by the worker and can acquire image information that is similar to or wider than the worker's field of view. Voice information perceived by the worker, that is, voice generated by the worker 10 itself. Alternatively, voice information that can be heard by the operator 10 can be obtained. The smart glass is one of such so-called wearable devices, and since it is difficult for the operator's hands to be restrained at the time of wearing, the support of the worker aiming at improving the work quality can be performed more efficiently. A plurality of emotion analysis terminals 100 are included in the work management system 1.

  The emotion analysis terminal 100 may be a terminal owned by the worker 10 or a terminal lent to the worker from an organization to which the worker 10 belongs. The work management apparatus 200 includes a process for evaluating work performed by the worker 10, a support process for identifying factors that cause the worker 10 to significantly change the feeling of the person in charge at the work destination, and other work. A process for notifying the worker 10 in advance of points to note when dealing with the person in charge of the work destination, including the worker 10 and the like.

  The emotion analysis terminal 100 and the work management apparatus 200 are connected via a network such as a public wireless network or a mobile phone network. The work management device 200 is a mobile phone device including, for example, a server device, a personal computer, a so-called smart phone, a tablet terminal, and the like, and is a processing device that can communicate with other devices.

  The emotion analysis terminal 100 includes a display control unit 110, a position specifying unit 120, a voice acquisition unit 130, a communication unit 140, an image processing unit 150, a voice analysis unit 160, a facial expression analysis unit 170, and an emotion analysis storage. Unit 180, emotion criteria storage unit 181, work performance storage unit 182, basic evaluation information storage unit 183, personal interaction storage unit 184, and emotion analysis unit 190.

  The display control unit 110 includes a wearer display unit 111 and a notification output unit 112. The wearer display unit 111 performs display for the wearer of the emotion analysis terminal 100, that is, the worker 10. In other words, the wearer display unit 111 is an inner display unit. For example, the visual display information is projected onto a lens surface close to the wearer side of the glasses so that the wearer can superimpose the information on the optical image through the lens. Is displayed.

  The notification output unit 112 performs display for a wearer of the emotion analysis terminal 100, that is, a person other than the worker 10. The notification output unit 112 controls the blinking of a light emitter such as an LED (Light Emitted Diode), for example, so that the emotion analysis terminal 100 is operating for a person who can visually recognize the wearer from the outside. That is, it is an outer display portion that indicates that there is.

  The position specifying unit 120 receives the satellite wave and specifies the three-dimensional coordinates of the current location as the position where the emotion analysis terminal 100 exists. Specifically, the position specifying unit 120 receives satellite waves transmitted from three or more artificial satellites such as GPS (Global Positioning System), GLONASS (Global Navigation Satellite System), and the like using the antenna. The position where the emotion analysis terminal 100 exists is specified using the position information obtained by determining the three-dimensional coordinates of the current location based on the above.

  The voice acquisition unit 130 acquires voice information regarding work. Specifically, the voice acquisition unit 130 acquires voice information outside the emotion analysis terminal 100 and stores it in the work record storage unit 182 as a voice recording file. The communication unit 140 controls communication with other devices. Specifically, the communication unit 140 outputs the voice code information to the work management apparatus 200 which is another apparatus together with the associated work information. The communication unit 140 is preferably a unit that controls short-range radio communication using, for example, Bluetooth, wireless LAN, NFC (Near Field Communication), or the like. However, it may control predetermined wired communication such as USB.

  The image processing unit 150 includes an image acquisition unit 151 and an image superimposing unit 152. The image acquisition unit 151 acquires an external image acquired by the emotion analysis terminal 100, that is, image information related to the worker's field of view. More ideally, the image acquisition unit 151 acquires an image in a range including all of the worker's field of view.

  When the image superimposing unit 152 receives information such as an image or text to be superimposed and displayed from the work management apparatus 200 or the like, the image superimposing unit 152 outputs the information to the worker 10 so as to be superimposed on the field of view of the worker 10. That is, in the case of a wearable device such as a headset terminal that outputs a captured image so as to cover the field of view of the worker 10 with a liquid crystal image or the like, the image superimposing unit 152 displays information by providing a transmission layer on the captured image. To do. When the wearable device is a spectacle-type wearable device that outputs a part of the field of view of the worker 10, the information is displayed.

  The voice analysis unit 160 analyzes the voice information acquired by the voice acquisition unit 130. Specifically, the voice analysis unit 160 encodes the voice information according to a predetermined standard for each predetermined type of attribute. For example, the voice analysis unit 160 quantifies each attribute such as pitch, voice volume, speech rate, voice content, etc., performs coding according to the quantified attribute amount, and associates it with the work.

  The facial expression analysis unit 170 analyzes the image information acquired by the image acquisition unit 151. Specifically, the facial expression analysis unit 170 recognizes the face area included in the image information and specifies the degree of smile based on the arrangement of predetermined feature points. For example, assuming that a full smile is 100% smile, the degree of smile is specified such that the smile rate decreases as the expression becomes closer. Note that the facial expression analysis unit 170 is not limited to the above analysis, and for example, a facial expression may be specified in consideration of changes in facial color.

  The emotion analysis storage unit 180 stores the result of emotion analysis. Specifically, emotion analysis storage unit 180 stores a result of emotion analysis using a code that is a result of analysis by voice analysis unit 160 and a degree of smile that is analyzed by facial expression analysis unit 170.

  FIG. 2 is a diagram illustrating a data structure of the emotion analysis storage unit 180. The emotion analysis storage unit 180 includes an implementation date 180A, a time 180B, a pitch 180C, a voice volume 180D, an utterance speed 180E, a voice 180F, a smile 180G, and a comprehensive determination 180H.

  The implementation date 180A is information that identifies the date on which the operator 10 performed the operation. The time 180B is information for specifying the time when the work is performed. The pitch 180C is information obtained by coding the average pitch for a predetermined period among the attributes of the voice information acquired at the time 180B on the implementation date 180A. The voice volume 180D is information obtained by coding the average of the volume for a predetermined period among the attributes of the voice information acquired at the time 180B on the implementation date 180A.

  The speech rate 180E is information obtained by coding the average speech rate for a predetermined period among the attributes of the voice information acquired at the time 180B on the implementation date 180A.

  The voice 180F is information calculated and coded according to a predetermined score for each predetermined phrase included in the utterance content among the attributes of the voice information acquired at the time 180B of the implementation date 180A.

  The smile 180G is information obtained by coding the average smile degree for a predetermined period among the smile degrees of the face included in the image information acquired at the time 180B on the implementation date 180A.

  Comprehensive determination 180H uses the degree of smile of the face included in the image information acquired at time 180B on the implementation date 180A and the information of the voice code included in the audio information acquired at time 180B on the implementation date 180A. It is the information calculated.

  The emotion reference storage unit 181 stores information serving as a reference used when coding voice information when performing emotion analysis.

  FIG. 3 is a diagram illustrating a data structure of the emotion criterion storage unit 181. The emotion reference storage unit 181 stores information for associating a code range with a reference range for specifying an emotion for each attribute of audio information. Specifically, the type 181A, the degree 181B, the range 181C, the emotion code (male voice) 181D, and the emotion code (female voice) 181E are stored in the emotion reference storage unit 181 in association with each other. .

  The type 181A is information that specifies the type of attribute of audio information. For example, the type 181A includes types such as a pitch (Octabe), a voice volume (decibel), a speech rate (pitch), and voice. The degree 181B is an expression serving as a reference for determining the amount of the quantified amount for each type specified by the type 181A. The range 181C is a range of amounts that can be taken for each degree 181B. For example, the range where the degree of pitch is “low” is between 1 degree and 3 degrees, the range where “normal” is between 4 degrees and 6 degrees, and the range where “high” is 7 degrees. The range is from 9 degrees to 9 degrees. The emotion code (male voice) 181D is a code value when the analysis target voice is a male voice. The emotion code (female voice) 181E is a code value when the analysis target voice is a female voice. The reason why the analysis target speech is divided between males and females is that, for example, even if they have the same pitch, they are encoded relatively when, for example, they have a high pitch as a male voice but a low pitch as a female voice. Or, when feeling comfortable, there may be cases where the vertical change in pitch is reversed between men and women, so that different codes are given.

  The work record storage unit 182 stores the work record of the worker 10, that is, the voice and image information recorded when the work is actually performed.

  FIG. 4 is a diagram illustrating a data structure of the work record storage unit 182. As illustrated in FIG. The work result storage unit 182 stores, for each work, a manual to be used, information for identifying files such as audio and video collected in the actual work, and a satisfaction amount, a dissatisfaction amount, a difference with respect to the code amount of the audio and video. The amount is stored in association with each other. Specifically, the work record storage unit 182 includes a device identifier 182A, an implementation date 182B, a recording start time 182C, a work identifier 182D, a work manual 182E, a recording information type 182F, and a recording file name 182G. The threshold excess count 182H, the satisfaction amount 182J, the dissatisfaction amount 182K, the difference 182L, and the notice 182M are stored in association with each other.

  The device identifier 182A is information for specifying the work target device 300. The implementation date 182B is information that identifies the date on which the work was actually performed. The recording start time 182C is information for specifying the time when the image acquisition at the time of work is started. The work identifier 182D is information that identifies the content of the work. The work manual 182E is information for specifying a manual in which details of work are described.

  The recorded information type 182F is information for specifying whether the recorded information is a sound or an image. The recording file name 182G is information for specifying a file in which one or both of sound and image are recorded. The threshold excess count 182H is information for specifying how many times the degree of satisfaction or dissatisfaction exceeds a predetermined threshold in the work recording. The satisfaction amount 182J is information for specifying the satisfaction degree totaled in the work record. The dissatisfaction amount 182K is information for specifying the degree of dissatisfaction totaled in the work records. The difference 182L is the difference between the comprehensive determination at the start of work recording and the comprehensive determination at the end of work recording. The notice 182M is information indicating a matter noticed in the work. For example, the notice 182M includes supplementary items of the work manual, cautions specific to the work target device 300, and the like.

  The evaluation basic information storage unit 183 stores information used for evaluation for each work of the worker 10.

  FIG. 5 is a diagram illustrating a data structure of the evaluation basic information storage unit 183. The evaluation basic information storage unit 183 includes, for each worker's work, voice information, a customer coefficient for adjusting the variation of the customer's evaluation standard, a work coefficient for adjusting the variation of the work difficulty level, Are stored in association with each other. Specifically, the evaluation basic information storage unit 183 includes an operator identifier 183A, an implementation date 183B, a start time 183C, a threshold excess count 183D, a satisfaction amount 183E, a dissatisfaction amount 183F, and a customer coefficient 183G. The work coefficient 183H and the difference 183J are stored in association with each other.

  The worker identifier 183A is information that identifies the worker 10. The implementation date 183B is information that identifies the date on which the work was actually performed. The start time 183C is information for specifying the earliest time among the times when the acquisition of the image or sound during work is started. The threshold excess count 183D is information that specifies how many times the degree of satisfaction or dissatisfaction exceeds a predetermined threshold in the work recording. The satisfaction amount 183E is information for specifying the satisfaction degree totaled in the work record. The dissatisfaction amount 183F is information for specifying the degree of dissatisfaction added up in the work record.

  The customer coefficient 183G is a coefficient for adjusting variation in evaluation criteria of a person in charge who performs management related to work. In general, even if the same work is performed with the same content, the evaluation from the person in charge often differs depending on the person in charge of managing the work. The coefficient is used so that the evaluation does not differ greatly between workers who perform a lot of work under the person in charge who makes a good evaluation relatively easily and workers who do a lot of work under the person in charge who does not. It can be adjusted.

  The work coefficient 183H is a coefficient for adjusting variation in the difficulty level of work. In general, those who are excellent in work skills often perform work with higher difficulty, and those who do not often perform work with lower difficulty. Since it is not appropriate that a person who has performed a low-level work without an accident has a significantly higher evaluation than a person who has generated a small accident in a high-level work, the work factor is The coefficient to adjust.

  The difference 183J is a difference between the comprehensive determination at the start of work recording and the comprehensive determination at the end of work recording.

  The personal interaction storage unit 184 stores voice and image information recorded when the actual work is performed regarding the personal interaction during the work, that is, the conversation with the person in charge of the work.

  FIG. 6 is a diagram illustrating a data structure of the personal reception storage unit 184. The personal interaction storage unit 184 associates information for identifying files such as audio and video collected in actual work with satisfaction, dissatisfaction, and difference for the code amount of the audio and video for each work. Store with attachments. Specifically, in the personal reception storage unit 184, the execution date 184A, the recording start time 184B, the work identifier 184C, the recording information type 184D, the recording file name 184E, the threshold excess count 184F, and the satisfaction amount 184G. , The dissatisfaction amount 184H, the difference 184J, and the notice 184K are stored in association with each other.

  The implementation date 184A is information that identifies the date on which the work was actually performed. The recording start time 184B is information for specifying the time when image acquisition at the time of conversation is started. The work identifier 184C is information that identifies the content of the work.

  The recorded information type 184D is information that specifies whether the recorded information is a sound or an image. The recording file name 184E is information for specifying a file in which either or both of sound and image are recorded. The threshold excess count 184F is information for specifying how many times the degree of satisfaction or dissatisfaction exceeds a predetermined threshold in the conversation recording. The satisfaction amount 184G is information for specifying the satisfaction degree totaled in the conversation record. The dissatisfaction amount 184H is information for specifying the degree of dissatisfaction added up in the conversation record. The difference 184J is a difference between the comprehensive determination at the start of work recording and the comprehensive determination at the end of work recording. The notice 184K is information indicating a matter noticed in the conversation. For example, the notice 184K includes unique cautions such as a speech style preferred by a person in charge of managing work, a policy of an organization to which the person in charge of managing work belongs, and the like.

  The emotion analysis unit 190 analyzes emotion using either or both of the voice information analyzed by the voice analysis unit 160 and the facial expression analyzed by the facial expression analysis unit. Specifically, the emotion analysis unit 190 associates the voice code information obtained as a result of the analysis by the voice analysis unit 160 with the facial expression information analyzed by the facial expression analysis unit 170 to determine whether the emotion is good or bad. Identify.

  The work management apparatus 200 is connected to the emotion analysis terminal 100 via a network. The work management apparatus 200 includes a display unit 210, a communication unit 220, a voice processing unit 231, a facial expression processing unit 232, a cause search unit 233, a work evaluation unit 234, a knowledge management unit 235, and a storage unit 250. And are included.

  The display unit 210 performs display output used for operations of the work management apparatus 200.

  The communication unit 220 receives a voice code obtained by coding a voice related to work by a predetermined method from the emotion analysis terminal 100 which is another device. The communication unit 220 receives a voice code and also receives image information related to the voice code.

  When the voice processing unit 231 receives a voice input used for the operation of the work management apparatus 200, the voice processing unit 231 stores the received voice in the storage unit 250 as a voice file.

  The facial expression processing unit 232 analyzes the facial expression of the image indicating the face included in the image information. The facial expression processing unit 232 recognizes the face based on the shape of the display object included in the image, performs face pattern matching processing, and the like, and specifies the degree of smile indicated by the facial expression.

  In the result of the work evaluation performed by the work evaluation unit 234, the cause search unit 233, when there is a change exceeding a predetermined degree in the quality of the emotion, is past a predetermined time (for example, for example) Audio information and related image information are reproduced starting from 2 minutes ago.

  The work evaluation unit 234 evaluates a voice emotion using a voice code based on the voice information related to the work received from the emotion analysis terminal 100 which is another device, and evaluates the emotion evaluation to evaluate the work related to the voice code. Use. In addition, the work evaluation unit 234 evaluates the emotion of the voice using the geometric information of the waveform formed by the temporal change of the quality of the emotion specified using the voice code. The work evaluation unit 234 evaluates emotions according to the degree of difficulty related to the work, and evaluates the work by adding corrections based on the difficulty level to the evaluation of emotions. In addition, the work evaluation unit 234 evaluates emotions according to the person in charge who manages the work, and evaluates the work by adding corrections based on the characteristics of the person in charge to the evaluation of emotions.

  In addition, when there is image information associated with the work, the work evaluation unit 234 evaluates the work by adding correction based on the emotion specified by the facial expression processing unit 232 to the evaluation of the voice emotion. Specifically, when the facial expression is a smile of a predetermined level or more, the emotion may be increased in a good direction, or the emotion code of a predetermined voice attribute such as a pitch is increased in the good direction. You may do it.

  The knowledge management unit 235 reads knowledge information associated with the work-related voice code from the work result storage unit 252 and the personal interaction storage unit 254 described later, and specifies and outputs them.

  The storage unit 250 includes a work information storage unit 251, a work record storage unit 252, an evaluation basic information storage unit 253, and a personal interaction storage unit 254.

  The work information storage unit 251 stores information related to work schedules.

  FIG. 7 is a diagram showing a data structure of the work information storage unit 251. As shown in FIG. The work information storage unit 251 stores, for each work target device 300, information specifying the date and time of the work, the manual used in the work, and the worker who is the person in charge of the work in association with each other. Specifically, the work information storage unit 251 stores the device identifier 251A, the execution date 251B, the work start time 251C, the work identifier 251D, the work manual 251E, and the person in charge 251F in association with each other. Has been.

  The device identifier 251A is information for specifying the work target device 300. The implementation date 251 </ b> B is information that identifies the date on which work is actually performed. The work start time 251 </ b> C is information for specifying the time to start work. The work identifier 251D is information that identifies the content of the work. The work manual 251E is information for specifying a manual in which details of work are described. The person in charge 251F is information for specifying a person in charge who manages the work. For example, it is information that identifies the work requester or the person in charge of managing the work target device 300.

  Each of the work performance storage unit 252, the basic evaluation information storage unit 253, and the personal interaction storage unit 254 has the same data structure as the work performance storage unit 182, the basic evaluation information storage unit 183, and the personal interaction storage unit 184 of the emotion analysis terminal 100. Is provided. Therefore, explanation is omitted.

  FIG. 8 is a diagram illustrating a hardware configuration of the emotion analysis terminal 100. The emotion analysis terminal 100 includes a voice reception device 101, a calculation device 102, an external output device 103, a main storage device 104, an imaging device 105, a wearer output device 106, a communication device 107, and an acceleration sensor device 108. And an external storage device 109 and a bus 100b for connecting the devices. In addition, the emotion analysis terminal 100 may include a contact input device. In the case of a glasses-type wearable device, for example, the contact input device may accept a contact to the vine portion of the glasses as a click input, and may accept a direction of sliding in the contact state as a scroll input. It is not limited to this.

  The voice reception device 101 collects sound including speech uttered by the worker 10 wearing the emotion analysis terminal 100 and surrounding sounds. For example, one or more microphones.

  The arithmetic device 102 is an arithmetic device such as a CPU (Central Processing Unit).

  When the emotion output terminal 100 is worn, the external output device 103 emits light such as an LED lamp by emitting light so that it can be seen from the outside of the wearer, specifically, from the outside. .

  The main storage device 104 is a memory device such as a RAM (Random Access Memory).

  The imaging device 105 is a camera device that acquires video information similar to or wider than the field of view of the worker wearing the emotion analysis terminal 100.

  The wearer output device 106 is a display device such as a projection device that projects an image on an optical lens surface so as to be visible to the wearer, or a transflective display device provided according to the optical lens surface. However, the present invention is not limited to this, and for example, a display device capable of irradiating light to be imaged on the retina of the worker may be used.

  The communication device 107 is a device that establishes a communication path with other devices such as the work management device 200 via a wireless network and transmits / receives information.

  The acceleration sensor device 108 is a device that detects a movement or posture change of the wearer 10 by acquiring a predetermined dimension of acceleration and specifying the acceleration direction and the magnitude of the acceleration.

  The external storage device 109 is a non-volatile storage device that can store digital information, such as a so-called hard disk (Hard Disk Drive), SSD (Solid State Drive), or flash memory.

  The display control unit 110 of the emotion analysis terminal 100 described above is realized by the external output device 103 and the wearer output device 106. The position specifying unit 120 is realized by a position detection device (not shown). The voice acquisition unit 130 is realized by a program that causes the arithmetic device 102 to perform processing and the voice reception device 101. This program is stored in the main storage device 104 or a ROM device (not shown), loaded onto the main storage device 104 for execution, and executed by the arithmetic device 102.

  The communication unit 140 is realized by the communication device 107. In addition, each of the image processing unit 150, the voice analysis unit 160, the facial expression analysis unit 170, and the emotion analysis unit 190 is realized by a program that causes the calculation device 102 to perform processing and the imaging device 105. This program is stored in the main storage device 104 or a ROM device (not shown), loaded onto the main storage device 104 for execution, and executed by the arithmetic device 102.

  The emotion analysis storage unit 180, the emotion reference storage unit 181, the work performance storage unit 182, the evaluation basic information storage unit 183, and the personal response storage unit 184 are each realized by the main storage device 104 and the external storage device 109.

  The hardware configuration of the emotion analysis terminal 100 has been described above.

  FIG. 9 is a diagram illustrating a hardware configuration of the work management apparatus 200. The work management device 200 includes at least an input receiving device 201, a computing device 202, an external storage device 203, a main storage device 204, a communication device 205, an output device 206, and a bus 211 that connects the devices. .

  The input reception device 201 is various input devices such as a keyboard, a mouse, and a touch panel.

  The arithmetic device 202 is an arithmetic device such as a CPU.

  The external storage device 203 is a non-volatile storage device such as a so-called hard disk, SSD, or flash memory that can store digital information.

  The main storage device 204 is a memory device such as a RAM.

  The communication device 205 is a device such as a network card that establishes a communication path with other devices such as the emotion analysis terminal 100 via a network or the like and transmits / receives information.

  The output device 206 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display.

  The display unit 210 of the work management device 200 described above is realized by the output device 206. The communication unit 220 is realized by the communication device 205. The voice processing unit 231, the facial expression processing unit 232, the cause searching unit 233, the work evaluation unit 234, and the knowledge management unit 235 are realized by a program that causes the arithmetic device 202 to perform processing. This program is stored in the main storage device 204 or a ROM device (not shown), loaded onto the main storage device 204 for execution, and executed by the arithmetic device 202.

  The storage unit 250 is realized by the external storage device 203 and the main storage device 204. The hardware configuration of the work management apparatus 200 has been described above.

  Note that the configurations of the emotion analysis terminal 100 and the work management device 200 can be classified into more components depending on the processing content. Moreover, it can also classify | categorize so that one component may perform more processes.

  Each control unit may be constructed by hardware (ASIC, GPU, etc.). Further, the processing of each control unit may be executed by one hardware, or may be executed by a plurality of hardware.

  [Description of Operation] Next, the operation of the work management system 1 in the present embodiment will be described. FIG. 10 is a flowchart of emotion evaluation processing in the present embodiment.

  The emotion evaluation process is started when the emotion analysis terminal 100 and the work management apparatus 200 are in an activated state and the emotion analysis terminal 100 receives a predetermined activation instruction.

  First, the emotion analysis unit 190 acquires work information (step S001). Specifically, emotion analysis unit 190 acquires information on the next work to be performed among the work information distributed in advance. For example, if there is no information on work corresponding to the current date or time, or information on work that has not been carried out but is scheduled to finish before the current date and time but has not yet been carried out, the current information Information on work that is scheduled to be performed after the date and time but has not yet been performed is requested and acquired from the work management apparatus 200 or the like.

  Then, the audio acquisition unit 130 and the image acquisition unit 151 acquire audio information and video information, respectively (step S002).

  Then, the voice analysis unit 160 scores the pitch, voice volume, speech rate, and content of the voice information according to the standard, and specifies the smile degree from the video information (step S003). Specifically, the voice analysis unit 160 quantitatively measures each attribute of the voice information, determines which range 181C of the emotion reference storage unit 181 includes the measured value, and the emotion code corresponding to the range Is identified. Then, the facial expression analysis unit 170 specifies a face area included on the video of the video information, and specifies the degree of smile. There are various methods for specifying the smile level. For example, the smile level may be estimated by pattern matching with a reference image corresponding to the prepared smile level of the subject. Alternatively, the degree of smile may be estimated according to how the corners of the mouth rise and how the eyes turn down.

  Then, emotion analysis unit 190 transmits the scoring result to work management device 200 (step S004).

  The emotion analysis unit 190 determines whether or not the day's work has been completed. Specifically, emotion analysis unit 190 refers to the information on the current day's work acquired in step S001, and determines whether or not the current day's work has been completed. If the current day's work is not yet finished, emotion analysis unit 190 returns control to step S002.

  If the current day's work has been completed (“Yes” in step S005), emotion analysis unit 190 requests work management apparatus 200 for an evaluation result (step S006). Note that the emotion analysis unit 190 exports audio information and video information related to the current day's work to a predetermined file format and transmits them to the work management apparatus 200.

  Then, emotion analyzer 190 receives and outputs display information indicating the evaluation result (step S007). Specifically, when the emotion analysis unit 190 receives the evaluation result requested in step S006 from the work management apparatus 200, the emotion analysis unit 190 passes the evaluation result to the image superimposing unit 152 to form an output image, and the output image is sent to the wearer display unit 111. Instruct to display.

  The above is the process flow of the emotion evaluation process. According to the emotion evaluation process, the worker 10 can collect the emotion evaluation information and transmit the encoded result to the work management apparatus 200 and receive the evaluation transmitted from the work management apparatus 200.

  FIG. 11 is a flowchart of the work evaluation process in the present embodiment. The work evaluation process is started when a scoring result is transmitted to the work management apparatus 200 in the emotion evaluation process.

  The communication unit 220 receives the scoring result and the smile level (step S101). Specifically, the communication unit 220 receives the result of scoring emotion and the degree of smile from the emotion analysis terminal 100 which is an external device.

  Then, the work evaluation unit 234 stores the scoring result and the degree of smile in the evaluation basic information storage unit 253 and the personal reception storage unit 254 (step S102). Specifically, the work evaluation unit 234 stores each of the implementation date, the start time, the number of times exceeding the threshold, the satisfaction amount, the dissatisfaction amount, and the difference among the codes included in the evaluation result received in step S001. Stored in the unit 253. In addition, the work evaluation unit 234, among the codes included in the evaluation result received in step S001, each of the execution date, the start time, the work identifier, the recording file name, the number of times the threshold is exceeded, the satisfaction amount, the dissatisfaction amount, and the difference. Stored in the personal interaction storage unit 254. Note that the work evaluation unit 234 stores the start time as the recording start time when storing it in the personal interaction storage unit 254.

  Then, the work evaluation unit 234 identifies a worker who uses the emotion analysis terminal 100 that is the transmission source of the data received in step S101, and determines whether or not the work of the worker on that day has been completed ( Step S103). Specifically, the work evaluation unit 234 refers to the work information storage unit 251 of the worker, and determines that the work on the current day has been completed when there is no work that has not been performed as the work on the current work day. If the current day's work is not finished (“No” in step S103), the work evaluation unit 234 returns control to step S101.

  When the current day's work is completed (“Yes” in step S103), the work evaluation unit 234 identifies a customer coefficient corresponding to the characteristics of the person in charge who manages the work (step S104). Specifically, the work evaluation unit 234 specifies a customer coefficient in order to evaluate the work by making corrections according to the characteristics of the person in charge who manages the work. For example, the work evaluation unit 234 reads the satisfaction amount and dissatisfaction amount related to the person in charge who has been performed in the past from the personal interaction storage unit 254, and when the dissatisfaction amount is greater than a predetermined level, , Set the customer coefficient high (that is, make the evaluation score high). Conversely, the work evaluation unit 234 sets the customer coefficient low when the satisfaction amount is large at a predetermined level or higher (that is, the evaluation score is lowered).

  Then, the work evaluation unit 234 specifies a work coefficient in accordance with the work content (step S105). Specifically, the work evaluation unit 234 specifies a work coefficient corresponding to the difficulty level according to the difficulty level predetermined for each work. For example, the work evaluation unit 234 sets the work coefficient low for easy work and sets the work coefficient high for difficult work.

  Then, the work evaluation unit 234 calculates an evaluation result, generates output information, and transmits the output information to the emotion analysis terminal 100 (step S106). Specifically, the work evaluation unit 234 subtracts the dissatisfaction amount from the satisfaction amount calculated in the work unit, adds the customer coefficient and the work coefficient, and calculates the evaluation result. Then, the work evaluation unit 234 aggregates the evaluation results for each worker and generates output information. For example, when the first work is 5 points, the second work is 10 points, and the third work is 6 points, information of 5 + 10 + 6 = 21 points is generated in the output information of the worker. And work evaluation part 234 will specify and transmit output information for every emotion analysis terminal 100, if a request is received from emotion analysis terminal 100. FIG.

  The above is the flow of the work evaluation process. According to the work evaluation process, it is possible to obtain an evaluation value of work through the implementation date for each worker. Moreover, since the evaluation value takes into account the characteristics of the person in charge who manages the work and the difficulty level of the work, a highly valid evaluation value can be obtained.

  FIG. 12 is a diagram illustrating an example of a customer emotion fluctuation graph 400. In the customer emotion fluctuation graph 400, the vertical axis 401 plots the degree of satisfaction regarding satisfaction, and the horizontal axis 402 plots the passage of time. Then, by recording the degree of emotion of the customer, that is, the person in charge of work management at a certain point in time, it becomes possible to quantitatively grasp the emotion. Each of the emotion analysis unit 190 of the emotion analysis terminal 100 and the work evaluation unit 234 of the work management device 200 is satisfied with a geometric area in which the points on the graph are considered to be continuous with respect to a portion exceeding the reference emotion level. Specify the degree. In addition, each of the emotion analysis unit 190 of the emotion analysis terminal 100 and the work evaluation unit 234 of the work management apparatus 200 has a geometric area in which the points on the graph are regarded as continuous with respect to a portion below the standard emotion level. To identify the degree of dissatisfaction. In addition, each of the emotion analysis unit 190 and the work evaluation unit 234 obtains the difference between the emotion level at the start of recording and the emotion level at the end of recording, and specifies the difference.

  FIG. 13 is a diagram illustrating an example of a work evaluation result screen 500 displayed in the work evaluation process. The work evaluation result screen 500 includes a review display area 501 that is an evaluation result and a barometer display area 502. In the comment display area 501, text predetermined according to the evaluation value is displayed. In the barometer display area 502, the evaluation value of the day is displayed. In the barometer display area 502, the ratio of the evaluation value to the total score of all work is displayed.

  FIG. 14 is a flowchart of cause search processing in the present embodiment.

  The cause search process is started when the work management apparatus 200 is in an activated state and the work management apparatus 200 receives a predetermined activation instruction.

  First, the cause search unit 233 specifies the date and emotion value of the target work (step S201). Specifically, the cause search unit 233 specifies a target to be a cause search target by receiving the start date and time of the target work and emotion value information in the work from the outside.

  And the cause search part 233 specifies the time when the emotion value deteriorated (step S202). Specifically, the cause search unit 233 plots the emotion values identified in step S201 in order of time, and identifies the fluctuation as a deterioration of the emotion value when the emotion value has deteriorated during a predetermined period.

  And the cause search part 233 determines whether the degree of deterioration of an emotion value is more than predetermined (step S203). Specifically, the cause search unit 233 specifies the deterioration amount per unit time of the emotion value for the predetermined time with the time specified in step S202 as a reference, and whether there is a deterioration exceeding a predetermined threshold value. Determine whether. If none of the deterioration levels is greater than or equal to the predetermined level (if “No” in step S203), the cause search unit 233 ends the cause search process.

  If the degree of deterioration of the emotion value is greater than or equal to a predetermined value (if “Yes” in step S203), the cause search unit 233 identifies audio and video information a predetermined time before the deteriorated time (step S203). S204). In general, when the degree of deterioration of the emotion value is greater than or equal to a predetermined level, it is considered that some act that causes it is often included immediately before that, for example, the time when the period in which the deterioration is observed starts If the audio or video is reproduced from about 2 minutes in the past, it may be easy to find the act that caused the deterioration. Therefore, the cause search unit 233 identifies audio and video information a predetermined time before the deteriorated time. In the processing, the cause search unit 233 identifies and reads the recording file included in the personal interaction storage unit 254, and after the time indicating a predetermined time before the deteriorated time (for example, 2 minutes before), until the deteriorated time Identify the information.

  And the cause search part 233 reproduces | regenerates an audio | voice and video information (step S205). Specifically, the cause search unit 233 reproduces the audio and video information specified in step S204.

  The above is the flow of the cause search process. According to the cause search process, it is possible to identify a record in which emotions have deteriorated sharply and reproduce the record immediately before the cause is considered to be recorded.

  FIG. 15 is a diagram illustrating an example of a cause display screen 600 displayed in the cause search process. The cause display screen 600 includes a cause explanation text display area 601, an emotion change display area 602, and a related moving image reproduction instruction reception area 603. In the cause explanation text display area 601, text information that is considered to be the cause of emotional deterioration is displayed. In the emotion change display area 602, emotion changes are displayed in time series. The related moving image reproduction instruction reception area 603 receives an instruction to reproduce audio or video information from a file in which a causal event is recorded. When the input is received in the related video playback instruction receiving area 603, the cause searching unit 233 allocates the recording file of the work as the related video and plays back the related video display screen 610 so that playback starts from the corresponding time. , Cause explanation sentence is output as voice.

  The above is the work management system 1 according to the first embodiment. According to the first embodiment, it is possible to support the work of evaluating the worker more accurately.

  The present invention is not limited to the first embodiment described above. The first embodiment described above can be variously modified within the scope of the technical idea of the present invention. For example, in the first embodiment described above, the emotion analysis terminal 100 is assumed to be a glasses-type wearable device, but is not limited thereto. For example, it may be a helmet-type wearable device that covers the head more widely, or a wearable device that is attached to only one eye. Moreover, the emotion analysis terminal 100 and the work management apparatus 200 may be integrated.

  FIGS. 16-19 is a figure which shows the structural example of the work management system which concerns on 2nd embodiment which provides the knowledge regarding work to the operator 10 before implementation of work among such modifications. In the second embodiment, the work management apparatus 200 has a configuration for providing knowledge information to the emotion analysis terminal 100. Such a second embodiment will be described. Note that the second embodiment is basically the same as the first embodiment, and therefore, a description will be given focusing on different configurations.

  FIG. 16 is a diagram illustrating a configuration example of the complaint prevention information storage unit 255 provided in the work management apparatus 200. The complaint prevention information storage unit 255 stores complaint prevention information that the worker should know in advance so as not to generate unnecessary complaints from the person in charge of the work or the client. The complaint prevention information storage unit 255 includes a work identifier 255A, an execution date 255B, a start time 255C, a work manual 255D, a difference 255E, a caution item 255F, and a knowledge video 255G.

  The work identifier 255A is information that identifies the content of the work. The implementation date 255B is information that identifies the date on which work is actually performed. The start time 255C is information for specifying the time to start work. The work manual 255D is information for identifying a manual in which details of work are described. The difference 255E is a difference obtained by subtracting the emotion value at the work start time from the emotion value at the work end time of the person in charge of work management. In the notes 255F, items to be noted as knowledge are arbitrarily stored. In the knowledge video 255G, video, images, audio information, and the like collected as knowledge are displayed for each work identifier 255A.

  FIG. 17 is a diagram illustrating a processing flow of the complaint prevention processing. When the work management apparatus 200 receives a request for execution of the complaint prevention process from a device external to the work management apparatus 200, the work management apparatus 200 starts the complaint prevention process.

  First, the complaint prevention information storage unit 255 receives information for identifying a work manual and information for identifying a person in charge of managing work from an external device that has requested execution of the complaint prevention process (step S301). Specifically, the complaint prevention information storage unit 255 receives an identifier for specifying a work manual, an organization and name of a person in charge.

  Then, the complaint prevention information storage unit 255 identifies the attention item and knowledge video corresponding to the information for identifying the work manual and the information for identifying the person in charge of managing the work (step S302). Specifically, the complaint prevention information storage unit 255 searches for the work manual 252E of the work record storage unit 252 based on the information for specifying the work manual, and reads the notice 252M related to the corresponding record, thereby specifying the precautions. . Further, the complaint prevention information storage unit 255 specifies a record in which a combination of information for specifying a work manual and information for specifying a person in charge of work is a combination of the work manual 251E and the person in charge 251F of the work information storage unit 251. The corresponding work identifier 251D is read out. Then, the complaint prevention information storage unit 255 searches the work identifier 254C of the personal interaction storage unit 254 with the read work identifier 251D, and reads the notice 254K related to the corresponding record, thereby specifying the caution item and the knowledge video. .

  Then, the complaint prevention information storage unit 255 transmits the caution item and the knowledge video (step S303). Specifically, the complaint prevention information storage unit 255 transmits output information including the attention item specified in step S302 and the knowledge video to an external device that has requested execution of the complaint prevention process.

  The above is the flow of complaint prevention processing. According to the complaint prevention process, it is possible to receive information on the work content and the person in charge of managing the work, and share knowledge about the work to be performed with other workers. Can be prevented from occurring.

  FIG. 18 is a diagram illustrating an example of a prevention information display screen 700 displayed in the complaint prevention process. The preventive information display screen 700 includes a caution item display area 701. In the caution item display area 701, cautions regarding the work and the work manager are displayed in text. For example, information such as “this customer is careful about forgetting to say hello to the customer” and “this customer is more likely to receive complaints if not working on time” is displayed.

  FIG. 19 is a diagram illustrating an example of a prevention information display screen 750 that is another example of a screen displayed in the complaint prevention processing. The prevention information display screen 750 includes a caution item display area 751 and a know-how image display area 752. In the caution item display area 751, cautions regarding the work and the work manager are displayed in text. For example, information such as “the connection of the target device should be dealt with by the following work method. If the method is mistaken, it causes a failure.” Is displayed. Also, in the know-how image display area 752, a recorded video or the like when the work is actually performed, such as an enlarged photograph of the connection part of the device, is displayed as an image or a moving image.

  The above is the second embodiment. According to the second embodiment, the worker 10 notifies the worker 10 of the information accumulated by other workers 10 regarding the work and the person in charge of work management, information on know-how, and know-how before the worker 10 starts the work. Therefore, the work efficiency of the worker 10 can be improved, and know-how information can be easily acquired, so that the generation of complaints can be avoided efficiently.

  In the above, it demonstrated using 1st embodiment and 2nd embodiment. The technical elements of the above-described embodiments may be applied independently, or may be applied by being divided into a plurality of parts such as program parts and hardware parts.

  In the above, this invention was demonstrated centering on embodiment.

DESCRIPTION OF SYMBOLS 1 ... Work management system, 10 ... Worker, 100 ... Emotion analysis terminal, 110 ... Display control part, 120 ... Position specification part, 130 ... Voice acquisition part, 140 ... Communication unit 150 Image processing unit 160 Voice processing unit 170 Expression analysis unit 180 Emotion analysis storage unit 181 Emotion reference storage unit 182 Work result storage unit, 183 ... basic evaluation information storage unit, 184 ... personal interaction storage unit, 190 ... emotion analysis unit, 200 ... work management device, 210 ... display unit, 220 ... Communication unit, 231 ... voice processing unit, 232 ... expression processing unit, 233 ... cause search unit, 234 ... work evaluation unit, 235 ... knowledge management unit, 250 ... storage unit 251 ... Work information storage unit 252 ... Work result storage unit 253 ... evaluation basic information storage unit, 254 ... interpersonal answering storage unit, 300 ... work target device

Claims (11)

A communication unit that receives a voice code obtained by coding a voice related to work from another device by a predetermined method;
Using the received voice code to evaluate the emotion of the voice, and using the evaluation to evaluate the work related to the voice code, the work evaluation unit,
A work management apparatus comprising: The work management apparatus according to claim 1,
The work evaluation unit performs the evaluation of the emotion of the voice using geometric information of a waveform constituted by a temporal change in the quality of the emotion identified using the voice code.
Work management device. The work management device according to claim 2,
In the waveform, when there is a change exceeding a predetermined degree in the quality of the emotion, a cause search unit that reproduces the sound starting from the past by a predetermined time from the time when the change occurred,
A work management apparatus comprising: The work management device according to any one of claims 1 to 3,
The work evaluation unit
The evaluation of the emotion is performed according to the degree of difficulty related to the work, and the work is evaluated by adding correction according to the difficulty level to the evaluation of the emotion.
Work management device. The work management device according to any one of claims 1 to 4,
The work evaluation unit
Performing the evaluation of the emotion according to a person in charge of managing the work, and evaluating the work by adding a correction according to the characteristics of the person in charge to the evaluation of the emotion
Work management device. The work management device according to any one of claims 1 to 5,
The communication unit receives the audio code and also receives image information related to the audio code,
A facial expression processing unit that identifies emotions represented by facial expressions included in the image information;
The work evaluation unit evaluates the work by adding a correction by the emotion specified by the facial expression processing unit to the evaluation of the emotion of the voice.
Work management device. The work management device according to any one of claims 1 to 6,
A storage unit for storing knowledge information received by the communication unit in association with a voice code related to the work;
A knowledge management department,
The communication unit accepts information specifying the work,
The knowledge management unit specifies and outputs knowledge information associated with a voice code related to the received work;
Work management device. An audio acquisition unit for acquiring audio information about the work;
A voice analysis unit that specifies voice code information indicating the quality of emotion according to the attribute of the voice information;
An emotion analysis unit associating the voice code information with the work;
A communication unit for outputting the voice code information to the other device together with the information of the associated work;
Emotion analysis terminal equipped with. The emotion analysis terminal according to claim 8,
An image acquisition unit for acquiring image information corresponding to the audio information;
A facial expression analysis unit that identifies facial expressions included in the image information,
The emotion analysis unit associates the voice code information with the facial expression information for the work;
Emotion analysis terminal. A work management program that causes a computer to execute procedures for managing work,
Causing the computer to function as control means;
For the control means,
A communication step for receiving a voice code obtained by coding a voice related to work from another device by a predetermined method;
A work evaluation step for evaluating the emotion of the voice using the received voice code, and using the evaluation to evaluate a work related to the voice code;
Work management program to execute A work management method for performing work management procedures using a computer,
The computer includes a control unit,
The controller is
A communication step for receiving a voice code obtained by coding a voice related to work from another device by a predetermined method;
A work evaluation step for evaluating the emotion of the voice using the received voice code, and using the evaluation to evaluate a work related to the voice code;
Work management method to execute.

Images