WO2016189775A1

WO2016189775A1 - Customer service evaluation device, customer service evaluation system provided with same, and customer service evaluation method

Info

Publication number: WO2016189775A1
Application number: PCT/JP2016/000858
Authority: WO
Inventors: 善史酒田; 國枝　賢徳; 萩原　明
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2015-05-25
Filing date: 2016-02-18
Publication date: 2016-12-01
Also published as: JP5939493B1; JP2016218911A

Abstract

The present invention enables proper evaluation of a service attitude of a customer service staff member, considering whether or not a conversation was initiated by the customer service staff member. The customer service evaluation device (5) is configured to comprise: an image input unit (21) to which an image picked up in a preset customer service area is inputted; a face image extraction unit (22) which extracts, from the image, a face image of a customer service staff member; a smile level determination unit (23) which determines a smile level of the customer service staff member on the basis of the face image of the customer service staff member; and a conversation-initiation determination unit (25) which determines whether a conversation was initiated by the customer service staff member, wherein when the conversation-initiation determination unit (25) determines that a conversation was not initiated by the customer service staff member, the service attitude of the customer service staff member is evaluated on the basis of the smile level of the customer service staff member determined by the smile level determination unit (23).

Description

Service evaluation device, service evaluation system and service evaluation method provided with the same

The present disclosure relates to a customer service evaluation device, a customer service evaluation system including the service evaluation system, and a customer service evaluation method for evaluating the customer service attitude based on the degree of smile.

In the service industry such as retail, restaurant, hotel, etc., it is known that a good customer service attitude of customers such as store staff and employees leads to customer satisfaction, and as a result, the customer acquisition rate and sales are improved. As a method for evaluating customer service attitudes, customer opinion surveys are generally conducted. However, since such customer service evaluation methods are performed with a large number of personnel, they are inefficient. There is a problem that it is both objective and poor in objectivity.

Therefore, for example, an apparatus is known that evaluates the customer service attitude by acquiring a customer service facial image and determining the customer smile level (smile level) based on the facial image ( Patent Document 1).

JP 2010-186373 A

The customer service evaluation device according to the present disclosure is a customer service evaluation device for evaluating the customer service attitude based on the degree of smile, and is a customer service evaluation device for evaluating the customer service attitude based on the smile level. An image input unit for inputting an image captured in a preset customer service area, a face image extraction unit for extracting a customer's face image from the image, and the customer's face image based on the customer's facial image. A smile degree determination unit that determines the degree of smile and an utterance determination unit that determines whether or not the customer speaks. When the utterance determination unit determines that the customer is not speaking, the smile degree determination unit The customer service attitude of the customer service is evaluated based on the degree of smile of the customer service determined in (1).

According to the present disclosure, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service.

FIG. 1 is an overall configuration diagram of a customer service evaluation system according to the first embodiment. FIG. 2 is an explanatory diagram illustrating an application example of the customer service evaluation system according to the first embodiment. FIG. 3 is a block diagram showing a schematic configuration of the customer service evaluation apparatus in FIG. FIG. 4A is a diagram for explaining a method for determining the smile level of a customer service. FIG. 4B is a diagram for explaining a method of determining a customer smile level. FIG. 4C is a diagram for explaining a method of determining the smile level of the customer service. FIG. 5 is a diagram illustrating an example of the customer service information table. FIG. 6 is a diagram illustrating an example of a smile degree evaluation value table. FIG. 7A is a diagram showing a modification of the customer service information table. FIG. 7B is a diagram illustrating an example of a face orientation evaluation value table. FIG. 8 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the first embodiment. FIG. 9 is a block diagram illustrating a schematic configuration of the customer service evaluation device according to the second embodiment. FIG. 10 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the second embodiment. FIG. 11 is an overall configuration diagram of the customer service evaluation system according to the third embodiment. FIG. 12 is an explanatory diagram illustrating an application example of the customer service evaluation system according to the third embodiment. FIG. 13 is a block diagram illustrating a schematic configuration of the customer service evaluation apparatus according to the third embodiment. FIG. 14 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the third embodiment. FIG. 15 is a block diagram illustrating a schematic configuration of the customer service evaluation device according to the fourth embodiment. FIG. 16 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the fourth embodiment. FIG. 17 is an explanatory diagram illustrating another application example of the customer service evaluation system.

Prior to the description of the embodiment, the problems in the prior art will be briefly described. The prior art described in Patent Document 1 described above has a problem in that the customer service attitude of the customer service cannot be correctly evaluated because the presence or absence of the customer service is not considered. During the utterance, the customer's mouth shape changes depending on the spoken language, so it is difficult to determine the smile level of the customer during the utterance based on the customer's face image, and the smile level of the customer during the utterance Will be appreciated. Therefore, even when a similar smile is made, the degree of smile of the customer who is speaking is lower than that of the customer who is not speaking. In addition, when obtaining the average smile level per predetermined customer service time, a customer who has a lot of utterances has a lower evaluation of a smile level than a customer who has a low number of utterances. Thus, unless the presence or absence of the customer's utterance is considered, the customer service attitude of the customer cannot be correctly evaluated.

Also, since the customer service does not smile except during customer service, the evaluation of the degree of smile during non-service is naturally low. Therefore, when calculating the average smile level per predetermined working hours, a customer who has a short customer service time has a lower smile rating than a customer who has a long customer service time. Therefore, in order to evaluate the customer service attitudes of a plurality of customers fairly, it is necessary to correctly evaluate the customer service attitudes according to the actual situation.

The present disclosure has been devised in view of the problems of the prior art as described above, and a customer service evaluation device capable of correctly evaluating a customer service attitude in consideration of presence or absence of a customer service utterance and The main purpose is to provide a customer service evaluation system and a customer service evaluation method provided with this.

A first disclosure made in order to solve the above-described problem is a customer service evaluation device for evaluating a customer service attitude, and an image input in which an image captured in a predetermined customer service area is input A face image extraction unit that extracts a customer's face image from the image, a smile degree determination unit that determines the degree of smile of the customer based on the customer's face image, and whether or not the customer's utterance is present A speech determination unit that evaluates the customer service attitude based on the degree of smile of the customer determined by the smile level determination unit when the customer determination unit determines that the customer is not speaking It is characterized by doing.

According to the customer service evaluation device according to the first disclosure, whether or not a customer speaks is determined, and when it is determined that the customer does not speak, the customer's smile is determined based on the degree of smile of the customer. Since the configuration is such that the customer service attitude is evaluated, the customer service attitude while the customer is not speaking can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service.

A second disclosure made in order to solve the above-mentioned problem is a customer service evaluation device for evaluating a customer service attitude, and an image input in which an image captured in a predetermined customer service area is input A face image extraction unit that extracts a customer's face image from the image, an utterance determination unit that determines the presence or absence of the customer's utterance, and a face image output unit that outputs a face image of the customer who is not speaking It is provided with.

According to the customer service evaluation device according to the second disclosure, since the configuration is such that the face image of the customer who is not speaking is output, the user can receive the customer who is not speaking the customer based on the output face image. Attitude can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service.

In the third disclosure, in the first or second disclosure, the utterance determination unit detects the movement of the customer's mouth from the facial image of the customer, and utters the customer's utterance based on the movement of the mouth. It is characterized by determining the presence or absence of.

According to the service evaluation device according to the third disclosure, the presence / absence of the customer's utterance is determined in order to determine the presence / absence of the customer's utterance based on the movement of the customer's mouth detected from the customer's face image. It is possible to easily and reliably determine.

The fourth disclosure further includes a voice input unit for inputting the voice of the customer in the first or second disclosure, and the speech determination unit is configured to determine the voice of the customer based on the voice of the customer. The presence or absence is determined.

According to the customer service evaluation apparatus according to the fourth disclosure, since the presence or absence of the customer's utterance is determined based on the customer's voice, it is possible to easily and reliably determine the presence or absence of the customer's utterance. .

In the fifth disclosure, in the first disclosure, the detection information input unit to which the detection information of the customer service is input, and the customer service status indicating whether or not the customer is in service based on the detection information are determined. And a customer service state determination unit, wherein when the customer service state determination unit determines that the customer service is in service, the customer service attitude of the customer service is evaluated.

According to the customer service evaluation device according to the fifth disclosure, it is determined whether or not the customer service is in service based on detection information of the customer service partner, and it is determined that the customer service is in service. In this case, since the customer service attitude of the customer service is evaluated, it is possible to evaluate the customer service attitude during the customer service. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the customer service status.

In the sixth disclosure, in the second disclosure, the detection information input unit to which the detection information of the customer service is input, and the customer service status indicating whether or not the customer is in service based on the detection information are determined. A face image output unit that outputs a face image of the customer who is not speaking and in service.

According to the customer service evaluation device according to the sixth disclosure, since the configuration is such that the face image during non-speaking and customer service of the customer is output, the user is serving the customer based on the output face image. The customer service attitude can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the customer service status.

In the seventh disclosure, the customer service evaluation device according to any of the first to fourth disclosures, and an image imaging device that captures an image in the customer service area and inputs the captured image to the customer service evaluation device. It is a customer service evaluation system characterized by having provided.

In the eighth disclosure, the customer service evaluation device according to the fifth or sixth disclosure, an image imaging device that captures an image in the customer service area, and inputs the captured image to the customer service evaluation device; A customer service evaluation system comprising: a customer service partner detection device that detects and inputs customer service partner detection information to a customer service evaluation device.

Further, the ninth disclosure is characterized in that, in the eighth disclosure, the customer partner detection device is an image pickup device, a sound acquisition device, or a proximity sensor.

A tenth disclosure is a customer service evaluation method for evaluating customer service attitudes of a customer, an image acquisition step of acquiring an image captured in a customer service area set in advance, and the customer service from the image. A face image extracting step for extracting a face image, a smile degree determining step for determining the degree of smile of the customer based on the customer's face image, an utterance determining step for determining whether or not the customer utters, and an utterance determination A customer service attitude evaluation step for evaluating the customer service attitude based on the smile level of the customer service determined in the smile level determination step when it is determined in the step that the customer service is not speaking It is characterized by.

An eleventh disclosure is a customer service evaluation method for evaluating a customer service attitude of a customer with a smile, an image acquisition step of acquiring an image captured in a customer service area set in advance, and a customer service from the image A face image extraction step for extracting a person's face image, an utterance determination step for determining whether or not the customer speaks, and a face image output step for outputting a face image of the customer who is not speaking To do.

Hereinafter, embodiments will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram of the customer service evaluation system according to the first embodiment, FIG. 2 is an explanatory diagram showing an application example of the customer service evaluation system according to the first embodiment, and FIG. It is a block diagram which shows schematic structure of this customer service evaluation apparatus.

As shown in FIG. 1, the customer service evaluation system 1 is constructed in a store 2 or the like, and evaluates the customer service attitude of a customer service (here, a store clerk) to a customer service customer (here, a customer who visited the store) based on the degree of smile. Make it possible to do. In addition, the customer service evaluation system 1 enables a user (for example, a store manager of the store 2 or a headquarters manager) to evaluate a customer service attitude based on a customer's face image. In the store 2, as a component of the customer service evaluation system 1, there is a camera (image capturing device) 3 that captures a customer service area 13 (see FIG. 2) set inside a cashier counter 11 (see FIG. 2) in the store. The customer service partner detection device 4 for detecting the customer service partner in the customer side area 14 (see FIG. 2) set on the outer front of the cashier counter 11 and the customer service attitude of the store clerk are installed in the backyard of the store. A customer service evaluation device 5 for evaluating based on the degree of smile is provided.

The camera 3 and the customer partner detection device 4 can communicate directly or indirectly with the customer service evaluation device 5 via a communication line 6 such as a LAN (Local Area Network). In the customer service evaluation system 1, the camera 3, the service partner detection device 4, and the customer service evaluation device 5 use a relay device 7 provided on the communication line 6 to connect a wide area network 8 such as the Internet based on a public line or a dedicated line. Via the headquarters management apparatus 9.

In this embodiment, the store 2 to which the customer service evaluation system 1 is applied will be described as an example of a retail store such as a convenience store, but is not limited thereto. As shown in FIG. 2 (plan view inside the store), in the store 2, store clerks S1 and S2 (hereinafter referred to as “S” when referring generically) are located in the customer service area 13 set inside the cashier counter 11. The customers C1 and C2 (hereinafter referred to as “C” in the generic name) lined up in the customer-side area 14 set in front of the cashier counter 11 settles the purchased product. The customer service area 13 and the customer side area 14 can be arbitrarily set, and both the areas may be arbitrary positions in the store. The two areas may overlap each other. Note that the number of salesclerks S is not limited to two. For example, there may be one salesclerk S in a time zone where customers C are few. Further, in a time zone where there are many customers C, there may be three or more clerk S.

The camera 3 is a CCD camera, for example, and is installed, for example, around the cash register 12 on the cash register counter 11 and continuously photographs the inside of the customer service area 13. That is, the customer service area 13 is a virtual range in which the camera 3 can shoot. The video imaged by the camera 3 is transmitted as a video signal to the customer service evaluation device 5 via the communication line 6. As long as the camera 3 can capture at least the face image of the clerk S, the function, arrangement, quantity, and the like are not particularly limited, and various changes can be made. For example, the camera 3 may be a known omnidirectional network camera installed on the ceiling above the cashier counter 11.

The customer service partner detection device 4 is, for example, an image pickup device (camera), a sound acquisition device (microphone), a proximity sensor, or the like. When a camera is used as the customer service partner detection device 4, for example, a known omnidirectional network camera installed on the ceiling in the store or a CCD camera installed around the cash register 12 on the cash register counter 11 can be used. The camera 3 continuously shoots the situation in the customer side area 14 in order to recognize the presence or absence of the customer in the customer side area 14. When a microphone is used as the customer service partner detection device 4, for example, a known omnidirectional network microphone installed on the ceiling in the store or a known directional microphone installed around the cash register 12 on the cash register counter 11 is used. Can be used. The microphone 4 continuously acquires (collects sound) in the customer side area 14 in order to recognize the presence or absence of the customer in the customer side area 14. Moreover, when using a proximity sensor as the customer service partner detection apparatus 4, for example, a known optical proximity sensor can be used. The sensor 4 detects the customer C in the customer side area 14.

Detection information (video, audio, detection result) of the customer C in the customer area 14 acquired or detected by the camera, microphone, and proximity sensor that is the customer partner detection device 4 is the customer service evaluation device via the communication line 6. 5 is transmitted. In addition, as long as at least the customer C can be detected, the function, arrangement, quantity, and the like of the camera, microphone, and proximity sensor are not particularly limited, and various changes can be made.

The customer service evaluation device 5 is a PC (Personal Computer) installed in the backyard of the store 2 and used by a user (store manager, etc.). As will be described later, the customer service evaluation device 5 acquires the video from the camera 3 and the detection information from the customer partner detection device 4, and executes the evaluation process of the customer service attitude of the clerk S based on the acquired video and the detection information. To do.

Although details are not shown, the customer service evaluation device 5 is a central processing unit (CPU) that centrally executes various information processing and control of peripheral devices based on a predetermined control program, and a RAM (functioning as a work area of the CPU). Random Access Memory (ROM), ROM (Read Only Memory) for storing control programs and data executed by the CPU, network interface for executing communication processing via the network, monitor (image output device), speaker, input device, and HDD ( (Hard Disk Drive) etc., and the CPU is assigned to at least some of the various functions of the customer service evaluation device 5 (e.g., evaluation process of customer service attitude of the store clerk S), which will be described in detail later. Control program (e.g., image recognition program) can be realized by executing a. The customer service evaluation device 5 is not limited to a PC, and other information processing devices (such as a server) capable of performing the same function can also be used. Moreover, you may substitute at least one part of the function of the customer service evaluation apparatus 5 by the process by other well-known hardware.

The headquarters management device 9 is a PC having the same configuration as the customer service evaluation device 5 and can execute the same processing as the customer service evaluation device 5. The headquarters management device 9 is used by a headquarters administrator who manages a plurality of stores 2 similar to the stores 2. A configuration in which the head office management device 9 shares part of the evaluation process of the customer service attitude of the clerk S by the customer service evaluation device 5 is also possible.

As shown in FIG. 3, the customer service evaluation device 5 includes an image input unit 21 to which video is input from an image pickup device (camera) 3, a face image extraction unit 22 that extracts a face image of the clerk S from the image, A smile degree determination unit 23 that determines the degree of smile of the clerk S based on the face image of the clerk S (hereinafter referred to as “smile degree”), and customer service information that stores the smile degree determined by the smile degree determination unit 23 A storage unit 24; an utterance determination unit 25 that determines the presence or absence of an utterance of the clerk S based on the face image of the clerk S; a detection information input unit 26 that receives detection information of the customer C from the customer service partner detection device 4; A customer service state determination unit 27 that determines whether or not the store clerk S is in service based on the detection information, and an evaluation of the customer service attitude of the store clerk S based on the smile level determined by the smile level determination unit 23 Evaluation value calculation unit 28 for calculating a value and evaluation value calculation unit 2 In and a evaluation value storage unit 29 for storing the evaluation value of customer attitudes calculated clerk S. The customer service information storage unit 24 also stores the determination results of the utterance determination unit 25 and the customer service state determination unit 27. Each unit is controlled by a control unit (not shown).

The face image of the clerk S located in the customer service area 13 photographed by the camera 3 is input to the image input unit 21 as a video signal. The video data of the input video signal includes the shooting time of the face image of the clerk S.

The face image extraction unit 22 performs a face image extraction process for extracting the face image of the clerk S from the image (a plurality of image frames) input to the image input unit 21 using a known face image recognition technique. At this time, the process of determining the clerk ID of the clerk S from which the face image is extracted is also performed. Specifically, referring to the face image DB (not shown) of the clerk S, the clerk ID of the clerk S from which the face image is extracted is determined.

The smile level determination unit 23 determines the smile level of the clerk S based on the shape of the mouth and eyes using a known face image recognition technique. The degree of smile is determined as a percentage (%). First, the facial expression of the clerk S is detected from the facial image of the clerk S using a known face image recognition technique. As shown in FIGS. 4A to 4C, when it is determined that the facial expression of the clerk S is FIG. 4A, the smile degree is determined to be 80%, and the facial expression of the clerk S is determined to be FIG. 4B. The smile degree is determined to be 60%, and when it is determined that the facial expression of the clerk S is FIG. 4C, the smile degree is determined to be 20%. Note that the smile degree determination method shown in FIGS. 4A to 4C is merely an example, and actually, it may be determined in more detail.

The customer service information storage unit 24 stores the smile level of the clerk S determined by the smile level determination unit 23. The smile level of the clerk S is recorded in the customer service information table (see FIG. 5) stored in the customer service information storage unit 24 together with the shooting time and the clerk ID of the clerk S. In the customer service information table of FIG. 5, the smile level of the clerk S1 is 80 points at 8:12:24, 70 points at 8:12:25, and 60 at 8:12:26. It is shown that it was 70 points at 8:12:27 and 20 points at 8:12:28. In addition, the smile degree of the clerk S2 was 30 points at 8:12:32, 60 points at 8:12:33, and 80 points at 8:12:34. Has been. In addition, the determination results of the utterance determination unit 25 and the customer service state determination unit 27 are also input to the customer service information storage unit 24 and recorded in the customer service information table in association with the smile level of the clerk S.

The utterance determination unit 25 determines the presence or absence of the utterance of the clerk S from which the face image is extracted using a known face image recognition technique. Specifically, the utterance determination unit 25 detects the movement of the clerk's S mouth from the face image of the clerk S, and determines the presence or absence of the utterance of the clerk S based on the movement of the mouth. If a predetermined mouth movement is detected, the clerk S is determined to be speaking, and if no predetermined mouth movement is detected within a predetermined number of image frames, the clerk S is determined not to speak. The The determination result in the utterance determination unit 25 is input to the customer service information storage unit 24 and is recorded as “speaking” or “not speaking” in the customer service information table (see FIG. 5).

The detection information input unit 26 receives the detection information of the customer C in the customer area 14 acquired by the customer partner detection device 4, that is, information indicating the presence or absence of the customer C in the customer area 14. The This detection information includes the time when the customer service partner detection device 4 acquires the information.

The customer service state determination unit 27 determines the customer service state indicating whether or not the clerk S is in service based on the detection information of the customer C input to the detection information input unit 26. When the customer C exists in the customer side area 14, it is determined that the customer is in service, and when the customer C does not exist in the customer side area 14, it is determined that the customer C is not in service. The determination result in the customer service state determination unit 27 is input to the customer service information storage unit 24 and is recorded in the customer service information table (see FIG. 5) as “serving customers” or “not receiving customers”. When the customer service state determination unit 27 determines that the customer service is in progress, there may be a case where the clerk S does not exist in the customer service area 13, but in such a case, the customer clerk S is not evaluated for the customer service attitude. .

The evaluation value calculation unit 28 calculates an evaluation value of the customer service attitude of each clerk S during customer service and during non-utterance based on the customer service information table (see FIG. 5) stored in the customer service information storage unit 24. Specifically, the average value of the smile levels while each clerk S is in customer service and not speaking is calculated. The calculated average value of the smile level of each clerk S during customer service and during non-speaking is recorded in a smile level evaluation value table (see FIG. 6) stored in the evaluation value storage unit 29.

In the customer service information table shown in FIG. 5, the store clerk S1 is in service / not speaking at 8:12:24 and is in service from 8:12:25 to 8:12:27.・ Speaking, it is shown that at 8:12:28, the customer was not serving or talking. In addition, the clerk S2 was not in service / speaking at 8:12:32 and was in service / not speaking from 8:12:33 to 8:12:34. It is shown. In addition, as a case of non-serving customers / speaking, a conversation with another store clerk or a greeting to a customer outside the customer side area 14 can be considered.

Then, the evaluation value calculation unit 28 calculates the average value of the smile level of each store clerk S during customer service and not speaking based on the customer service information table shown in FIG. The store clerk S1 is in customer service and is not speaking at 8:12:24, and the smile level at that time is 80, so the average value of the smile level of the store clerk S1 during customer service and not speaking is 80. . In addition, the store clerk S2 is in service and is not speaking from 8:12:33 to 8:12:34, and the smile level at that time is 60 and 80, so the store clerk S2 is in service and is not speaking. The average smile level is 70. In this way, the evaluation value calculation unit 28 can calculate the average value of the smile level while each store clerk S is serving customers and not speaking. In the smile degree evaluation value table shown in FIG. 6, the average value of the smile degree during the customer service and non-speaking of the clerk S1 is 80, and the average value of the smile degree during the customer service and non-speaking of the clerk S2 is 70. It is shown that there is.

Further, the utterance determination unit 25 may determine the face direction or the line-of-sight direction of the clerk S using a known face image recognition technique. During the customer service, it is preferable that the face of the clerk S is facing the customer C. Therefore, the evaluation value of the clerk S's face or line of sight is added to the evaluation of the clerk S's attitude to the customer. Good. In this case, the determination result in the speech determination unit 25 is input to the customer service information storage unit 24 and added to the customer service information table shown in FIG. FIG. 7A is obtained by adding the determination result of the face direction of the clerk S to the customer service information table of FIG. The face orientation of the clerk S is recorded as “customer direction” or “other than customer direction”. Then, the evaluation value calculation unit 28 calculates the evaluation value of the orientation of the face of the clerk S during customer service based on the customer service information table shown in FIG. 7A. The calculated evaluation value of the face direction during customer service is recorded in a face direction evaluation value table (see FIG. 7B) stored in the evaluation value storage unit 29.

In the customer service information table shown in FIG. 7A, the direction of the face of the clerk S1 is the customer direction at 8:12:24, and the time at 8:12:25 is other than the customer direction. It is shown that the time from minutes 26 seconds to 8:12:27 is the customer direction, and the time of 8:12:28 is other than the customer direction. The direction of the face of store clerk S2 is 8:12:32 except for the customer direction, 8:12:33 is the customer direction, and 8:12:34 is not the customer direction. It was shown that it was.

Then, the evaluation value calculation unit 28 calculates an evaluation value of the orientation of the face of each clerk S during customer service based on the customer service information table shown in FIG. 7A. The evaluation value of the face direction is represented by a ratio (percent (%)) that the face direction of the clerk S who is serving the customer is facing the customer direction. The store clerk S1 is in service from 8:12:24 to 8:12:27, and the direction of the face of the store clerk S1 at that time is 8:12:24, 8:12:26, The time at 8:12:27 was for the customer, and the time at 8:12:25 was not for the customer. Therefore, the evaluation value of the direction of the face of the store clerk S1 during the customer service is 75%. The store clerk S2 is in service from 8:12:33 to 8:12:34. The face of the store clerk S2 at that time is 8:12:33. , 8:12:34 was not in the direction of the customer. Therefore, the evaluation value of the direction of the face of the store clerk S2 during customer service is 50%. In this way, the evaluation value calculation unit 28 can calculate the evaluation value of the face orientation of each clerk S during customer service. In the face direction evaluation value table shown in FIG. 7B, the evaluation value of the face direction during the customer service of the clerk S1 is 75%, and the evaluation value of the face direction during the customer service of the clerk S2 is 50%. It is shown.

Furthermore, the customer service evaluation device 5 includes an evaluation value output unit 30 that outputs the evaluation value of the customer service attitude of the clerk S stored in the evaluation value storage unit 29 as desired by the user. As described above, the evaluation value of the customer service attitude of the clerk S is the average value of the smile degree during the customer service and non-speaking of the clerk S recorded in the smile evaluation value table (see FIG. 6), or It is an evaluation value of the direction of the face of the clerk S who is in the customer service recorded in the direction evaluation value table (see FIG. 7A).

Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the first embodiment shown in FIG. 3 will be described with reference to FIG.

First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image pickup device (camera) 3 arranged in the store (ST101). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST102). Next, the smile level determination unit 23 determines the smile level of the clerk S based on the face image of the clerk S extracted by the face image extraction unit 22 (ST103).

In subsequent ST 104, the utterance determination unit 25 determines the presence or absence of the clerk S's utterance based on the face image of the clerk S extracted by the face image extraction unit 22. The presence / absence of the clerk's utterance is recorded in the customer service information table (see FIG. 5). If it is determined that the clerk S is speaking (speaking), the process returns to step S101. If it is determined that the clerk S is not speaking (not speaking), the process proceeds to step S105.

In step S105, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer partner detection device 4 arranged in the store. As described above, the detection information of the customer C is information indicating whether or not there is a customer service partner in the customer area 14.

In subsequent ST 106, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service, and when the customer C is not present in the customer side area 14, it is determined that the customer is not in service. The customer service status of the clerk S is recorded in the customer service information table (see FIG. 5). If it is determined that the customer is in service, the process proceeds to step S107. If it is determined that the customer is not in service, the process returns to step S101.

In ST107, the evaluation value calculation unit 28 calculates the evaluation value of the customer service attitude of the clerk S based on the smile level determined by the smile level determination unit 23. The evaluation value of the customer service attitude of the clerk S calculated by the evaluation value calculation unit 28 is stored in the evaluation value storage unit 29 and is output from the evaluation value output unit 30 according to the user's desire.

Thus, according to the customer service evaluation device 5 according to the first embodiment, when the store clerk S is not speaking and serving customers, the evaluation value of the customer service attitude of the store clerk S can be calculated. Therefore, it is possible to correctly evaluate the customer service attitude of the store clerk S in consideration of the presence / absence of the customer's utterance and the customer service status.

(Second Embodiment)
FIG. 9 is a block diagram showing a schematic configuration of the customer service evaluation device according to the second embodiment, and FIG. 10 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the second embodiment. The customer service evaluation device according to the second embodiment is used in the customer service evaluation system 1 shown in FIG.

The customer service evaluation device 5 according to the second embodiment illustrated in FIG. 9 is similar to the customer service evaluation device 5 according to the first embodiment illustrated in FIG. 3, and includes an image input unit 21, a face image extraction unit 22, and an utterance determination. Unit 25, detection information input unit 26, and customer service state determination unit 27. The customer service evaluation device 5 according to the second embodiment further includes a face image information storage unit 32 and a face image output unit 33 that are not provided in the customer service evaluation device 5 according to the first embodiment. The same parts as those in the customer service evaluation device 5 according to the first embodiment shown in FIG.

The face image information storage unit 32 stores the face image of the clerk S acquired by the face image extraction unit 22. The face image of the clerk S is stored in the face image information storage unit 32 together with the shooting time and the clerk ID of the clerk S. The face image information storage unit 32 also receives the determination results of the utterance determination unit 25 and the customer service state determination unit 27 and records them in association with the face image of the clerk S.

The face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in accordance with an instruction from the user. At this time, only a face image in which the customer service state is “busy” and the utterance state is “not speaking” may be read and output. The face image of the clerk S may be output to a known image output device such as a liquid crystal monitor connected to the customer service evaluation device 5, or as a video signal (video data) via the wide area network 8, the head office management device 9. May be sent to.

Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the second embodiment shown in FIG. 9 will be described with reference to FIG.

First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image pickup device (camera) 3 arranged in the store (ST201). Subsequently, the face image extraction unit 22 extracts the face image of the customer service (store clerk S) from the image input to the image input unit 21 (ST202). The face image of the clerk S extracted by the face image extraction unit 22 is stored in the face image information storage unit 32 (ST203).

Next, in ST204, the utterance determination unit 25 determines the presence or absence of the clerk S's utterance based on the face image of the clerk S extracted by the face image extraction unit 22. Whether or not the clerk S speaks is recorded in the face image information storage unit 32. If it is determined that the clerk S is speaking, the process returns to step S201. If it is determined that the clerk S is not speaking, the process proceeds to step S205.

In step 205, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer service partner detection device 4 (see FIG. 1) arranged in the store. In subsequent ST 206, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service, and when the customer C is not present in the customer side area 14, it is determined that the customer is not in service. The customer service state of the clerk S is recorded in the face image information storage unit 32. If it is determined that the customer is in service, the process proceeds to step S207. If it is determined that the customer is not in service, the process returns to step S201.

In ST207, the face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in response to an instruction from the user. At this time, only the face image of the store clerk S who is in customer service and not speaking may be read and output.

In this way, according to the customer service evaluation device 5 according to the second embodiment, it is possible to output a face image of the clerk S who is not speaking and in service. Then, the user can evaluate the customer service attitude while the customer is not speaking and during customer service based on the output face image. Therefore, it is possible to correctly evaluate the customer service attitude of the store clerk S in consideration of the presence / absence of the customer's utterance and the customer service status.

(Third embodiment)
FIG. 11 is an overall configuration diagram of the customer service evaluation system according to the third embodiment, FIG. 12 is an explanatory diagram illustrating an application example of the customer service evaluation system according to the third embodiment, and FIG. 13 is a third embodiment. FIG. 14 is a block diagram illustrating a schematic configuration of the customer service evaluation device according to the embodiment, and FIG. 14 is a flowchart illustrating a flow of customer service evaluation processing by the customer service evaluation device according to the third embodiment.

11 is different from the customer service evaluation system 1 according to the first embodiment shown in FIG. 1 in that the customer service evaluation system according to the third embodiment shown in FIG. Since other points are the same as the customer service evaluation system 1 according to the first embodiment shown in FIG. 1, the same reference numerals are given and the description thereof is omitted.

The voice acquisition device 10 is a general directional microphone, and is arranged around the cash register 12 of the cashier counter 11 as shown in FIG. The microphone 10 continuously acquires (collects sound) in the customer service area 13. In addition, as long as the microphone 10 can acquire at least the voice of the clerk S, the function, arrangement, quantity, and the like are not particularly limited, and various changes can be made. For example, a configuration in which microphones are arranged at a plurality of locations according to the arrangement of the store clerk S in the store, or a configuration in which microphones are attached to the clothes of the store clerk S is also possible.

The customer service evaluation device according to the third embodiment shown in FIG. 13 includes a voice input unit 31 and a point that the speech determination unit 25 is connected to the voice input unit 31 instead of the face image extraction unit 22. However, it is different from the customer service evaluation device 5 according to the first embodiment shown in FIG. Since the other points are the same as those in the customer service evaluation device 5 according to the first embodiment shown in FIG. 3, the same reference numerals are given and the description thereof is omitted.

The voice of the clerk S in the customer service area 13 acquired by the microphone 10 is input to the voice input unit 31. This audio information includes the time when the microphone 10 acquired the audio. The utterance determination unit 25 is connected to the voice input unit 31 and determines the presence or absence of the utterance of the clerk S based on the voice input to the voice input unit 31.

Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the third embodiment shown in FIG. 13 will be described with reference to FIG.

First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image capturing device (camera) 3 arranged in the store (ST301). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST302). Next, the smile level determination unit 23 determines the smile level of the clerk S based on the face image of the clerk S extracted by the face image extraction unit 22 (ST303).

In subsequent ST304, the voice input unit 31 acquires the voice of the clerk S in the customer service area 13 from the voice acquisition device (microphone) 10 disposed in the store. In continuing ST305, the speech determination part 25 determines the presence or absence of the speech of the salesclerk S based on the voice of the salesclerk S which the voice input part 31 acquired. When the clerk's S voice is acquired, it is determined that the utterance is being performed, and when the clerk's S voice is not acquired, it is determined that the utterance is not being spoken. The presence / absence of the clerk's utterance is recorded in the customer service information table (see FIG. 5). If it is determined that the clerk S is speaking, the process returns to step S301. If it is determined that the clerk S is not speaking, the process proceeds to step S306.

In Step S306, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer service partner detection device 4 arranged in the store. In subsequent ST307, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service. The customer service status of the clerk S is recorded in the customer service information table (see FIG. 5). If it is determined that the customer is in service, the process proceeds to step S308. If it is determined that the customer is not in service, the process returns to step S301.

In ST308, the evaluation value calculation unit 28 calculates the evaluation value of the customer service attitude of the clerk S based on the smile level determined by the smile level determination unit 23. The evaluation value of the customer service attitude of the clerk S calculated by the evaluation value calculation unit 28 is stored in the evaluation value storage unit 29 and is output from the evaluation value output unit 30 according to the user's desire.

In this way, according to the customer service evaluation device 5 according to the third embodiment, since the presence or absence of the clerk S can be determined based on the voice of the clerk S, the presence or absence of the clerk S can be more easily and easily determined. It is possible to make a reliable determination. Since the evaluation value of the customer service attitude of the store clerk S can be calculated when the store clerk S is not speaking and customer service, the customer service attitude of the store clerk S is considered in consideration of the presence or absence of the customer utterance and the customer service status. Can be correctly evaluated.

(Fourth embodiment)
FIG. 15 is a block diagram showing a schematic configuration of the customer service evaluation device according to the fourth embodiment, and FIG. 16 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the fourth embodiment. The customer service evaluation device according to the fourth embodiment is used in the customer service evaluation system 1 shown in FIG.

The customer service evaluation device 5 according to the fourth embodiment has a point that the voice input unit 31 is provided and that the speech determination unit 25 is connected to the voice input unit 31 instead of the face image extraction unit 22. It is different from the customer service evaluation device 5 according to the second embodiment shown in FIG. Since the other points are the same as those in the customer service evaluation device 5 according to the second embodiment shown in FIG. 9, the same reference numerals are given and description thereof is omitted.

The voice input unit 31 receives the voice of the clerk S in the customer service area 13 acquired by the microphone 10. This audio information includes the time when the microphone 10 acquired the audio. The utterance determination unit 25 is connected to the voice input unit 31 and determines the presence or absence of the utterance of the clerk S based on the voice input to the voice input unit 31.

Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the fourth embodiment shown in FIG. 15 will be described with reference to FIG.

First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image capturing device (camera) 3 arranged in the store (ST401). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST402). The face image of the clerk S extracted by the face image extraction unit 22 is stored in the face image information storage unit 32 (ST403).

In subsequent ST404, the voice input unit 31 acquires the voice of the clerk S in the customer service area 13 from the voice acquisition device (microphone) 10 disposed in the store. In continuing ST405, the speech determination part 25 determines the presence or absence of the speech of the salesclerk S based on the voice of the saleswoman S which the voice input part 31 acquired. Whether or not the clerk S speaks is recorded in the face image information storage unit 32. If it is determined that the clerk S is speaking, the process returns to step S401. If it is determined that the clerk S is not speaking, the process proceeds to step S406.

In step S406, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer partner detection device 4 arranged in the store. In subsequent ST407, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. The customer service state of the clerk S is recorded in the face image information storage unit 32. If it is determined that the customer is in service, the process proceeds to step S408. If it is determined that the customer is not in service, the process returns to step S401.

In ST408, the face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in response to an instruction from the user. At this time, only the face image of the store clerk S who is in customer service and not speaking may be read and output.

In this way, according to the customer service evaluation device 5 according to the fourth embodiment, the presence or absence of the clerk S can be determined based on the voice of the clerk S. It is possible to make a reliable determination. Since the face image of the clerk S who is not speaking and during the customer service can be output, the customer service attitude of the clerk S can be correctly evaluated in consideration of the presence or absence of the clerk S and the customer service state. .

Further, the customer service area is not limited to the vicinity of the cashier counter 11 and can be arbitrarily set. For example, as shown in FIG. 17, the entire store can be a customer service area. In this case, the camera 3 and the customer partner detection device 4 are preferably installed on a ceiling at an appropriate place in the store. When the microphone 10 is used, the microphone 10 is preferably attached to the clothes of the store clerks S1 to S3.

In addition, the customer service evaluation system, customer service evaluation device, and customer service evaluation method of the present disclosure can be applied to commercial facilities such as restaurants and department stores, and customer service counters such as banks, travel agencies, and real estate companies. However, the customer service area can be set arbitrarily. For example, in the case of a restaurant, it is possible to set the area around the sales counter or cash register counter as a customer service area. In the case of a department store, the entire sales area can be used as a customer service area. Can set a counter facing the customer in the customer service area.

As mentioned above, although this indication was explained based on specific embodiments, these embodiments are illustrations to the last, and this indication is not limited by these embodiments. It should be noted that the constituent elements of the customer service evaluation apparatus according to the present disclosure, the customer service evaluation system provided with the same, and the customer service evaluation method shown in the above embodiments are not necessarily all essential, and at least as long as they do not depart from the scope of the present disclosure. It is possible to select appropriately.

For example, in the above embodiment, the presence / absence of the customer's utterance and the customer service status are determined, and the customer service attitude of the customer service is evaluated when the customer is not speaking and customer service. The customer service attitude of the customer may be evaluated only when the customer is not speaking.

The customer service evaluation device, the customer service evaluation system and the customer service evaluation method provided with the same according to the present disclosure make it possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service and the customer service state. It is useful as a customer service evaluation apparatus according to the above, a customer service evaluation system equipped with the same, and a customer service evaluation method.

1 Customer Service Evaluation System 2 Store 3 Camera (Image Capture Device)
4 Service partner detection device 5 Service evaluation device 21 Image input unit 22 Face image extraction unit 23 Smile degree determination unit 24 Service information storage unit 25 Speech determination unit 26 Detection information input unit 27 Service state determination unit 28 Evaluation value calculation unit 29 Evaluation value Storage unit 30 Evaluation value output unit

Claims

A customer service evaluation device for evaluating customer service attitudes,
An image input unit for inputting an image captured in a predetermined customer service area;
A face image extracting unit for extracting the face image of the customer from the image;
A smile degree determination unit that determines the degree of smile of the customer based on the customer's face image;
An utterance determination unit for determining the presence or absence of the utterance of the customer,
When the utterance determination unit determines that the customer is not speaking, evaluating the customer service attitude of the customer based on the degree of smile of the customer determined by the smile degree determination unit Characteristic customer service evaluation device.
A customer service evaluation device for evaluating customer service attitudes,
An image input unit for inputting an image captured in a predetermined customer service area;
A face image extracting unit for extracting the face image of the customer from the image;
An utterance determination unit for determining the presence or absence of the utterance of the customer;
A customer service evaluation apparatus comprising: a face image output unit that outputs a face image of the customer who is not speaking.
The said utterance determination part detects the movement of the said customer's mouth from the said customer's face image, and determines the presence or absence of the said customer's utterance based on the movement of the mouth. The customer service evaluation device described.
A voice input unit for receiving the voice of the customer;
The customer service evaluation device according to claim 1, wherein the speech determination unit determines whether or not the customer speaks based on the voice of the customer.
A detection information input unit for inputting the detection information of the service partner;
A customer service state determination unit that determines whether or not the customer service is in service based on the detection information;
The customer service evaluation device according to claim 1, wherein when the customer service state determination unit determines that the customer service is in service, the customer service attitude of the customer service is evaluated.
A detection information input unit for inputting the detection information of the service partner;
A customer service state determination unit that determines whether or not the customer service is in service based on the detection information;
3. The customer service evaluation device according to claim 2, wherein the face image output unit outputs a face image when the customer is not speaking and during customer service.
A customer service evaluation device according to claim 1;
A service evaluation system comprising: an image capturing device that captures an image in the service area and inputs the captured image to the service evaluation device.
A customer service evaluation device according to claim 5;
An image capturing device that captures an image in the customer service area and inputs the captured image to the customer service evaluation device;
A customer service evaluation system comprising: a customer service partner detection device that detects a customer service partner and inputs detection information of the customer service customer to the customer service evaluation device.
The customer service evaluation system according to claim 8, wherein the customer service partner detection device is an image pickup device, a sound acquisition device, or a proximity sensor.
A customer service evaluation method for evaluating customer service attitudes,
An image acquisition step of acquiring an image captured in a preset service area;
A face image extraction step of extracting the customer's face image from the image;
A smile degree determination step for determining a degree of smile of the customer based on the customer's face image;
An utterance determination step for determining the presence or absence of the utterance of the customer;
A customer who evaluates the customer service attitude of the customer based on the smile level of the customer determined in the smile degree determination step when the customer is determined not to speak in the speech determination step A customer service evaluation method comprising: an attitude evaluation step.
A customer service evaluation method for evaluating customer service attitudes,
An image acquisition step of acquiring an image captured in a preset service area;
A face image extraction step of extracting the customer's face image from the image;
An utterance determination step for determining the presence or absence of the utterance of the customer;
A customer service evaluation method comprising: a facial image output step of outputting a facial image of the customer who is not speaking.