WO2016051693A1

WO2016051693A1 - Service monitoring system and service monitoring method

Info

Publication number: WO2016051693A1
Application number: PCT/JP2015/004661
Authority: WO
Inventors: 久裕田中; 昭年泉; 正成宮本; 信一重永; 亮太藤井; 広志田中; 寿嗣辻
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2014-09-30
Filing date: 2015-09-14
Publication date: 2016-04-07

Abstract

A service monitoring system is provided with: a sound collection unit that collects the sound of an employee's voice in a predetermined sound collection region; a storage unit in which service event data including a determination condition for each predetermined service event, terminal operation history data indicating an employee operation history with respect to a predetermined work terminal, and employee voice data are stored in association with one another; a detection unit that detects an employee's service event on the basis of the service event data and the terminal operation history data; a calculation unit that calculates a service speech evaluation value corresponding to a predetermined speech keyword on the basis of the employee voice data at the time of the detected event; and an output unit that stores the service speech evaluation value in association with employee identification information and the employee voice data identified by the employee's service position and service time.

Description

Service monitoring system and service monitoring method

The present disclosure relates to a customer service monitoring system and a customer service monitoring method for monitoring a situation during customer service using an employee's voice.

Conventionally, it has been known that customer satisfaction greatly affects sales performance in various customer service operations, and store managers (for example, store managers) are required to measure customer satisfaction quantitatively. It has become so.

の一 One example of a method for measuring customer satisfaction is a method of covering investigation by an investigator. In the store survey by the investigator, the survey results obtained depending on the environment where the store survey is conducted are different, and the improvement advice based on the survey results may not be accurate. Differences in the environment in which surveys are conducted include, for example, the degree of congestion and the number of store staff (employees) in the stores subject to store surveys for each time period during which the store surveys were conducted, the survey staff's survey technology, and the store staff receiving the store surveys This is the difference in the skill level of customer service (for example, the difference in working years).

Therefore, in Patent Document 1, a service evaluation diagnosis system that presents advice information based on the reality based on the corrected survey result after correcting the influence on the survey result based on the difference in the environment in which the store survey is performed. Is disclosed. In Patent Document 1, a survey result of a store survey (for example, a store survey) is input by an investigator who operates a portable information terminal.

Another example of a method for measuring customer satisfaction is to recognize store clerk's emotion and customer's emotion based on the store clerk's voice and customer's voice included in the store clerk's customer conversation, and based on this recognition result. There is known a customer service data recording device for calculating store clerk satisfaction data and customer satisfaction data (see, for example, Patent Document 2).

The customer service data recording device shown in Patent Document 2 records customer service data in which a store clerk satisfaction data, a customer satisfaction data, and a sales record of a store clerk for a customer are associated with each other in a database. In Patent Document 2, unlike Patent Document 1, an input operation by an investigator who operates a portable information terminal is not necessary.

This disclosure does not use human resources like investigators, and protects customer privacy extensively, and monitors customer utterances during various customer service events for customers in the store. It is an object of the present invention to provide a customer monitoring system and a customer monitoring method for accurately and objectively evaluating the situation.

Japanese Patent No. 5336675 Japanese Patent No. 5533219

The customer service monitoring system according to the present disclosure includes a sound collection unit that collects employee voices in a predetermined sound collection area, a first storage unit that stores customer event data including determination conditions for each predetermined customer event, and a predetermined storage unit. A second storage unit that stores the terminal operation history data indicating the operation history of the employee for the business terminal and the voice data of the employee collected by the sound collection unit in association with each other, and the first storage unit Based on the customer service event data stored in and the terminal operation history data stored in the second storage unit, a detection unit for detecting the customer service event of the employee, and the detection unit detected by the detection unit In a service event, a service utterance evaluation value corresponding to a predetermined utterance keyword at the time of operation of the business terminal is calculated based on the voice data of the employee stored in the second storage unit. The service utterance evaluation value calculated by the determination unit and the calculation unit is stored in association with the employee's voice data specified by the employee identification information, the customer service position and the customer service time. And an output unit.

Further, the customer service monitoring method of the present disclosure is a customer service monitoring method in a customer service monitoring system including a sound collection unit that collects an employee's voice in a predetermined sound collection region, and includes a determination condition for each predetermined customer service event. The customer service event data is stored in the first storage unit, and the terminal operation history data indicating the operation history of the employee with respect to the predetermined business terminal is associated with the voice data of the employee collected by the sound collecting unit. The customer service event is detected based on the customer service event data stored in the second memory and stored in the first memory and the terminal operation history data stored in the second memory. Then, in the detected customer service event, based on the employee's voice data stored in the second storage unit, a predetermined utterance keyword at the time of operating the business terminal The customer service utterance evaluation value corresponding to the employee is calculated, and the calculated customer service utterance evaluation value is associated with the employee's voice data specified by the employee identification information, the employee service position, and the customer service time. Remember.

FIG. 1 is a diagram illustrating an example of an image in a store where a customer service monitoring system according to the present embodiment is installed. FIG. 2 is a block diagram illustrating a first system configuration example of the customer service monitoring system according to the present embodiment. FIG. 3 is a block diagram illustrating a second system configuration example of the customer service monitoring system according to the present embodiment. FIG. 4 is a block diagram illustrating a third system configuration example of the customer service monitoring system according to the present embodiment. FIG. 5 is a block diagram illustrating a fourth system configuration example of the customer service monitoring system according to the present embodiment. FIG. 6 is a flowchart for explaining an example of the overall operation procedure of the customer service monitoring system of this embodiment. FIG. 7 is a flowchart for explaining an example of a detailed operation procedure of the event detection availability determination process. FIG. 8 is a flowchart for explaining an example of a detailed operation procedure of the customer service event detection process. FIG. 9 is a flowchart illustrating an example of a detailed operation procedure of customer service event information processing. FIG. 10 is a diagram illustrating an example of the customer service situation DB. FIG. 11 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system illustrated in FIG. 2. FIG. 12 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system illustrated in FIG. 3. FIG. 13 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system illustrated in FIG. 4. FIG. 14 is a diagram showing an example of a customer service event information DB corresponding to the customer service monitoring system shown in FIG. FIG. 15 is a flowchart for explaining an example of the operation procedure of the customer service utterance evaluation process. FIG. 16 is a flowchart for explaining an example of the operation procedure of the noise level determination process. FIG. 17 is a flowchart for explaining an example of the operation procedure of the service keyword utterance determination process. FIG. 18 is a flowchart for explaining an example of the operation procedure of the scoring process. FIG. 19A is a flowchart for explaining an example of an operation procedure of speech length determination processing. FIG. 19B is a flowchart illustrating an example of an operation procedure of a frequency characteristic determination process. FIG. 20 is a diagram illustrating a specific example of an utterance length determination process using model voice data. FIG. 21 is a diagram showing a specific example of the frequency characteristic determination process using the fundamental frequency of each phoneme of the model voice data. FIG. 22A is a diagram illustrating an example of an utterance assumption keyword table constituting a part of the customer service utterance evaluation DB. FIG. 22B is a diagram illustrating an example of a service utterance model list that forms part of the service utterance evaluation DB. FIG. 23 is a flowchart illustrating an example of an operation procedure of a browsing process by a limited viewer or a service utterance evaluation value correction process. FIG. 24 is a flowchart for explaining an example of a detailed operation procedure of the service utterance evaluation value correction process. FIG. 25 is a flowchart for explaining the continuation of the detailed operation procedure of the modification process of the customer service utterance evaluation value shown in FIG. FIG. 26A is a diagram illustrating an example of a viewer DB. FIG. 26B is a diagram illustrating an example of a customer service DB. FIG. 27 is a diagram illustrating an example of a login screen to the customer service status DB to be browsed in the customer service monitoring system. FIG. 28 is a diagram illustrating an example of a totaling result of customer service utterance evaluation values of all customer service customers per day as a customer service status display screen. FIG. 29 is a diagram illustrating an example of a totaling result of customer service utterance evaluation values of all customers with respect to customers in a day time zone. FIG. 30A is a diagram illustrating an example of a totaling result of customer utterance evaluation values of one customer for each time slot of one day. FIG. 30B is a diagram illustrating an example of a total result of customer service utterance evaluation values for each customer service per day. FIG. 31 is a diagram illustrating an example of a totaling result of customer service utterance evaluation values for each store on a day. FIG. 32 is a diagram illustrating a specific example of each record displayed on the detail display screen of the customer service situation DB. FIG. 33 is a diagram illustrating an example of a modification operation for the customer utterance evaluation value of a specific record displayed on the detail display screen of the customer service status DB. FIG. 34 is a diagram illustrating an example of the customer utterance evaluation value after correction of a specific record displayed on the detail display screen of the customer service situation DB. FIG. 35 is a diagram illustrating an example of a customer service position correcting operation on the customer service status preview screen. FIG. 36 is a diagram illustrating an example of coordinates of the customer service position after correction of a specific record displayed on the detail display screen of the customer service DB. FIG. 37 is a diagram illustrating an example of the relationship between the customer microphone device, the microphone device, the microphone array device, and the privacy protection mark. FIG. 38 is an explanatory diagram illustrating an example of shift cut-out processing of monitoring data. FIG. 39 is an explanatory diagram illustrating an example of a variation of the monitoring data cut-out process. FIG. 40 is a flowchart for explaining another example of the detailed operation procedure of customer service event information processing. FIG. 41 is a diagram illustrating an example of an utterance assumption section table for each service event that constitutes a part of the service utterance evaluation DB.

Hereinafter, an embodiment (hereinafter referred to as “the present embodiment”) that specifically discloses the customer service monitoring system and the customer service monitoring method according to the present disclosure will be described with reference to the drawings. The customer service monitoring system according to the present embodiment is installed in a store (for example, a retail store, a wholesale store, a department store, a convenience store, a supermarket, a restaurant, or a bank) where customer service is performed, and serves a customer of a store clerk (employee) to a customer. The situation is monitored, and the customer service attitude (hospitality) of the store clerk who performs various customer service events (for example, store entrance / exit greetings, accounting start greetings, etc., see below for details) is objectively evaluated. In the following description, a quantitative index (value) as a result of objectively evaluating the customer service attitude (customer service situation) of the store clerk in the customer service monitoring system is referred to as a “customer utterance evaluation value”. In addition, although this embodiment demonstrates the evaluation of the customer service situation with respect to the customer of the clerk, the customer service monitoring system and the customer service monitoring method according to the present disclosure are a customer service to an employee other than the clerk (for example, a bank clerk, an employee, etc.) It is also applicable to situational assessment.

In addition, this indication is a method including each operation | movement which each apparatus (for example, customer service evaluation apparatus mentioned later) which comprises a customer service monitoring system, each apparatus (for example, customer service evaluation apparatus) which comprises a directivity control system, or the said method It can also be expressed as a program to be executed by the customer service evaluation device which is a computer.

(Outline of customer service monitoring system)
FIG. 1 is a diagram illustrating an example of an image in a store where a customer service monitoring system 100 according to the present embodiment is installed. In FIG. 1, for example, in a store, two employees each serve a plurality of customers while operating the POS terminal 5 at a cashier counter in which two POS terminals 5 are installed. At the store, at least one microphone device M1 for picking up sound around the cashier counter, which is a sound pickup area, is installed, and at least one camera device C1 for taking an image so as to include the cashier counter is included. In addition, a sensor device S1 for detecting the entrance and exit of the customer to the store is installed near the entrance of the store. In the customer service monitoring system 100, the customer service evaluation device 3 (for example, PC :) for calculating the customer service utterance evaluation value by monitoring (monitoring) the customer service situation in the backyard (for example, monitoring room) of the store. Personal Computer) is installed and operated by, for example, a store manager (for example, store manager).

The customer service monitoring system 100 detects, for example, data indicating the operation history of a store clerk who operates the POS terminal 5 (hereinafter referred to as “POS operation history data”) and presence / absence of a customer service event for each of various customer service events to be described later. A customer service event by a store clerk is detected based on customer service event data including a customer service event determination condition. The POS operation history data includes, for example, a customer ID of the customer (store clerk) (a store clerk identification information), a customer ID input operation history for inputting the customer ID into the POS terminal 5, and the age of the customer (customer). Includes input history, transaction completion operation history, and the like.

When the customer service monitoring system 100 detects a customer service event, for example, based on data obtained by the microphone device M1, the camera device C1, the sensor device S1, or a combination thereof, a predetermined utterance when operating the POS terminal 5 A service utterance evaluation value corresponding to the assumed keyword (utterance keyword) (in other words, a service utterance evaluation value for a clerk who utters the utterance assumed keyword) is calculated. In addition, the customer service monitoring system 100 stores the calculated customer service utterance evaluation value in association with the clerk's voice data specified by the clerk's identification information, the clerk's customer service position, and the customer service time. The “salesperson (employee) voice data” here refers to, for example, voice recorded by the microphone device M1, the customer microphone device SM1, or the microphone array device AM1 of the voice uttered by the store clerk at the position of the store clerk. Data.

(Configuration example of customer service monitoring system)
Next, a system configuration example of the customer service monitoring system according to the present embodiment will be described with reference to FIGS. FIG. 2 is a block diagram illustrating a first system configuration example of the customer service monitoring system 100 according to the present embodiment. FIG. 3 is a block diagram illustrating a second system configuration example of the customer service monitoring system 100A according to the present embodiment. FIG. 4 is a block diagram illustrating a third system configuration example of the customer service monitoring system 100B according to the present embodiment. FIG. 5 is a block diagram illustrating a fourth system configuration example of the customer service monitoring system 100C according to the present embodiment. 2 to FIG. 5, the customer service monitoring system 100 shown in FIG. 2 will be described in detail, and the structures of the customer

service monitoring systems

100A, 100B, and 100C shown in FIG. 3, FIG. 4, and FIG. Differences from the customer service monitoring system 100 will be described, and the same components will be denoted by the same reference numerals, and description thereof will be simplified or omitted.

The customer service monitoring system 100 shown in FIG. 2 includes at least one microphone device M1,..., ML (L is an integer of 1 or more; the same applies hereinafter), a management server 2, a customer service evaluation device 3, a recorder device 4, The POS terminal 5 is included. In the customer service monitoring system 100 shown in FIG. 2, at least one microphone device M1,..., ML, management server 2, customer service evaluation device 3, recorder device 4, and POS terminal 5 are connected via a network NW. Are connected to each other. The network NW may be a wired network (for example, an intranet or the Internet), or may be a wireless network (for example, a wireless LAN (Local Area Network)).

At least one microphone device M1,..., ML as an example of a sound collection unit is installed on a predetermined installation surface (for example, a ceiling surface) of a predetermined sound collection region (for example, a cash register counter of a store), and The voice of the employee (store clerk) is collected, and the voice data of the clerk obtained by the sound collection is transmitted to the recorder device 4. The directivities of the microphone devices M1,..., ML are already determined by the design specifications from the time of manufacture, including the non-directivity having no directivity, and cannot be changed. In addition, the microphone devices M1,..., ML do not collect only the voice of the employee (store clerk), but for example, the voice spoken by the customer when the customer and the employee (clerk) are talking is leaked. It is possible to pick up sound.

Further, the microphone devices M1,..., ML may be provided for customer service for evaluating the customer service situation of the clerk's customer, or may be provided for monitoring such as crime prevention in the store (see FIG. 37). . The microphone device for monitoring is installed in a place where it is difficult to see in the store, a place far away from the cashier counter in the store, and the like.

FIG. 37 is a diagram showing an example of the relationship between the customer microphone devices SM1,..., SML, microphone devices M1,... ML, microphone array devices AM1,. The privacy protection mark is an example of predetermined information indicating protection of the privacy of the customer, for example. In order to protect the privacy of the customer, the customer's voice may be mixed with the clerk's voice data and collected. This is information indicating in advance that it is not a target for customer service evaluation using the voice data.

That is, as shown in FIG. 37, the microphone device provided for the customer service purpose stores information indicating that the microphone device is provided for the customer service purpose in advance, and the microphone device provided for the monitor purpose is provided for the monitor purpose. Information indicating this is held in advance. The privacy protection mark is given to the clerk's voice data obtained by the sound collection of the microphone device provided for monitoring purposes by the processing of the microphone device. However, the privacy protection mark is not given to the clerk's voice data obtained by collecting sound from the microphone device provided for customer service.

The management server 2 as an example of the first storage unit calculates the customer service utterance evaluation value of the store clerk for each customer service event in the customer service evaluation device 3, or the customer service utterance of the store clerk for each customer service event calculated by the customer service evaluation device 3. Various data necessary for browsing the evaluation value (further modified as necessary) is stored (saved) as a management DB (Database) 2a. Further, the management server 2 stores the customer service utterance evaluation value of the store clerk for each customer service event calculated by the customer service evaluation device 3 in the management DB 2a.

The management DB 2a includes a customer service event information DB, a viewer DB, a customer service DB, a customer service utterance evaluation DB, and a customer service situation DB. Details of the contents of each DB will be described later. Note that the management server 2 may not be disposed in the store itself where the customer service monitoring system 100 is installed, and may be, for example, online storage (for example, storage used in a cloud service) connected via the network NW.

The customer service evaluation device 3 detects various customer service events in a predetermined sound collection area (for example, in a store), and further, based on the detected clerk's voice data during the customer service event, The customer utterance evaluation value corresponding to the utterance assumption keyword is calculated. The customer service evaluation device 3 includes, for example, a data communication device such as a PC (including a laptop and a desktop), a smartphone, a tablet terminal, a mobile phone, or a PDA (Personal Digital Assistant), and includes an operation unit 31, a memory 32, and the like. The customer utterance evaluation unit 33, the output unit 34, the display device 35, the speaker device 36, and the monitoring data extraction unit 38.

The operation unit 31 is a user interface (UI: User Interface) for notifying the customer utterance evaluation unit 33 or the output unit 34 of the operation contents of the user (for example, the store manager), for example, a pointing device such as a mouse or a keyboard. It is. Moreover, the operation part 31 may be comprised using the touch panel or touchpad which is arrange | positioned corresponding to the screen of the display apparatus 35, for example, and can be operated with a user's finger | toe FG or a stylus pen.

The memory 32 is configured by using, for example, a RAM (Random Access Memory), functions as a work memory when each part of the customer service evaluation device 3 operates, and further stores data necessary when each part of the customer service evaluation device 3 operates. .

The customer service utterance evaluation unit 33 includes, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor), and includes a customer service event detection unit 331 and a service utterance evaluation value calculation unit 332. It is the composition which has.

The customer service event detection unit 331 as an example of the detection unit is a customer service event information DB (customer service event data, see later) of the management DB 2a of the management server 2 and a store clerk's operation history for the POS terminal 5 as an example of a predetermined business terminal. The customer service event of the store clerk is detected based on the POS operation history data (terminal operation history data). Details of the service event detection method will be described later.

The customer service utterance evaluation value calculation unit 332 as an example of a calculation unit is based on the clerk's voice data stored (stored) in the recorder device 4 in the customer service event detected by the customer service event detection unit 331. A service utterance evaluation value corresponding to a predetermined utterance assumed keyword at the time of operation is calculated. Details of the method for calculating the customer service utterance evaluation value will be described later.

The output unit 34 is configured using, for example, a CPU, MPU, or DSP, and the customer service utterance evaluation value calculated by the customer service utterance evaluation value calculation unit 332 is printed on identification information of the store clerk (for example, on a name tag worn by the store clerk, The data is stored in the memory 32 or the management DB 2a of the management server 2 in association with the clerk's voice data specified by the clerk's customer service position (for example, coordinate information) and the customer service time.

The output unit 34 has a communication function (wired communication function, wireless communication function) via the network NW with each device of the customer service monitoring system 100, and controls the operation of the display device 35 and the speaker device 36. Various screens related to the customer service monitoring system 100 are displayed on the display device 35 according to a predetermined input operation by the user, or voice packets transmitted from the microphone devices M1,... ML are received and output from the speaker device 36. Or

The display device 35 as an example of the display unit is configured by using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), for example, customer monitoring under the control of the output unit 34 according to a user input operation. Various screens (see later) related to the system 100 are displayed.

The speaker device 36 as an example of the audio output unit outputs audio data included in an audio packet transmitted from the microphone devices M1, ..., ML. The display device 35 and the speaker device 36 may be configured integrally with the customer service evaluation device 3 or may be different devices.

The monitoring data extracting unit 38 as an example of the voice data extracting unit calculates the customer service utterance evaluation value of the store clerk from the monitoring data 4a stored in the recorder device 4 by the customer service utterance evaluation value calculator 332 of the customer service utterance evaluation unit 33. The monitoring data 4ak (k is an integer of 1 or more, for example, see FIG. 38) is extracted every predetermined time interval (for example, about 10 to 20 seconds) necessary for the extraction. Details of the monitoring data extraction processing in the monitoring data extraction unit 38 will be described later with reference to FIGS.

The recorder device 4 as an example of the second storage unit stores, as monitoring data 4a, voice data of a clerk uttered at a customer service event and POS operation history data indicating an operation history of the POS terminal 5, for example. The clerk's voice data is voice data collected by at least one microphone device M1, ..., ML and transmitted to the recorder device 4. The POS operation history data is, for example, data acquired at the POS terminal 5 and further acquired by the management server 2 or the customer service evaluation device 3 from the POS terminal 5 and transmitted to the recorder device 4.

A POS terminal 5 as an example of a business terminal is installed at a cash register counter in a store and has an input device 51, a display device 52, and a memory 53. The POS terminal 5 stores, for example, store sales information and information regarding the price of each product in the memory 53. 2 to 5, only one POS terminal 5 is illustrated, but a plurality of POS terminals 5 may be connected via the network NW.

As with the operation unit 31, the input device 51 is a user interface (UI) for receiving an input operation of a user (for example, a store clerk) and notifying the POS terminal 5, and is, for example, a pointing device such as a mouse or a keyboard. Further, the input device 51 may be configured using a touch panel or a touch pad that is arranged corresponding to the screen of the display device 52 and can be operated by a user's finger FG or stylus pen, for example.

The display device 52 is configured by using an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) in the same manner as the display device 35, and according to a user input operation, store sales information, information on the price of each product, and products Displays a screen related to the settlement of

The memory 53 is configured by using a RAM (Random Access Memory) similarly to the memory 32, functions as a work memory when each part of the POS terminal 5 is operated, and further stores data necessary when each part of the POS terminal 5 is operated. Remember.

In the customer service monitoring system 100A shown in FIG. 3, at least one microphone device M1,..., ML of the customer service monitoring system 100 shown in FIG. 2 is replaced with at least one customer microphone device SM1,. The other configuration is the same as that of the customer service monitoring system 100 shown in FIG. The directivities of the customer microphone devices SM1,..., SML are already determined by design specifications from the time of manufacture, and cannot be changed.

The at least one customer microphone device SM1,..., SML as an example of the sound collection unit is configured by using, for example, a pin microphone and is individually attached to each store clerk in the store to collect the sound of the corresponding store clerk. Then, the clerk's voice data obtained by sound collection is transmitted to the recorder device 4. Note that the customer microphone devices SM1,..., SML do not pick up only the voice of the employee (store clerk), for example, the voice spoken by the customer when the customer and the employee (clerk) are talking. It is possible to leak and pick up sound.

Since the customer service microphone device is often provided for customer service purposes, the customer service microphone devices SM1,..., SML of this embodiment hold information indicating that they are provided for customer service purposes in advance. Therefore, as shown in FIG. 37, the privacy protection mark is not given to the clerk's voice data obtained by the sound collection of the customer microphone devices SM1,..., SML.

In the customer service monitoring system 100A shown in FIG. 3, the monitoring data 4a stored (stored) in the recorder device 4 is the same as the customer service monitoring system 100 shown in FIG. This is voice data of a clerk picked up by a device (for example, a customer microphone device SM1).

In the customer service monitoring system 100B shown in FIG. 4, at least one camera device C1,..., CM and at least one sensor device S1,. Other configurations are the same as those of the customer service monitoring system 100 shown in FIG.

At least one camera device C1,..., CM (M is an integer of 1 or more) as an example of an imaging unit is installed fixedly on a ceiling surface of a store, for example, and has a function as a surveillance camera or a security camera. , By remote operation from the customer service evaluation device 3 connected to the network NW, using the zoom function (for example, zoom-in process, zoom-out process) and the optical axis movement function (pan, tilt), respectively, the camera devices C1,. Captures an image within the angle of view of the CM.

The installation positions and directions of the respective camera devices C1,..., CM are registered in advance in, for example, the memory 32 of the customer service evaluation device 3, and control information related to pan / tilt / zoom is transmitted to the customer service evaluation device 3 as needed to display images. The positional relationship between each image position and the directing direction is always associated with each other. Further, when each of the camera devices C1,..., CM is an omnidirectional camera, for example, predetermined video data (that is, omnidirectional video data) indicating omnidirectional video in the sound collection area or omnidirectional video data is predetermined. The plane image data generated by performing the panorama conversion by performing the distortion correction process is transmitted to the customer service evaluation device 3 via the network NW. Note that the angle of view and the optical axis of each of the camera devices C1, ..., CM may be fixed.

The output unit 34 causes the display device 35 to display video data transmitted from any one of the camera devices C1,..., CM according to, for example, a user input operation.

The at least one sensor device S1,..., SN as an example of the customer detection unit detects the appearance or exit of the customer from the store (in other words, the customer enters or exits the store), and information on the detection result is obtained as sensor data. As shown in FIG. A plurality of sensor devices S1,..., SN may be provided according to the type and number of customer service events that can be detected by the customer service monitoring system 100.

In the service monitoring system 100B shown in FIG. 4 and the service monitoring system 100C shown in FIG. 5, a microphone device that picks up a predetermined position (preset position) in the store and a camera device that images the predetermined position are associated in advance. ing. For this reason, the preset ID that is the identification information of the preset position and the camera ID that is the identification information of the camera device that captures the preset position are associated in advance.

In the customer service monitoring system 100B shown in FIG. 4, the monitoring data 4b stored (stored) in the recorder device 4 includes the POS operation history data for each store clerk and the microphone device (for example, the microphone device) in the customer service monitoring system 100 shown in FIG. In addition to the clerk's voice data collected by SM1), video data transmitted from at least one camera device C1,..., CM and at least one sensor device S1,. The sensor data transmitted from is further added.

5, the customer monitoring system 100C shown in FIG. 2 replaces at least one microphone device M1,..., ML of the customer monitoring system 100 with at least one microphone array device AM1,. , A directivity control unit 37 is added, and at least one camera device C1,..., CM and at least one sensor device S1,..., SN are further added. This is the same as the customer service monitoring system 100 shown in FIG.

The at least one microphone array device AM1,..., AML as an example of the sound collection unit is installed on a predetermined installation surface (for example, a ceiling surface) of a predetermined sound collection region (for example, a cash register counter of a store). Pick up the clerk's voice. Specifically, at least one microphone array device AM1,..., AML includes a plurality of microphones as an example of sound collection elements, and a plurality of microphones are used to center the installation positions of the microphone array devices AM1,. As an example, a voice (for example, a clerk's voice) from a 360 ° direction (omnidirectional) is collected. In addition, the microphone array devices AM1,..., AML do not pick up only the voice of the employee (store clerk), and for example, the voice spoken by the customer when the customer and the employee (clerk) are talking is leaked. It is possible to pick up sound.

Further, the microphone array devices AM1,..., AML may be provided for customer service for evaluating the customer service situation of the clerk's customer, or may be provided for surveillance such as store crime prevention (see FIG. 37). ). The microphone array device for monitoring is installed in a place where it is difficult to see in the store, a place far away from the cashier counter in the store, and the like.

That is, as shown in FIG. 37, the microphone array device provided for customer service holds information indicating that it has been provided for customer service, and the array microphone device provided for monitoring is provided for monitoring purposes. Information indicating that it has been received is held in advance. A privacy protection mark is given to the sound data of the store clerk obtained by the sound collection of the microphone array device provided for monitoring purposes by the processing of the array microphone device. However, the privacy protection mark is not given to the clerk's voice data obtained by collecting sound from the microphone array device provided for customer service.

The at least one microphone array device AM1,..., AML transmits a voice packet including the voice collected by each microphone as voice data to the recorder device 4 via the network NW.

The operation unit 31 designates an image on the screen displayed on the display device 35 (for example, an image captured by any one of the camera devices C1,..., CM; the same applies hereinafter) by a user operation. The coordinate data indicating the position of the image on the screen is acquired and output to the customer service utterance evaluation unit 33 or the output unit 34.

Each camera device C1,..., CM is designated when an arbitrary position is designated by the user's finger FG or stylus pen in a state in which the video data captured by each camera device is displayed on the screen of the display device 35. The coordinate data of the position is received from the customer service evaluation device 3, and the distance and direction (horizontal angle and vertical) from each camera device to the position in the real space corresponding to the designated position (hereinafter simply referred to as “voice position”). Data including corners, and so on) is calculated and transmitted to the customer service evaluation device 3. Note that the distance and direction data calculation processing in any camera device is a known technique, and thus description thereof is omitted.

The directivity control unit 37 designates from one of the microphone array devices associated with the camera device that captured the video in accordance with the user's position designation operation from the video displayed on the screen of the display device 35. Coordinates indicating the directivity direction toward the voice position corresponding to the position are calculated. Since the calculation method of the coordinate which shows the directivity direction of the directivity control part 37 is a well-known technique, detailed description is abbreviate | omitted.

The directivity control unit 37 acquires, for example, data on the distance and direction from the installation position of the camera device C1 to the audio position from the camera device C1, and uses these data, for example, the microphone array device AM1 (for example, the camera device C1 and the like). The coordinates indicating the directivity direction from the installation position of the microphone array apparatus AM1 to the voice position are calculated. For example, when the housing of the microphone array device AM1 and the camera device C1 are integrally attached so as to surround the housing of the camera device C1, the direction from the camera device C1 to the sound position (horizontal angle, vertical angle) ) Can be used as coordinates indicating the directivity direction from the microphone array device AM1 to the voice position.

When the housing of the camera device C1 and the housing of the microphone array device AM1 are mounted apart from each other, the directivity control unit 37 calculates calibration parameter data calculated in advance and the camera device C1. The coordinates indicating the directivity direction from the microphone array device AM1 to the voice position are calculated using the data from the direction to the voice position (horizontal angle, vertical angle). The calibration is an operation for calculating or acquiring a predetermined calibration parameter required for the directivity control unit 37 of the customer service evaluation device 3C to calculate the coordinates indicating the directivity direction. Suppose that

The coordinates indicating the directivity direction are indicated by a horizontal angle in the directivity direction from the microphone array device AM1 toward the sound position and a vertical angle in the directivity direction from the microphone array device AM1 toward the sound position. The sound position is the actual monitoring target or sound collection target corresponding to the designated position designated by the user's finger FG or stylus pen in the video displayed on the screen of the display device 35 by the operation unit 31. Position (see FIG. 1).

The directivity control unit 37 emphasizes the voice data by forming the directivity in the direction indicated by the calculated coordinates using the voice data of the store clerk included in the voice packet transmitted from the microphone array device AM1, for example. Then, the audio data after the enhancement processing is generated and passed to the output unit 34. Note that the enhancement processing in the directivity control unit 37 may be performed by any microphone array device corresponding to the camera device that captured the video selected by the user.

In the customer service monitoring system 100C shown in FIG. 5, the monitoring data 4b stored (stored) in the recorder device 4 includes the POS operation history data for each store clerk and the microphone device (for example, the microphone device) in the customer service monitoring system 100 shown in FIG. In addition to the clerk's voice data collected by M1), video data transmitted from at least one camera device C1,..., CM and at least one sensor device S1,. The sensor data transmitted from is further added.

5, the microphone array devices AM1,..., AML are connected to the network NW. However, some or all of the microphone devices M1,... ML shown in FIG. A part or all of the customer microphone devices SM1,... SML shown in FIG.

Next, an overall operation procedure common to the customer

service monitoring systems

100, 100A, 100B, and 100C of the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart for explaining an example of the overall operation procedure of the customer service monitoring system of this embodiment. In the following description, the

service monitoring system

100, 100A, 100B, 100C basically operates in the same manner, but for the sake of simplicity, for example, the system of the service monitoring system 100 shown in FIG. It demonstrates using a structure, and it demonstrates with reference to the structure of each customer

service monitoring system

100A, 100B, 100C shown to FIGS. 3-5 as needed.

In FIG. 6, when the customer service evaluation is completed (S1, YES), the operation of the customer service monitoring system 100 shown in FIG. 6 is completed. Note that the case where the customer service evaluation is completed includes, for example, a case where the “end” button of the customer service evaluation application installed in the customer service evaluation device 3 is pressed, or a case where the customer service evaluation device 3 is shut down. However, the present invention is not limited to these cases.

On the other hand, when the customer service evaluation is not completed (S1, NO), the monitoring data extraction unit 38 of the customer service evaluation device 3 calculates the customer service speech evaluation value calculation unit 332 of the customer service speech evaluation unit 33 to calculate the customer service speech evaluation value of the store clerk. Therefore, the monitoring data 4a acquired from the recorder device 4 is cut out at predetermined time intervals (for example, about 10 to 20 seconds). The length of the monitoring data cut out by the monitoring data extraction unit 38 is based on a customer service keyword (for example, “Thank you very much” after or before or after the customer service event (for example, accounting completion operation) that is assumed in advance. ")" Can be entered until the end of the utterance. At that time, the monitoring data extraction unit 38 of the customer service evaluation device 3 shifts the cutting start time, which is a starting point for cutting the monitoring data 4a, for a predetermined time (for example, about 1 second) from the cutting start time of the immediately preceding monitoring data. (S2, see FIG. 38). However, the monitoring data extraction unit 38 of the customer service evaluation device 3 does not perform a shift process when the monitoring data 4a is cut for the first time. FIG. 38 is an explanatory diagram showing an example of the shift cut-out process of the monitoring data 4a. In the process of step S2, if there is a customer service event at the boundary point of the predetermined time interval of the monitoring data (for example, monitoring data 4a2 shown in FIG. 38) acquired in step S3, the service keyword of the customer service keyword is extracted from the cut monitoring data 4a2. It is provided in order to avoid that detection becomes difficult. Among the plurality of monitoring data 4a1, 4a2, 4a3,... Generated by performing the process of step S2, the voice data of the customer service keyword assumed in advance is not interrupted in the middle, but from the beginning to the end. Monitoring data that has been recorded up to all will be included.

For example, as shown in FIG. 38, in the monitoring data 4a stored in the recorder device 4, the POS operation history data indicating that the accounting completion operation occurred at the time t1, and the salesclerk from the time t2 to the time t3, “Thank you. Is assumed to be stored. In this case, since the monitoring data 4a1 first cut by the monitoring data extraction unit 38 in step S2 does not include the clerk's voice data corresponding to the transaction completion operation, the customer service event cannot be detected accurately. Customer service evaluation is impossible. In the next step S2, the monitoring data extraction unit 38 of the customer service evaluation device 3 monitors the cutting start time, which is the starting point for cutting out the monitoring data 4a, by shifting the monitoring data 4a1 by a predetermined time ts from the cutting start time of the monitoring data 4a1. To extract. The data size of the monitoring data 4ak (k: an integer equal to or larger than 1) for a predetermined time interval acquired by cutting is the same. However, since the monitoring data 4a2 does not store all the voice data of the clerk's “Thank you” utterance, the customer service event cannot be detected, and the customer service evaluation cannot be performed in the same manner. In the next step S2, the monitoring data extraction unit 38 of the customer service evaluation device 3 performs monitoring by shifting the cutting start time, which is a starting point for cutting the monitoring data 4a, by a predetermined time ts from the cutting start time of the monitoring data 4a2. Data 4a3 is extracted. In the monitoring data 4a3, all the voice data of the clerk's “thank you” utterance is stored, so that the customer service evaluation device 3 can detect the customer service event.

As described above, the customer service evaluation device 3 includes the customer service keywords in each of the monitoring data 4a1, 4a2, 4a3,... That is always the same length (that is, the predetermined time ts) extracted by the monitoring data extraction unit 38. For example, it is only necessary to try to detect whether or not it is detected by voice recognition. Processing such as detection of speech start time and speech end time and setting of a range to be recognized by speech are not required. Therefore, it is possible to reliably detect a customer service keyword assumed in advance. Note that variations of the monitoring data cut start time will be described later with reference to FIGS. The method for cutting the monitoring data 4b is the same as the method for cutting the monitoring data 4a, and the description thereof is omitted.

After step S2, the monitoring data extraction unit 38 of the customer service evaluation device 3 acquires monitoring data 4a from the recorder device 4 every predetermined time interval (for example, about 10 seconds) from the start time set in step S2 (S3). The acquired monitoring data 4a (specifically, the clerk's POS operation history data and voice data included in the monitoring data 4a) and the start time and end time of the monitoring data 4a are stored in the memory 32 in association with each other (S4). ). The end time is a time obtained by adding a predetermined time interval from the start time.

After step S4, the customer service utterance evaluation unit 33 of the customer service evaluation device 3 performs an event detection availability determination process (S5), and if the event detection availability flag is set to “permitted” (S6, YES), the customer service is performed. The output unit 34 of the evaluation device 3 receives the monitoring data 4a for each predetermined time interval held in the memory 32 in step S4 (in other words, the monitoring data 4a for each predetermined time interval acquired from the recorder device 4). (I.e., customer service event detector 331, customer service utterance evaluation value calculator 332) (S7). After step S7, a service event detection process is performed in the service event detection unit 331 of the service utterance evaluation unit 33 of the service evaluation device 3 (S8).

On the other hand, if the event detection enable / disable flag is set to “NO” (S6, NO) or after step S8, the operation of the customer service monitoring system 100 returns to step S1.

FIG. 7 is a flowchart for explaining an example of a detailed operation procedure of the event detection availability determination process. In FIG. 7, the customer service utterance evaluation unit 33 of the customer service evaluation device 3 performs a predetermined area (for example, a header area, a part of the payload area, or other optional areas) of the monitoring data 4a for each predetermined time interval held in the memory 32 in step S4. ) Includes a privacy protection mark as an example of predetermined information indicating customer privacy protection (S5-1, YES), an event detection enable / disable flag indicating presence / absence of customer service event detection processing is set to “No”. (That is, omit the service event process without performing the service event process) (S5-2). After step S5-2, the operation of the customer service monitoring system 100 proceeds to step S6.

On the other hand, the customer service utterance evaluation unit 33 protects the privacy of the customer in a predetermined area (for example, a header area, a part of the payload area, or other optional areas) of the monitoring data 4a held in the memory 32 in step S4 at every predetermined time interval. If it is determined that the privacy protection mark as an example of the predetermined information is not included (S5-1, NO), the customer utterance evaluation unit 33 of the customer service evaluation device 3 uses the monitoring data as the voice of the customer who entered the store. It is determined whether it is not included in 4a (S5-3).

For example, the customer service utterance evaluation unit 33 includes keywords in the voice data included in the monitoring data 4a that are likely to be uttered by the customer in the store (more specifically, for example, the voice included in the monitoring data 4a). If it is determined that the word spotting processing result of a keyword that is likely to be spoken by the customer in the store with respect to the data is equal to or higher than a predetermined level), it is determined that the voice of the customer is included in the monitoring data 4a (S5). -4, YES).

Alternatively, the customer service utterance evaluation unit 33 collects voices other than the clerk registered in advance in the customer mic device individually attached to the clerk (more specifically, for example, collected voice data If the voice clerk recognition result of the clerk registered in advance is determined to be below a predetermined level), it may be determined that the customer's voice is included in the monitoring data 4a (S5-4, YES). .

Alternatively, the customer service utterance evaluation unit 33 of the customer service evaluation device 3C detects a face other than the face image of the store clerk registered in advance by performing image processing on the video data included in the monitoring data 4b, and further detects the detected face. If it is determined that the voice uttered from the position of the face or the voice of the detected face position that forms directivity and is emphasized includes human voice, the voice of the customer is monitored It may be determined that it is included in the data 4a (S5-4, YES).

When the customer service utterance evaluation unit 33 of the customer service evaluation device 3 (or the customer service evaluation device 3C) determines that the customer's voice is included as a result of the customer voice presence / absence determination process (S5-4, YES), the event The detection enable / disable flag is set to “NO” (S5-5).

On the other hand, when the customer service utterance evaluation unit 33 of the customer service evaluation device 3 (or customer service evaluation device 3C) determines that the customer's voice is not included as a result of the customer voice presence / absence determination process (S5-4, NO). Then, the event detection enable / disable flag is set to “enable” (S5-6). After step S5-5 and after step S5-6, the operation of the customer service monitoring system 100 proceeds to step S6.

Next, before describing the customer service event detection process in step S8 shown in FIG. 6, the determination conditions for each predetermined customer service event corresponding to the customer

service monitoring systems

100, 100A, 100B, and 100C shown in FIGS. 2 to 5 are included. An example of a customer service event information DB as an example of customer service event data will be described with reference to FIGS. Each customer service event information DB shown in FIGS. 11 to 14 is stored in the management DB 2 a of the management server 2.

FIG. 11 is a diagram showing an example of a customer service event information DB corresponding to the customer service monitoring system 100 shown in FIG. FIG. 12 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system 100A illustrated in FIG. FIG. 13 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system 100B illustrated in FIG. FIG. 14 is a diagram illustrating an example of a customer service event information DB corresponding to the customer service monitoring system 100C illustrated in FIG. In the description of FIG. 12 to FIG. 14, description of contents overlapping with those of FIG. 11 will be omitted, and different contents will be described.

The customer service event information DB shown in FIG. 11 includes a customer service event ID, a customer service event name, a customer service event determination condition (that is, a condition for determining whether or not a customer service event is detected in the monitoring data 4a), and a customer service event. The type and type of data corresponding to each item of output information (that is, information output when a customer service event is detected) are defined.

The customer service event determination condition of the customer service event information DB shown in FIG. 11 stipulates that a customer service event detection trigger is a predetermined operation (POS operation) performed on the POS terminal 5.

The customer service event output information shown in FIG. 11 stipulates that a preset ID, customer service ID, and customer service event ID are output.

In the customer service event determination condition of the customer service event information DB shown in FIG. 12, the customer service event detection trigger is different for each customer service event. Specifically, a predetermined operation (POS operation) on the POS terminal 5 is performed, voice data Each of them is specified to contain a specific keyword.

The customer service event output information in the customer service event information DB shown in FIG. 12 is different for each customer service event, and specifically, a preset ID (identification information of a predetermined position in the store; the same applies hereinafter) and a customer service ID (store employee identification information. The same applies hereinafter) and the customer service event ID, and only the customer service ID (identification information of the customer microphone device worn by the store clerk; the same applies hereinafter) is defined.

The customer service event information DB shown in FIG. 13 includes a customer service event ID, an item indicating whether or not the event is a customer service event, a customer service event name, a customer service event determination condition (that is, a customer service event is detected in the monitoring data 4a). And the type of data corresponding to each item of customer service event output information (that is, information output when a customer service event is detected) are defined.

In the customer service event determination condition of the customer service event information DB shown in FIG. 13, the customer service event detection trigger is different for each customer service event. Specifically, the sensor device S1 (for example, an automatic door) installed near the store entrance is opened and closed. When a store clerk stays at a predetermined position (preset position) corresponding to a predetermined preset ID and there is a position corresponding to the predetermined visitor position preset ID (that is, there is a customer (customer) at the customer service event) It is specified that the customer stays at a highly likely position) for a predetermined time (for example, about 5 seconds) and that a predetermined operation (POS operation) for the POS terminal 5 has been performed.

The customer service event output information of the customer service event information DB shown in FIG. 13 is different for each customer service event, and specifically includes a combination of a microphone ID (see later), a camera ID (see later), a customer ID, and a customer service event ID. A combination of a preset ID, a customer service ID, and a customer service event ID is defined.

The customer service event output information in the customer service event information DB shown in FIG. 14 differs for each customer service event. Specifically, the customer service location coordinates, camera ID, customer service ID, customer service event ID combination, preset ID, and customer service A combination of ID and customer service event ID is defined. The customer service position coordinates are used when the directivity control unit 37 forms the sound directivity in the direction from the microphone array device that picks up the sound data of each store clerk toward each store clerk.

Next, details of the service event detection process (see step S8 shown in FIG. 6) using the specific example of the service event information DB shown in FIGS. 11 to 14 will be described with reference to FIGS. FIG. 8 is a flowchart for explaining an example of a detailed operation procedure of the customer service event detection process. FIG. 9 is a flowchart illustrating an example of a detailed operation procedure of customer service event information processing.

In the description of FIGS. 8 and 9, in order to make the description concrete and easy to understand, FIGS. 11 to 14 corresponding to the system configurations of the

customer monitoring systems

100, 100A, 100B, and 100C shown in FIGS. A specific description will be given with reference to the contents of each record of the customer service event information DB shown. In addition, when duplicate customer service event ID records are defined in the customer service event information DB shown in FIG. 11 to FIG. 14, duplicate explanation will be omitted and different contents will be explained.

(Service Event Detection Processing in the Service Monitoring System 100 shown in FIG. 2)
First, the customer service event detection unit 331 receives the monitoring data 4a from the customer service utterance evaluation unit 33 at predetermined time intervals (for example, about 10 seconds) in which the start time and the end time are determined (S8-1). The customer service event information DB (see FIG. 11) stored in the management DB 2a is read (S8-2).

The customer service event detection unit 331 acquires a record (service event ID “EID1”, customer service event name “accounting completion greeting”) in the first line of the customer service event information DB that has not been acquired (S8-3), and receives customer service event information. Start processing. The customer service event detection unit 331 acquires the POS operation history data as the verification target data from the monitoring data 4a (S11), and detects a customer event determination condition detection trigger for the transaction completion operation from the POS operation history data (that is, for the POS terminal 5). It is verified whether or not a predetermined operation (operation) has been satisfied (S12).

If the detection trigger of the customer service event determination condition for the transaction completion operation is not satisfied from the POS operation history data (NO in S13), the customer service event information processing shown in FIG. 9 is terminated, and the service of the customer service event detection unit 331 performs step S8. Proceed to -5.

On the other hand, when the detection trigger of the customer service event determination condition for the transaction completion operation is satisfied from the POS operation history data (S13, YES), the customer service event detection unit 331 receives the customer service event output information ( Specifically, the corresponding preset ID (corresponding one of 1 to PN), customer service ID (identification information of store clerk: corresponding to 1 to EN), customer service event ID) are stored in the customer service status DB (FIG. 10). (Refer to) is stored (held) (S14).

In the detected customer service event, the identification information (customer ID: 1 to EN) of the customer (store clerk) who operated the corresponding POS terminal 5 is, for example, a bar printed on the name tag when the operation of the POS terminal 5 is started. The customer ID obtained by reading the code with the barcode reader is used.

In the customer service monitoring system 100 shown in FIG. 2, the directivity of the microphone devices M1,..., ML is predetermined from the time of manufacture, and the directivity cannot be changed. That is, in the customer service monitoring system 100 shown in FIG. 2, since the voice is not collected by the microphone array devices AM1,..., AML (S15, NO), directivity formation processing is impossible, and the customer service event detection unit 331 is Then, voice data of the clerk corresponding to the detected customer service event is acquired from the monitoring data 4a (S16), and the voice data and customer service event ID are input to the customer service utterance evaluation value calculation unit 332 (S18). The clerk's voice data corresponding to the detected customer service event is, for example, voice data collected by the microphone device associated with the POS terminal 5 in which the customer service event “accounting completion operation” is detected.

The customer service utterance evaluation value calculation unit 332 executes the customer service utterance evaluation process shown in FIG. 15 (S19), and stores (holds) the service utterance evaluation output value in the customer service status DB (S20). Thereby, the service event information processing shown in FIG. 9 ends. In FIG. 8, after step S8-4, if not all records in the customer service event information DB have been acquired (S8-5, NO), the process of the customer service event detector 331 returns to step S8-3. On the other hand, when all the records of the customer service event information DB have been acquired (S8-5, YES), the process of the customer service event detector 331 ends.

(Service event detection process in the service monitoring system 100A shown in FIG. 3)
After step S8-2, the customer service event detecting unit 331 acquires a record (service customer event ID “EID2”, customer service event name “greeting in store”) in the second line of the customer service event information DB that has not been acquired (S8 -3) Start customer service event information processing. The customer service event detection unit 331 acquires all voice data collected by the customer microphone device as verification target data from the monitoring data 4a (S11), performs voice recognition processing on all the voice data, and identifies the result. It is verified whether or not the keyword “I welcome you” is included (S12).

If the voice recognition process result of all the voice data does not include the specific keyword “Welcome” (S13, NO), the customer service event information processing shown in FIG. Return to S8-5.

On the other hand, when the specific keyword “welcome” is included in any voice recognition processing result of the voice data (S13, YES), the customer service event detection unit 331 outputs a customer service event in the customer service event information DB shown in FIG. Information (specifically, customer service ID (store employee identification information: one corresponding to 1 to EN), customer service event ID) is stored (held) in the customer service DB (see FIG. 10) (S14).

In the customer service monitoring system 100A shown in FIG. 3, the directivities of the customer microphone devices SM1,..., SML are predetermined from the time of manufacture, and the directivity cannot be changed. That is, in the service monitoring system 100A shown in FIG. 3, since voice is not picked up by the microphone array devices AM1,..., AML (S15, NO), directivity formation processing is impossible, and the service event detection unit 331 Then, voice data of the clerk corresponding to the detected customer service event is acquired from the monitoring data 4a (S16), and the voice data and customer service event ID are input to the customer service utterance evaluation value calculation unit 332 (S18). The salesclerk's voice data corresponding to the detected customer service event is voice data that includes the specific keyword “welcome” in the voice recognition processing result in the customer service event detection unit 331.

(Service event detection process in the service monitoring system 100B shown in FIG. 4)
After step S8-2, the customer service event detection unit 331 acquires a record (service event ID “EID1”, customer service event name “entrance / exit greeting”) in the first line of the service event information DB that has not been acquired ( S8-3), customer service event information processing is started. In addition, since the customer service event of this “greeting entrance / exit” is a customer service event for all customers (all store clerks), a microphone ID and a camera ID capable of grasping all store clerk are output as customer service event output information. The same applies hereinafter. The customer service event detection unit 331 acquires video data as verification target data and detection results (automatic door opening / closing history data) included in the sensor data from the monitoring data 4b (S11), and the automatic door opening / closing history data includes the automatic door opening / closing history data. It is checked whether or not there is an opening / closing operation (S12).

If the automatic door opening / closing history data does not include an automatic door opening / closing operation (S13, NO), the customer service event information processing shown in FIG. 9 ends, and the process of the customer service event detection unit 331 returns to step S8-5.

On the other hand, when the automatic door opening / closing history data includes an automatic door opening / closing operation (S13, YES), the customer service event detection unit 331 receives the service event output information (specifically, the customer service event information DB shown in FIG. 13). Microphone ID (identifying information of microphone device: 1 to MN), camera ID (identifying information of camera device: 1 to CN), customer service ID (clerk identification information: 1 to EN) And the customer service event ID) are stored (held) in the customer service status DB (see FIG. 10) (S14). The camera ID is output as identification information of a camera device that captures the position where each store clerk is closest when the automatic door opening / closing operation is performed by the customer service event detection unit 331 performing image processing on predetermined video data.

In the customer service monitoring system 100B shown in FIG. 4, the directivity of the microphone devices M1... ML is predetermined from the time of manufacture, and the directivity cannot be changed. That is, in the customer service monitoring system 100B shown in FIG. 4, since the voice is not collected by the microphone array devices AM1,..., AML (S15, NO), the directivity forming process is impossible, and the customer service event detecting unit 331 is Then, voice data of each clerk corresponding to the detected customer service event is acquired from the monitoring data 4b (S16), and the voice data and customer service event ID are input to the customer service utterance evaluation value calculation unit 332 (S18).

The voice data of each store clerk corresponding to the detected customer service event is determined to be closest to the position where each store clerk was present during the automatic door opening / closing operation by image processing the predetermined video data by the customer service event detection unit 331. Voice data collected by the microphone device. From the microphone device, a microphone ID as the identification information is output.

The predetermined video data is, for example, video data or a combination of a plurality of video data captured by at least one camera device necessary for understanding the entire area in the store, and the corresponding camera device is fixed. However, it may be changed as appropriate according to the user's input operation, and so on.

The customer service utterance evaluation value calculation unit 332 executes the customer service utterance evaluation process shown in FIG. 15 (S19), and stores (holds) the service utterance evaluation output value in the customer service status DB (S20). Thereby, the service event information processing shown in FIG. 9 ends. In FIG. 8, after step S8-4, if not all records in the customer service event information DB have been acquired (S8-5, NO), the process of the customer service event detector 331 returns to step S8-3.

Accordingly, the customer service event detection unit 331 acquires the second line record (customer service event ID “EID2”, customer service event name “accounting start greeting”) of the customer service event information DB that has not been acquired (S8-3). Start event information processing. The customer service event detection unit 331 acquires the predetermined video data as the verification target data from the monitoring data 4b (S11), and performs image processing on the video data so that the store clerk performs a “accounting start greeting”. It exists in a position (for example, the standing position of the cashier counter), and the customer (visitor) is in a predetermined position (for example, a predetermined standby position in front of the cashier counter or in the store) for a predetermined period (for example, about 5 seconds) ) It is verified whether or not the stay has been made (S12). By determining whether or not the customer has stayed for a predetermined period or longer, it is possible to exclude the case where the customer passes through the cashier counter.

As a result of the image processing of the video data, the store clerk does not exist at a predetermined position (for example, the operation standing position of the cashier counter) for performing “accounting start greeting”, or the customer (visitor) has a predetermined position (for example, before the cashier counter) Alternatively, if it is determined that the user has not stayed for a predetermined period (for example, about 5 seconds) at a predetermined standby position in the store (S13, NO), the customer service event information processing shown in FIG. 9 ends. Then, the process of the customer service event detection unit 331 returns to step S8-5.

On the other hand, as a result of image processing of the video data, the store clerk is present at a predetermined position (for example, an operation standing position of the cashier counter) for performing “accounting start greeting”, and a customer (visitor) has a predetermined position (for example, cashier counter). If it is determined that the user has stayed for a predetermined period of time (for example, about 5 seconds) at a predetermined standby position in front of or in the store) (S13, YES), the customer service event detection unit 331 is shown in FIG. Customer service event output information in the customer service event information DB (specifically, a preset ID (identifying information of a predetermined position: applicable to 1 to PN) and a customer service ID (identifying information of a clerk: 1 to EN) Stuff) and customer service event ID) are stored (held) in the customer service situation DB (see FIG. 10) (S14).

In the customer service monitoring system 100B shown in FIG. 4, the directivity of the microphone devices M1... ML is predetermined from the time of manufacture, and the directivity cannot be changed. That is, in the customer service monitoring system 100B shown in FIG. 4, since the voice is not collected by the microphone array devices AM1,..., AML (S15, NO), the directivity forming process is impossible, and the customer service event detecting unit 331 is Then, voice data of the store clerk corresponding to the detected customer service event is acquired from the monitoring data 4b (S16), and the voice data and customer service event ID are input to the customer service utterance evaluation value calculation unit 332 (S18). The clerk's voice data corresponding to the detected customer service event is the clerk's voice data collected by the microphone device associated with the predetermined position (preset position).

(Service Event Detection Processing in the Service Monitoring System 100C shown in FIG. 5)
After step S8-2, the customer service event detection unit 331 acquires a record (service event ID “EID1”, customer service event name “entrance / exit greeting”) in the first line of the service event information DB that has not been acquired ( S8-3), customer service event information processing is started. The customer service event detection unit 331 acquires video data as verification target data and detection results (automatic door opening / closing history data) included in the sensor data from the monitoring data 4b (S11), and the automatic door opening / closing history data includes the automatic door opening / closing history data. It is checked whether or not there is an opening / closing operation (S12).

On the other hand, when the automatic door opening / closing history data includes an automatic door opening / closing operation (S13, YES), the customer service event detection unit 331 receives the service event output information (specifically, the customer service event information DB shown in FIG. 13). Each store employee's customer service location coordinates, camera ID (camera device identification information: 1 to CN), customer service ID (store employee identification: 1 to EN), customer service event ID) Is stored (held) in the customer service situation DB (see FIG. 10) (S14).

The customer service position coordinates are obtained by image processing of predetermined video data by the customer service event detection unit 331, and are output as the coordinates of the position where each store clerk exists in the video data displayed on the screen of the display device 35. . The camera ID is output as identification information of a camera device that images the position where each store clerk was closest when the automatic door opening / closing operation is performed by the customer service event detection unit 331 performing image processing on predetermined video data. . Since the microphone ID is associated with the camera ID in advance, the microphone ID is selected and output when the camera ID is selected.

In the customer service monitoring system 100C shown in FIG. 5, sound is collected by the microphone array devices AM1,..., AML (S15, YES). The unit 331 inputs, to the directivity control unit 37, the data on the customer position coordinates of each store clerk corresponding to the detected customer service event and the voice data of each store clerk corresponding to the customer service event included in the monitoring data 4b. The directivity control unit 37 acquires voice data after directivity is formed in the direction from the microphone array device closest to each store clerk toward each store clerk with respect to the voice data of each store clerk (S17). The customer service event detection unit 331 obtains in the service customer utterance evaluation value calculation unit 332 the data on the customer service staff position coordinates (for example, the coordinates of the position of the sales staff displayed on the screen of the display device 35) and the data in step S17. The received voice data and customer service event ID are input (S18).

Next, an example of a customer service situation DB including customer service event output information (see FIGS. 11 to 14) output as a result of the customer service event detection process shown in FIG. 8 will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of the customer service situation DB.

The customer service status DB shown in FIG. 10 includes a customer service status data ID, a customer service utterance evaluation value, an event start time, an event end time, a customer service ID, a customer service event ID, a customer service location (preset), and a customer service. Data corresponding to each item of the person position (outside the preset) is defined.

In the customer service status data ID “ID1”, the customer service utterance evaluation value is V11... V1n, and the customer service position is detected because the customer service (store clerk) is not at the default position (preset position). And the coordinates indicating the position of the sales clerk on the video data displayed on the screen of the display device 35 (coordinate position on the screen). The camera device with the camera ID “C1” may be an omnidirectional camera device, a camera device having a fixed angle of view, or a PTZ camera device having a pan / tilt / zoom function.

Further, “1” as the subscript on the left side of the customer service utterance evaluation value V11 corresponds to the customer service event ID “EID1”, and “1” as the subscript on the right side of the customer service utterance evaluation value V11 is set at every predetermined time interval. The customer service event identification information when a customer service event with the same customer service event ID is detected in the monitoring data 4a and 4b cut out at the same time. n is an integer of 1 or more. For example, when a plurality of customer service events having the same customer service ID are detected in the monitoring data 4a and 4b of about 10 seconds, n is an integer of 2 or more.

In the customer service status data ID “ID2”, the customer service utterance evaluation value is V21... V2m, and m is an integer of 1 or more as in the case of n described above. The customer service position is configured by a preset ID indicating the default position (preset position) since it is detected that the customer (store clerk) is present at the default position (preset position).

Accordingly, the customer

service monitoring systems

100, 100A, 100B, and 100C shown in FIGS. 2 to 5 have the customer service situation DB shown in FIG. 10 as described above, so that the voice data from the start time to the end time of the corresponding customer service event. And video data can be output (reproduced) by the customer service evaluation device 3, and a store manager (for example, a store manager) can carefully observe and review the customer service status of the store clerk at the customer service event with sound and video. it can. Since the voice data is stored (stored) in the recorder device 4, the customer service evaluation device 3 obtains the voice data collected when the customer service event corresponding to the customer service event ID is detected from the recorder device 4. Output (play).

Next, details of the operation procedure of the customer service utterance evaluation process (see step S19) shown in FIG. 9 will be described with reference to FIG. FIG. 15 is a flowchart for explaining an example of the operation procedure of the customer service utterance evaluation process.

In FIG. 15, the customer service utterance evaluation value calculation unit 332 acquires the voice data and customer service event ID passed from the customer service event detection unit 331 in step S18 (S21), performs a noise level determination process (S22), and receives the customer service. A keyword utterance determination process is performed (S23). After step S23, the customer service utterance evaluation value calculation unit 332 determines whether or not the flag of the detection state (see later) is “1” (S24). When the flag of the detection state is “1” (S24, YES), the customer service utterance evaluation value calculation unit 332 performs a scoring process (S25). On the other hand, when the flag of the detection state is not “1” (S24, NO), the service utterance evaluation value calculation unit 332 sets the service utterance evaluation value to a zero point or deducts a predetermined score (S26).

After step S25 or step S26, the service utterance evaluation value calculation unit 332 outputs the detected keyword ID (described later) and the service utterance evaluation value as scoring data to the service utterance evaluation unit 33 (S27).

Next, details of the operation procedure of the noise level determination process (see step S22) shown in FIG. 15 will be described with reference to FIG. FIG. 16 is a flowchart for explaining an example of the operation procedure of the noise level determination process.

In FIG. 16, the customer service utterance evaluation value calculation unit 332 determines whether or not the noise level around the sound collection area (for example, a store) acquired by the customer service evaluation device 3 is equal to or less than a predetermined value x [dB] (S22). -1). The noise level is collected by, for example, any one of a microphone device, a customer service microphone device, and a microphone array device, and is transmitted to the customer service evaluation device 3. When the customer service utterance evaluation value calculation unit 332 determines that the noise level is equal to or less than the predetermined value x [dB] (S22-1, YES), the utterance determination threshold (see later) is determined to be α1 (S22-). 2).

On the other hand, if the noise level exceeds the predetermined value x [dB] (S22-1: NO), the customer service utterance evaluation value calculation unit 332 determines whether the noise level is equal to or lower than the predetermined value y (> x) [dB]. It is determined whether or not (S22-3). If the customer service utterance evaluation value calculation unit 332 determines that the noise level is equal to or lower than the predetermined value y [dB] (S22-3, YES), it determines the utterance determination threshold (see later) to α2 (S22-). 4). On the other hand, when it is determined that the noise level exceeds the predetermined value y [dB] (S22-3, NO), the customer service utterance evaluation value calculation unit 332 determines α3 as the utterance determination threshold (see below) (S22). -5).

Next, details of the operation procedure of the service keyword utterance determination process (see step S23) shown in FIG. 15 will be described with reference to FIG. FIG. 17 is a flowchart for explaining an example of the operation procedure of the service keyword utterance determination process.

In FIG. 17, the customer service utterance evaluation value calculation unit 332 sets the detection state flag to “0” (S23-1). The flag of the detection state is information indicating a state in which an exemplary utterance assumption keyword (see FIG. 22A) that is likely to be uttered by a store clerk in a customer service event or to be uttered is uttered.

The service utterance evaluation value calculation unit 332 inputs the voice data acquired in step S21 to the speech recognition engine of the service utterance evaluation value calculation unit 332 (S23-2), and further, all utterance assumption keywords corresponding to the service event ID And a keyword ID for identifying each utterance assumed keyword is acquired from the customer service utterance evaluation DB of the management DB 2a of the management server 2 (S23-3).

The customer service utterance evaluation value calculation unit 332 determines whether the speech utterance keyword acquired in step S23-3 is included in the speech recognition result by the speech recognition engine (S23-4). If it is determined that the speech recognition keyword is not included in the speech recognition result by the speech recognition engine (S23-4, NO), the processing of the customer utterance evaluation value calculation unit 332 shown in FIG. 17 ends.

On the other hand, if the customer service utterance evaluation value calculation unit 332 determines that the speech recognition result obtained by the speech recognition engine includes the speech utterance keyword acquired in step S23-3 (S23-4, YES), the speech recognition It is determined whether or not the evaluation value of the processing result is greater than or equal to the utterance determination threshold (any one of α1, α2, and α3) determined in step S22-2, step S22-4, or step S22-5 (S23-). 5). When it is determined that the evaluation value of the speech recognition processing result is less than the utterance determination threshold value (any of α1, α2, and α3) (S23-5, NO), the customer utterance evaluation value calculation unit shown in FIG. The processing at 332 ends.

On the other hand, when the customer service utterance evaluation value calculation unit 332 determines that the evaluation value of the speech recognition processing result is equal to or greater than the utterance determination threshold (any one of α1, α2, and α3) (S23-5, YES), The detection state flag is changed to “1” (S23-6), and the voice data acquired in step S21 is cut only to the utterance portion of the keyword corresponding to the utterance assumed keyword, and is updated and saved (S23-). 7). Even if there are extra noises before and after the utterance part, by cutting out only the utterance part of the keyword, the noise sound before and after it will be cut, which improves the accuracy of voice recognition. The accuracy of the scoring process in the subsequent step S25 is also ensured.

Next, details of the operation procedure of the scoring process (see step S25) shown in FIG. 15 will be described with reference to FIG. FIG. 18 is a flowchart for explaining an example of the operation procedure of the scoring process.

In FIG. 18, the customer service utterance evaluation value calculation unit 332 performs utterance length determination processing using the voice data updated in step S23-7 (S25-1), and further performs frequency characteristic determination processing (S25). -2). Further, the customer service utterance evaluation value calculation unit 332 obtains scoring data (specifically, the keyword detected from the voice data updated in step S23-7) as a result of the utterance length determination process and the frequency characteristic determination process. A set of a keyword ID for identifying the same utterance assumed keyword and a customer service utterance evaluation value) is stored in the memory 32 (S25-3).

Next, details of the operation procedure of the speech length determination process (see step S25-1) and the frequency characteristic determination process (see step S25-2) shown in FIG. 18 will be described with reference to FIGS. 19A and 19B. FIG. 19A is a flowchart for explaining an example of an operation procedure of speech length determination processing. FIG. 19B is a flowchart illustrating an example of an operation procedure of a frequency characteristic determination process.

In FIG. 19A, the service utterance evaluation value calculation unit 332 refers to the service utterance evaluation DB of the management DB 2a of the management server 2, and model voice data specified by the service utterance model ID corresponding to the service event ID acquired in step S21. Is acquired from the management DB 2a of the management server 2 (S31). The model voice data is an example of keyword voice data including voice data of an utterance assumed keyword for each predetermined customer service event. The customer service utterance evaluation value calculation unit 332 determines whether or not the length of the voice data (for example, the clerk's utterance portion) updated in step S23-7 is within a typical predetermined range (S32).

FIG. 20 is a diagram showing a specific example of the utterance length determination process using the model voice data. In FIG. 20, the horizontal axis indicates time. For example, “Welcome” with an utterance length l0 of an exemplary predetermined range of “Welcome” uttered in a customer service event of “greeting in store”, and a predetermined range “I welcome you” (refer to No. 1 shown in FIG. 20) and “I welcome you” (refer to No. 2 shown in FIG. 20) having an utterance length l2.

The service utterance evaluation value calculation unit 332 determines that the utterance length of the voice data updated in step S23-7 exceeds a predetermined range (for example, 10%) from the utterance length of the model voice data (utterance length l0) ( For example, in No. 1 and No. 2 shown in FIG. 20 (S32, NO), a predetermined score is subtracted from the customer service utterance evaluation value (S34).

For example, No. shown in FIG. In the first case, the utterance length of the uttered “Ira-saisei” is shorter than the utterance length of “Ira-saisei” in the model voice data by a predetermined range. In this case, the service utterance evaluation value calculation unit 332 has a predetermined number of points. As a result, “100 × (0.910−11) / 10” is deducted. 11 corresponds to No. 11 shown in FIG. The utterance length of “I welcome you” uttered in case 1 is shown. More specifically, FIG. 20 shows that the utterance length of the model voice data “Irasementation” is 1 second and the predetermined range is ± 10% of the utterance length of the model voice data “Irasementation”. No. If the utterance length of “I welcome you” uttered in case 1 is between 0.9 seconds and 1.1 seconds, no deduction will be given, but if it is 0.7 seconds, for example, 20 points ( = 100 x (0.9 x 1 sec-0.7 sec)) is deducted.

Also, for example, as shown in FIG. In the case of 2, the utterance length of the uttered “Islaimase” is longer than the utterance length of the “speaking” of the model voice data by a predetermined range. In this case, the customer utterance evaluation value calculation unit 332 As a result, “100 × (l2−1.1l0) / l0” is deducted. l2 is No. 12 shown in FIG. The utterance length of “I welcome you” uttered in case 2 is shown. More specifically, FIG. 20 shows that the utterance length of the model voice data “Irasementation” is 1 second and the predetermined range is ± 10% of the utterance length of the model voice data “Irasementation”. No. If the utterance length of “I welcome you” uttered in case 2 is between 0.9 seconds and 1.1 seconds, no deduction will be given, but if it is 1.3 seconds, for example, 20 points ( = 100 × (1.3 seconds−1.1 × 1 seconds)) is deducted.

On the other hand, the service utterance evaluation value calculation unit 332 determines that the utterance length of the voice data updated in step S23-7 exceeds a predetermined range (for example, 10%) from the utterance length of the model voice data (utterance length l0). If not (S32, YES) or after step S34, the scoring data (specifically, the keyword ID for identifying the keyword detected in the voice data updated in step S23-7 and the points deducted in step S34) The initial value (for example, a set with the customer utterance evaluation value that is 100 points) that was not deducted after or in step S34 is held in the memory 32 (S33).

In FIG. 19B, the service utterance evaluation value calculation unit 332 refers to the service utterance evaluation DB of the management DB 2a of the management server 2, and model voice data specified by the service utterance model ID corresponding to the service event ID acquired in step S21. Is acquired from the management DB 2a of the management server 2 (S41). The service utterance evaluation value calculation unit 332 has a frequency characteristic (for example, frequency) for each phoneme (sound of each word) of the voice data updated in step S23-7 within a predetermined range from a basic frequency for each phoneme. It is determined whether it is within (S42).

FIG. 21 is a diagram showing a specific example of frequency characteristic determination processing using the fundamental frequency of each phoneme of the model voice data. In FIG. 21, the horizontal axis indicates time, the dotted circle indicates the fundamental frequencies f1 to f7 of each phoneme of the model speech data, and the solid circle indicates the frequency f of each phoneme of the speech data updated in step S23-7. '1 to f'7, for example, a predetermined range for each of the fundamental frequencies f1 to f7 of the phoneme of “I welcome you” uttered at the customer service event “greeting at the store” (each of the linear types shown in FIG. 21) (See solid arrows).

For each phoneme of the voice data updated in step S23-7, the service level evaluation value calculation unit 332 sets the frequency characteristics (for example, frequency) of each phoneme to a predetermined range (for example, 60) for each frequency characteristic of each phoneme of the model voice data. [Hz]) If determined to exceed (NO in S42), the predetermined score is subtracted from the customer utterance evaluation value according to the number of phonemes exceeding the predetermined range (S44).

For example, in the case shown in FIG. 21, since the frequencies f′1 and f′6 of the phoneme “ra” and the phoneme “ma” exceed the predetermined ranges for the corresponding fundamental frequencies f1 and f6, the customer utterance evaluation value calculation is performed. If the frequency difference (for example, | f1-f′1 |) is 60 [Hz] to 120 [Hz], the unit 332 deducts five points for each corresponding phoneme, and the frequency difference (for example, | f1-f′1). If |) exceeds 120 [Hz], 10 points are deducted for each corresponding phoneme. Depending on the region and the industry, it may be preferable that the utterance of the utterance assumed keyword is uttered in an upward tone (refer to the alternate long and short dash line shown in FIG. 21). For example, for a predetermined number of phonemes including the ending or the ending May be used by increasing the value of the fundamental frequency.

On the other hand, the service utterance evaluation value calculation unit 332 sets the frequency characteristics (for example, frequency) of each phoneme to a predetermined range (for each frequency characteristic of each phoneme of the model voice data) for each phoneme of the voice data updated in step S23-7. For example, when it is determined that the frequency does not exceed 60 [Hz] (S42, YES) or after step S44, it is detected in the scoring data (specifically, the audio data updated in step S23-7) The memory ID is stored in the memory 32 with the keyword ID for identifying the selected keyword and the customer utterance evaluation value, which is the initial value (for example, 100 points) that has been deducted in step S44 or not deducted in step S44 (S43).

FIG. 22A is a diagram showing an example of an utterance assumption keyword table constituting a part of the customer service utterance evaluation DB. FIG. 22B is a diagram illustrating an example of a list of service utterance models that constitute a part of the service utterance evaluation DB.

In the utterance assumption keyword table shown in FIG. 22A, data corresponding to each item of the service event ID, the service event name, the keyword ID, the utterance assumption keyword, and the service utterance model ID are defined. The keyword ID identifies an utterance assumption keyword. The customer service utterance model ID is associated with model voice data as shown in FIG. 22B. As shown in FIG. 22A, one or more utterance assumption keywords may be defined corresponding to one customer utterance model ID (see the record of customer service event ID “EID2” shown in FIG. 22A).

Next, the operation procedure of the browsing process or the correction process of the customer service status DB in the customer

service monitoring system

100, 100A, 100B, 100C of this embodiment will be described with reference to FIG. FIG. 23 is a flowchart illustrating an example of an operation procedure of a browsing process by a limited viewer or a service utterance evaluation value correction process.

In FIG. 23, for example, on the login screen WD1 (see FIG. 27) for the customer service status browsing screen displayed on the display device 35, the login ID is input by the input operation (for example, touch operation with the finger FG) of the customer who requests the customer service status DB. And the password are input, and the login button LGI is pressed (S51). Further, the customer service evaluation device 3 refers to the viewer DB (see FIG. 26A) of the management DB 2a of the management server 2 and inputs information (specifically, entered on the login screen WD1 of FIG. 27) by the browse requester. The access right, authority level, and password are acquired from (password) (S51). FIG. 26A is a diagram illustrating an example of a viewer DB. FIG. 26B is a diagram illustrating an example of a customer service DB.

In the viewer DB shown in FIG. 26A, the type and type of data for each item of the viewer ID, password, browser authority, and authority level are defined. There are two types of viewer authority defined: authority that allows both browsing and correction operations, and authority that allows only browsing operations. The password may be, for example, an actually input password or a hash value (digest) of the input password.

In the customer service DB shown in FIG. 26B, the data type and type for each item of the customer ID indicating the store clerk's identification information, the store ID indicating the store identification information, and the name of the store clerk are defined.

FIG. 27 is a diagram showing an example of a login screen to the customer service status DB to be browsed in the customer service monitoring system. The customer service evaluation device 3 determines whether or not the browsing requester input on the login screen in step S51 has the access right of the browsing operation defined in the browser DB and the passwords match (S52). If it is determined that there is no access right for the browsing operation or that the passwords do not match (S52, NO), the processing of the customer service evaluation device 3 shown in FIG. 23 ends.

On the other hand, when the customer service evaluation device 3 determines that the user has access rights for the browsing operation defined in the viewer DB and the passwords match (S52, YES), the customer service status of the management DB 2a of the management server 2 is determined. The DB is accessed, and for example, the total result of the customer service utterance evaluation values of all the store clerks for the customer (customer) per day is displayed on the display device 35 as a customer service status display screen WD2 (see FIG. 28) (S53).

FIG. 28 is a diagram illustrating an example of a totaling result of customer service utterance evaluation values of all customer service customers per day as a customer service status display screen WD2. In FIG. 28, the number of customer service events detected for a customer who visited the store is 255, and among the customer service events of the store entrance greeting, the number of utterances of the utterance assumed keyword of “I welcome you” is 230. The ratio compared to the total number of 255 is 90%, and the number of utterances of the “Thank you” utterance assumed keyword was executed in the customer service event of closing greetings, and the ratio compared to the total number of 255 was 76 %.

Also, in FIG. 28, the number of customer service events related to cashier reception detected for customers who visited the store is 180, of which the point card confirmation omission (that is, the customer service event for prompting the presentation of the point card is not detected). 8), the ratio compared to the total number 180 is 4%, and the point card confirmation of the store clerk confirms that the presentation rate of the customer's point card is 10%. It is shown that the number of missed warming confirmations (that is, a customer service event for confirming that the lunch box is warmed in the microwave oven was not detected) was 3, and 2% of the total number was 180%. ing.

It should be noted that the output unit 34 of the customer service evaluation apparatus 3 aggregates and redisplays the data of each item shown in FIG. 28 by switching to a week or month instead of a day according to a predetermined input operation. May be.

In the customer service status display screen WD2 shown in FIG. 28, when the logout button LGO is selected by the touch operation with the finger FG of the user (who has the authority level of the browsing operation) (S54, YES), the customer service evaluation device 3 The output unit 34 closes (closes) all the browsing screens displayed on the display device 35 (S55). On the other hand, if the logout button LGO is not selected (S54, NO), the detail display button IDT on the customer service status display screen WD2 shown in FIG. 28 is selected, and the corresponding access right (authority level L1 that allows a correction operation). When it is determined that there is a service (S56, YES), the output unit 34 of the customer service evaluation device 3 switches from the customer service status display screen WD2 shown in FIG. 28 to the detailed display screen WD7 shown in FIG. It is displayed (S57). After step S57, the customer service evaluation device 3 performs a service utterance evaluation value correction process (S58). FIG. 32 is a diagram illustrating a specific example of each record displayed on the detail display screen WD7 of the customer service situation DB.

On the other hand, the detail display button IDT on the customer service status display screen WD2 is not selected and there is no corresponding access right, or after step S58, the process of the customer service evaluation device 3 shown in FIG. 23 returns to step S54.

Next, details of the customer service utterance evaluation value correction process (see step S58) shown in FIG. 23 will be described with reference to FIGS. FIG. 24 is a flowchart for explaining an example of a detailed operation procedure of the service utterance evaluation value correction process. FIG. 25 is a flowchart for explaining the continuation of the detailed operation procedure of the modification process of the customer service utterance evaluation value shown in FIG.

In FIG. 24, in the state where the detailed display screen WD7 shown in FIG. 32 is displayed on the display device 35, for example, the record RC1 of the customer service situation data ID (see FIG. 33) that the user (correction requester) wants to correct is the user's finger FG. (S58-1). FIG. 33 is a diagram illustrating an example of a modification operation for the customer utterance evaluation value of the specific record RC1 displayed on the detail display screen WD7 of the customer service situation DB.

The customer service utterance evaluation unit 33 of the customer service evaluation device 3 accesses the customer service status DB of the management DB 2a of the management server 2 via the output unit 34, and the event start time corresponding to the customer service event ID specified in step S58-1. The event end time is extracted, and video data and audio data corresponding to the event start time and event end time are acquired from the recorder device 4 and passed to the output unit 34 (S58-2).

Further, the output unit 34 switches from the detailed display screen WD7 to the customer service situation preview screen WD8 shown in FIG. 35 to display on the display device 35, and outputs (reproduces) the acquired video data on the display device 35, while outputting the audio data. Output from the speaker device 36 (S58-2). FIG. 35 is a diagram illustrating an example of a customer service position correcting operation on the customer service status preview screen.

If the customer service monitoring system 100C shown in FIG. 5 is used, the customer service utterance evaluation unit 33 acquires the customer service location data corresponding to the customer service status data ID from the customer service status DB, and sends it to the directivity control unit 37 together with the voice data. hand over. The directivity control unit 37 uses the voice data and the customer service location data to transmit the voice in the direction from the microphone array device that picks up the customer ID of the customer service ID corresponding to the customer service status data ID to the store employee closest. Directivity is formed and passed to the output unit 34 (S58-2).

After step S58-2, the customer service utterance evaluation unit 33 is in an active state so that the position change button of the customer service status preview screen WD8 shown in FIG. 35 can be selected, and the customer service status preview is performed by the user's (correction requester) finger FG. It is determined whether or not the stop button on the screen WD8 is selected and there is an access right (authority level L1 at which correction operation is possible) corresponding to the user (correction requester) (S58-3). In the customer

service monitoring systems

100, 100A and 100B shown in FIGS. 2 to 4 except for the customer service monitoring system 100C shown in FIG. 5, the microphone array device is not used, so the position change button on the customer service situation preview screen WD8 shown in FIG. Is in a non-selectable state (inactive state).

When the position change button of the customer service situation preview screen WD8 is in an inactive state, when the stop button of the customer service situation preview screen WD8 is not selected by the finger FG of the user (correction requester) or access corresponding to the user (correction requester) If there is no right, the process proceeds to step S58-8 (see FIG. 25).

On the other hand, the position change button of the customer service status preview screen WD8 is in an active state that can be selected, and the stop button of the customer service status preview screen WD8 is selected by the user's (correction requester) finger FG, and the corresponding access right (correction) If it is determined that there is an authority level L1 that can be operated (S58-3, YES), the pointing direction is designated by the user (correction requester) finger FG on the customer service status preview screen WD8 ( In step S58-4, the directivity control unit 37 changes the directivity direction for forming the sound directivity in the direction specified in step S58-4 (S58-5). The output unit 34 causes the speaker device 36 to output the voice data after the directivity formation with the directivity direction changed. This voice data is confirmed by the user (correction requester) (S58-5).

When the position change button on the customer service status preview screen WD8 is selected by the user's (correction requester) finger FG (S58-6, YES), the customer service utterance evaluation unit 33 displays the customer on the detail display screen WD7 shown in FIG. The information is changed to the coordinates indicating the position in the pointing direction designated in step S58-4 (that is, the coordinates indicating the position on the screen displayed on the display device 35) and displayed on the display device 35. The customer position of the corresponding record in the situation DB is changed (corrected) and overwritten and stored (S58-7, see FIG. 36). FIG. 36 is a diagram illustrating an example of coordinates of the customer service position after correction of a specific record displayed on the detail display screen of the customer service DB. In FIG. 36, for example, a cell CL2 of an arbitrary coordinate (that is, a coordinate indicating a designated position on the screen of the display device 35 in which the directing direction is changed) of the customer service position (outside the preset) of the record of the customer service status data ID “4” Is shown to have changed.

On the other hand, when the position change button on the customer service status preview screen WD8 is not selected with the finger FG of the user (correction requester) (S58-6, NO), the customer service utterance evaluation value correction process shown in FIG. Return to.

After step S58-7, in FIG. 25, the customer service utterance evaluation unit 33 receives the customer selected by the finger FG of the user (correction requester) on the detail display screen WD7 (see FIG. 32) displayed by the display device 35. Whether or not the service utterance evaluation value cell CL1 (see FIG. 34) of the record of the situation data ID (see step S58-1) is double-tapped and there is a corresponding access right (authorization level L1 that can be modified). Is determined (S58-8). FIG. 34 is a diagram illustrating an example of a customer utterance evaluation value after correction of a specific record displayed on the detail display screen WD7 of the customer service situation DB.

When the cell CL1 of the customer utterance evaluation value of the record of the customer service status data ID selected by the finger FG of the user (correction requester) on the detailed display screen WD7 is not double-tapped, or when there is no corresponding access right ( S58-8, NO), the service utterance evaluation value correction process shown in FIG. 25 is terminated, and the process of the service evaluation apparatus 3 returns to step S54.

On the other hand, the cell CL1 of the customer utterance evaluation value of the record of the customer service status data ID (see step S58-1) selected by the user (correction requester) finger FG on the detailed display screen WD7 (see FIG. 32) is double-tapped. And when it is determined that there is a corresponding access right (authority level L1 at which a correction operation can be performed) (YES in S58-8), for example, the operation is performed by using the finger FG of the correction requester to double tap. If the service utterance evaluation value of the cell CL1 has been corrected (changed) (S58-9, YES), the service utterance evaluation unit 33 receives the corrected (changed) service utterance evaluation value (see FIG. 34). The situation DB is overwritten and saved (stored) (S58-10).

If the service utterance evaluation value of the double-tapped cell CL1 is not corrected (changed) by an operation using the finger FG of the correction requester (S58-9, NO), the service utterance evaluation value shown in FIG. The correction process ends, and the process of the customer service evaluation device 3 returns to step S54.

FIG. 29 is a diagram showing an example of a totaling result of customer utterance evaluation values of all customers with respect to customers in a day time zone. FIG. 30A is a diagram illustrating an example of a totaling result of customer utterance evaluation values of one customer for each time slot of one day. FIG. 30B is a diagram illustrating an example of a total result of customer service utterance evaluation values for each customer service per day. FIG. 31 is a diagram illustrating an example of a totaling result of customer service utterance evaluation values for each store on a day.

Also, the output unit 34 of the customer service evaluation device 3 is shown in FIG. 29 from the customer service status display screen WD2 shown in FIG. 28 by a predetermined input operation of the user (that is, a person having at least the authority level L1 shown in FIG. 26A). It may be switched to the customer service status display screen WD3 and displayed on the display device 35. In FIG. 29, the horizontal axis shows the time of the day, and the vertical axis shows the number of customers (customers) (see black bars) and the number of salesclerks who could greet the customers (in other words, , The number of customer service event names “sales greetings” properly executed by store clerk, see white bars).

Further, the output unit 34 of the customer service evaluation device 3 is shown in FIG. 30A from the customer service status display screen WD3 shown in FIG. 29 by a predetermined input operation of the user (that is, a person having at least the authority level L1 shown in FIG. 26A). In this way, the service status display screen WD4 limited to a specific time zone may be switched to display on the display device 35. In FIG. 30A, the horizontal axis indicates the time of the day, and the vertical axis indicates the number of customers (see black bars) and the number of cases that the store clerk could greet the customer (in other words, , The number of customer service event names “sales greetings” properly executed by store clerk, see white bars).

Further, the output unit 34 of the customer service evaluation device 3 is shown in FIG. 30B from the customer service status display screen WD2 shown in FIG. 28 by a predetermined input operation of the user (that is, a person having at least the authority level L1 shown in FIG. 26A). It may be switched to the customer service status display screen WD5 and displayed on the display device 35. In FIG. 30B, the greeting rate, the average score, and the number of cashiers corresponding to each store clerk (for example, four people) are shown in comparison on a daily basis. It should be noted that the output unit 34 of the customer service evaluation device 3 aggregates and redisplays the data of each item shown in FIG. 30B by switching to a weekly or monthly unit instead of a daily unit according to a predetermined input operation. May be.

Further, the output unit 34 of the customer service evaluation device 3 is shown in FIG. 31 from the customer service status display screen WD2 shown in FIG. 28 by a predetermined input operation of a user (that is, a person having at least the authority level L1 shown in FIG. 26A). It may be switched to the customer service status display screen WD6 and displayed on the display device 35. In FIG. 31, the number of customers visiting the store, the greeting rate, the average score, and the number of cashiers corresponding to each store (for example, four) are shown in comparison on a daily basis. Note that the output unit 34 of the customer service evaluation device 3 switches the data for each item shown in FIG. 31 and redisplays the data by switching to a week or month instead of a day according to a predetermined input operation. May be.

As described above, the customer

service monitoring systems

100, 100A, 100B, and 100C of the present embodiment have the customer service event information DB (customer service event data) including customer service event determination conditions for each predetermined customer service event and the POS terminal 5 (predetermined business terminal). Based on the POS operation history data indicating the operation history of the POS terminal 5 of the clerk (employee) for the clerk, the customer service event of the clerk is detected, and based on the voice data of the clerk included in the monitoring data 4a or the monitoring data 4b, A service utterance evaluation value corresponding to a predetermined utterance keyword when the POS terminal 5 is operated is calculated. In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C determine the calculated customer service utterance evaluation values based on the customer service ID (sales employee identification information), the customer service position (sales staff service location), and the customer service time. Are stored in association with the audio data.

As a result, the customer

service monitoring systems

100, 100A, 100B, and 100C can protect the customer's customer's privacy extensively without using human resources like an investigator, as in the prior art. By monitoring the customer utterances of the corresponding customer (employee) during various customer service events for customers in the sound domain (for example, stores), the objectivity of the customer utterance content of the customer can be obtained as a customer utterance evaluation value. It is possible to accurately and objectively evaluate the customer service situation of employees.

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C have data (that is, privacy protection marks) indicating predetermined information indicating customer privacy protection in employee voice data stored in the recorder device 4 as the second storage unit. ) Is omitted, the detection of employee service events is omitted, so that customer privacy is more clearly protected when customer service events are detected by excluding customer service events that involve customers. can do.

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C further store in the management DB 2a of the management server 2 the utterance assumption keyword table (keyword data) of the customer utterance evaluation DB including the utterance assumption keywords for each predetermined customer service event. If the speech utterance keyword corresponding to the customer service event is not included in the clerk's voice data, the customer service utterance evaluation value is set to zero or a predetermined score is deducted from the customer service utterance evaluation value. It is possible to accurately evaluate the customer service situation for a store clerk who does not speak.

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C further store in the management DB 2a of the management server 2 the utterance assumption keyword table (keyword data) of the customer utterance evaluation DB including the utterance assumption keywords for each predetermined customer service event. If the store clerk's voice data includes an utterance assumption keyword corresponding to a customer service event, the clerk's voice data is cut out only to the utterance part of the keyword corresponding to the utterance assumption keyword, and is overwritten and saved, so unnecessary noise The scoring process accuracy can be improved by cutting the sound, and the volume of the clerk's voice data can be reduced. Furthermore, it can contribute to the calculation of an accurate customer utterance evaluation value.

Further, the customer

service monitoring systems

100, 100A, 100B, and 100C use the customer service utterance model list (keyword speech data) of the customer service utterance evaluation DB including the speech data of the utterance assumed keywords for each predetermined customer service event as a management DB 2a of the management server 2. In the case where the utterance length of the utterance assumption keyword of the clerk's voice data that has been stored and updated exceeds the utterance length of the utterance assumption keyword of the keyword voice data, the predetermined score is subtracted from the customer utterance evaluation value Therefore, it is possible to accurately evaluate the customer service situation for a store clerk who utters an utterance assumption keyword that deviates from an exemplary utterance length at the customer service event.

Further, the customer

service monitoring systems

100, 100A, 100B, and 100C use the customer service utterance model list (keyword speech data) of the customer service utterance evaluation DB including the speech data of the utterance assumed keywords for each predetermined customer service event as a management DB 2a of the management server 2. In the case where the frequency for each phoneme of the utterance assumption keyword of the clerk's speech data updated and exceeds the predetermined range from the basic frequency for each phoneme of the utterance assumption keyword of the keyword speech data, a predetermined score is obtained from the customer utterance evaluation value. Since the points are deducted, it is possible to accurately evaluate the customer service situation with respect to the store clerk who utters by deviating from the typical basic frequency at the customer service event.

Further, the customer service monitoring system 100A detects the customer service event of the store clerk based on the voice data and customer service event data of each store clerk picked up by the customer microphone devices SM1,..., SML individually attached to the store clerk. Compared with the case where the store clerk is far away from the microphone device other than the customer microphone device (for example, a microphone installed on the ceiling surface), the store clerk's voice can be clearly picked up. A customer service event can be accurately detected.

In addition, the customer

service monitoring systems

100B and 100C further store in the recorder device 4 video data at a predetermined position in a predetermined sound collection area (for example, a store) obtained by imaging of the camera devices C1, ..., CM as monitoring data 4b. Since the customer service event of the store clerk is detected based on this video data, the video data having a predetermined position near the POS terminal 5 where the customer service event of the accounting start greeting of a predetermined sound collection area (for example, store) is performed is processed. By doing so, it is possible to accurately evaluate whether or not the customer service event of the accounting start greeting is properly performed.

Moreover, the customer

service monitoring systems

100B and 100C monitor the detection results of the appearance or exit of a customer (for example, the customer entering or leaving the store) in a predetermined sound collection area obtained by the sensor devices S1,. Further, the data is recorded in the recorder device 4 as data 4b, and a customer service event of a store clerk is detected based on the detection result. Therefore, a sensor that gives a motivation for a customer service event for entering and leaving a store in a predetermined sound collection area (for example, a store) Depending on the detection result of the device (for example, an automatic door that opens and closes), it is possible to accurately evaluate whether or not the customer service event for entering and leaving the store is properly performed.

Further, the customer service monitoring system 100C has a voice directivity from any one of the microphone array devices AM1,..., AML to a predetermined directivity direction (for example, a fixed position (for example, a cashier counter) at the time of customer service). When calculating the customer service utterance evaluation value based on the voice data of the store clerk who formed the clerk, the voice of the store clerk (employee) is emphasized, so the calculation accuracy of the service utterance evaluation value is higher than when the directivity is not formed It is possible to improve, and the customer service utterance evaluation value for the store clerk can be accurately calculated.

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C display authority information for the browsing operation on the detailed display screen WD7 (service customer utterance evaluation value display screen) of the customer service situation DB in the customer utterance evaluation value display device 35 for each customer service event. The authority level (authority data) that is included is further stored in the viewer DB of the management DB 2a of the management server 2, and the authority information of the browsing requester (user) on the service utterance evaluation value display screen is included in the authority data. If the condition is satisfied, the customer service utterance evaluation value display screen can be displayed on the display device 35.

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C further include, as authority data, authority information for correcting the customer service utterance evaluation value for the service status DB detail display screen WD7 (customer service utterance evaluation value display screen). If the authority information of the requester for correction of the evaluation value (user) satisfies the authority information of the correction operation included in the authority data, the customer utterance evaluation value on the customer service utterance evaluation value display screen according to the service utterance evaluation value correction operation Can be updated (corrected).

In addition, the customer

service monitoring systems

100B and 100C further store video data at a predetermined position in a predetermined sound collection area obtained by imaging of the camera devices C1,..., CM as monitoring data 4b in the recorder device 4 for browsing operation and The customer service utterance evaluation value on the customer service utterance evaluation value display screen is updated in accordance with the operation of correcting the customer service position of the customer service event of the store clerk while the video data is output by the operation of the user who has the authority information for both corrective operations ( Correction).

In addition, the customer

service monitoring systems

100, 100A, 100B, and 100C set the service utterance evaluation value for each customer service event on the service utterance evaluation value display screen according to a predetermined input operation by a user who has at least browsing authority information. Each display can be displayed on the display device 35 in a comparative manner, and comparison for each predetermined item can be easily performed.

In the present embodiment, the shift process of step S2 shown in FIG. 6 is not limited to the method of shifting the cut start time that is the starting point for cutting the monitoring data 4a by a predetermined time ts. Variations of the shift process will be described with reference to FIGS. FIG. 39 is an explanatory diagram showing an example of a variation of the cutting process of the monitoring data 4a. FIG. 40 is a flowchart for explaining another example of the detailed operation procedure of customer service event information processing. FIG. 41 is a diagram illustrating an example of an utterance assumption section table for each service event that constitutes a part of the service utterance evaluation DB. In the description of FIG. 40, the same processes as those shown in FIG. 9 are assigned the same step numbers, and the description is simplified or omitted.

For example, as in FIG. 38, in the monitoring data 4a shown in FIG. 39, the POS operation history data indicating that the accounting completion operation occurred at time t1, and the salesclerk uttered “Thank you” from time t2 to time t3. Assume that audio data is stored. In this case, when the shift process described with reference to FIG. 38 is used, the monitoring data extraction unit 38 of the customer service evaluation device 3 receives the audio data for a predetermined time interval (that is, the cutout range RG0) from the head of the monitoring data 4a. The monitoring data 4a1 cut out from the POS operation history data is extracted. However, as in FIG. 38, the monitoring data 4a1 does not include the clerk's voice data corresponding to the transaction completion operation. Therefore, the customer service event cannot be detected, and accurate customer service evaluation is impossible.

Therefore, the monitoring data extraction unit 38 of the customer service evaluation device 3 acquires information related to the utterance assumption section corresponding to the customer service event ID with reference to the utterance assumption section table shown in FIG. 41 after step S14 shown in FIG. The monitoring data 4a2 ′ is reacquired from the monitoring data 4a1 by changing the range of the audio data to be cut (S14-1 shown in FIG. 40). The processing after step S14-1 is the same as the processing after step S15 shown in FIG.

Here, the details of step S14-1 will be described with reference to FIG. In the utterance assumption section table shown in FIG. 41, a service event ID, a service event name, and an utterance section are associated. For example, when the customer service event ID is “accounting completion greeting” with “EID1”, the section assumed to be uttered by the store clerk is 10 seconds from the time when the customer service event (that is, the accounting completion greeting) is detected. . Therefore, the monitoring data extraction unit 38 of the customer service evaluation device 3 uses the customer service event ID as a starting point for cutting the audio data of the monitoring data 4a when the customer service event (for example, accounting completion greeting) is detected. Is changed to the time t1, and the audio data for 10 seconds from the time t1 is re-acquired as the audio data in the cutout range RG1. As a result, the monitoring data extraction unit 38 of the customer service evaluation device 3 updates the monitoring data 4a1 acquired at the first time to the monitoring data 4a2 ′, so that the audio data corresponding to the voice section spoken by the store clerk at the customer service event is obtained. As a result, the customer service event detection accuracy is improved.

In addition, as shown in FIG. 41, the cut-out range of the voice data is different for each customer service event, and a customer service event (that is, a customer service start greeting) is detected in the customer service event ID “EID2” “account start greeting”. The total time is 10 seconds over 5 seconds before and after the starting point (see, for example, the cutout range RG3 shown in FIG. 39).

In addition, in the “entry greeting” with the customer service event ID “EID3”, the cut-out range of the audio data is the customer service event (that is, the customer greeting) similar to the “accounting completion greeting” with the customer service event ID “EID1”. It is 10 seconds from the time when is detected. Further, in the “sales greeting” with the customer service event ID “EID4”, the cut-out range of the audio data is a period of 10 seconds before the customer service event (that is, the salutation greeting) is detected (for example, FIG. 39). The cutting range RG2 shown in FIG.

As a result, the monitoring data extraction unit 38 of the customer service evaluation device 3 holds, as an utterance assumption interval table, information related to a speech segment that is assumed to include the speech of the clerk at the customer service event corresponding to the detected customer service event ID. By using this, it is possible to cut out and extract the voice data of the optimum voice section spoken by the store clerk for each customer service event as monitoring data, and to perform accurate customer service evaluation.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present disclosure. Understood.

This disclosure does not use human resources like investigators, and protects customer privacy extensively, and monitors customer utterances during various customer service events for customers in the store. It is useful as a service monitoring system and service monitoring method that improves the convenience of accurately and objectively evaluating the situation.

2 Management server 2a Management DB
3, 3C Customer service evaluation device 4 Recorder device 4a, 4b Monitoring data 5 POS terminal 31

Operation unit

32, 53 Memory 33 Service utterance evaluation unit 34

Output unit

35, 52 Display device 36 Speaker device 37 Directivity control unit 51 Input device 331 Customer service Event detection unit 332 Service utterance evaluation

value calculation unit

100, 100A, 100B, 100C Service monitoring system AM1, AML Microphone array device C1, CM Camera device M1, ML Microphone device S1, SN Sensor device SM1, SML Customer service microphone device

Claims

A sound collection unit for collecting employee's voice in a predetermined sound collection area;
A first storage unit for storing customer service event data including determination conditions for each predetermined customer service event;
A second storage unit that stores terminal operation history data indicating an operation history of an employee for a predetermined business terminal and voice data of the employee collected by the sound collection unit in association with each other;
A detection unit for detecting the customer service event of the employee based on the customer service event data stored in the first storage unit and the terminal operation history data stored in the second storage unit;
In the customer service event detected by the detection unit, based on the employee voice data stored in the second storage unit, a customer service speech evaluation value corresponding to a predetermined speech keyword at the time of operation of the business terminal is calculated. A calculation unit to calculate,
An output unit that stores the customer utterance evaluation value calculated by the calculation unit in association with the employee identification data, the employee voice data specified by the employee reception position and the customer reception time, and Comprising
Service monitoring system.
The customer service monitoring system according to claim 1,
The detection unit omits detection of the customer service event of the employee when the employee's voice data stored in the second storage unit includes information indicating customer privacy protection.
Service monitoring system.
The customer service monitoring system according to claim 1,
The first storage unit further stores keyword data including an utterance assumption keyword for each predetermined customer service event,
The calculation unit, when the employee speech data stored in the second storage unit does not include the speech utterance keyword corresponding to the customer service event detected by the detection unit, Set the evaluation value to zero or deduct a predetermined point from the customer utterance evaluation value,
Service monitoring system.
The customer service monitoring system according to claim 1,
The first storage unit further stores keyword data including an utterance assumption keyword for each predetermined customer service event,
The calculation unit may include the second utterance keyword corresponding to the customer service event detected by the detection unit in the employee voice data stored in the second storage unit. Updating the employee's voice data stored in the storage unit to the voice data of the utterance assumption keyword;
Service monitoring system.
The customer service monitoring system according to claim 4,
The first storage unit further stores keyword voice data including voice data of an assumed utterance keyword for each predetermined customer service event,
The calculation unit calculates a first predetermined value from the utterance length of the utterance assumption keyword of the keyword voice data stored in the first storage unit as the utterance length of the utterance assumption keyword of the updated voice data of the employee. When exceeding the range, a predetermined score is deducted from the customer service utterance evaluation value,
Service monitoring system.
The customer service monitoring system according to claim 4,
The first storage unit further stores keyword voice data including voice data of an assumed utterance keyword for each predetermined customer service event,
The calculation unit calculates the frequency for each phoneme of the utterance assumed keyword of the updated voice data of the employee from the fundamental frequency for each phoneme of the utterance assumed keyword of the keyword speech data stored in the first storage unit. When exceeding the second predetermined range, a predetermined score is subtracted from the customer utterance evaluation value,
Service monitoring system.
The customer service monitoring system according to claim 1,
The sound collection unit is a microphone device that is individually attached to the employee,
The detection unit further detects the customer service event of the employee based on the customer service event data stored in the first storage unit and the employee's voice data collected by the microphone device;
Service monitoring system.
The customer service monitoring system according to claim 1,
An imaging unit that images a predetermined position of the predetermined sound collection area;
The second storage unit further stores video data at a predetermined position of the predetermined sound collection area captured by the imaging unit,
The detection unit detects the customer service event of the employee based on video data at a predetermined position of the predetermined sound collection area stored in the second storage unit;
Service monitoring system.
The customer service monitoring system according to claim 8,
A customer detection unit for detecting the appearance or exit of the customer with respect to the predetermined sound collection area,
The second storage unit further stores a detection result of the appearance or exit of the customer by the customer detection unit,
The detection unit detects the customer service event of the employee based on the detection result of the appearance or leaving of the customer by the customer detection unit,
Service monitoring system.
The customer service monitoring system according to claim 8 or 9,
A directivity control unit that forms voice directivity from the sound collection unit in a predetermined directivity direction based on the employee voice data stored in the second storage unit;
The calculation unit calculates the customer service utterance evaluation value based on the voice data of the employee in which the voice directivity is formed by the directivity control unit.
Service monitoring system.
The customer service monitoring system according to claim 1,
The first storage unit further stores authority data including authority information for browsing operation of the customer service utterance evaluation value display screen in the display unit of the customer service utterance evaluation value calculated by the calculator for each customer service event,
The output unit evaluates the service utterance when the authority information of the requester of browsing on the service utterance evaluation value display screen satisfies the authority information of the browsing operation included in the authority data stored in the first storage unit. Display a value display screen on the display unit;
Service monitoring system.
The customer service monitoring system according to claim 11,
The authority data includes authority information for correcting the service utterance evaluation value for the service utterance evaluation value display screen,
The output unit, when the authority information of the requester for correcting the customer service utterance evaluation value satisfies the authority information of the correction operation included in the authority data stored in the first storage unit, In response to a correction operation, the customer service utterance evaluation value on the customer service utterance evaluation value display screen is updated.
Service monitoring system.
The customer service monitoring system according to claim 12,
An imaging unit that images a predetermined position of the predetermined sound collection area;
The second storage unit further stores video data at a predetermined position of the predetermined sound collection area captured by the imaging unit,
The output unit is a customer service event detected by the detection unit while the video data at the predetermined position of the predetermined sound collection area stored in the second storage unit is being output. Updating the customer service utterance evaluation value on the customer service utterance evaluation value display screen according to the correction operation of
Service monitoring system.
The customer service monitoring system according to claim 11,
The output unit causes the display unit to display the service utterance evaluation value for each service event on the service utterance evaluation value display screen for each predetermined item in response to a predetermined input operation.
Service monitoring system.
The customer service monitoring system according to claim 1,
A voice data extraction unit for extracting the employee's voice data stored in the second storage unit;
The first storage unit stores information related to an utterance assumption section in which the employee is assumed to utter for each customer service event,
The voice data extraction unit extracts voice data of the employee corresponding to the speech utterance section corresponding to the customer service event detected by the detection unit, using information on the speech utterance section.
Service monitoring system.
A customer service monitoring method in a customer service monitoring system including a sound collection unit that collects an employee's voice in a predetermined sound collection region,
Storing customer service event data including determination conditions for each predetermined customer service event in the first storage unit;
Terminal operation history data indicating an employee's operation history with respect to a predetermined business terminal and voice data of the employee collected by the sound collection unit are associated with each other and stored in the second storage unit;
Detecting the customer service event of the employee based on the customer service event data stored in the first storage unit and the terminal operation history data stored in the second storage unit;
In the detected customer service event, based on the employee's voice data stored in the second storage unit, calculate a customer service utterance evaluation value corresponding to a predetermined utterance keyword when operating the business terminal,
The calculated customer service utterance evaluation value is stored in association with the employee's voice data specified by the employee identification information, the customer service position and the customer service time,
Customer service monitoring method.