JP2011221683A - Customer service support device, customer service support method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a salesclerk to recognize customer satisfaction level in real time and improve customer service skills of the salesclerk.SOLUTION: A customer service support device comprises: a voice acquisition part 102 which acquires voice data in conversation between a salesclerk and a customer; a voice data extracting part which extracts the voice data of the customer from the acquired voice data; a customer satisfaction level calculation part 156 which calculates the customer satisfaction level based on the extracted voice data of the customer; and a voice output part 103 which notifies of the salesclerk the customer satisfaction level calculated by the calculation part 156.

Description

  The present invention relates to a customer service support device, a customer service support method, and a program for supporting customer service by a store clerk.

  Conventionally, in stores such as retail stores, attempts have been made to collect customer data and sales data and use them as marketing data for the purpose of sales promotion. For example, Patent Document 1 proposes a marketing data collection / analysis technique that can accurately grasp a consumer's co-sale tendency from the analysis of POS data and reflect it in sales promotion.

JP 2007-94592 A

By the way, in the field of marketing, it is said that it is better to increase customer satisfaction at the expense of productivity and efficiency through consumer repeaters. That is, in customer service by a store clerk, it is required to recognize customer satisfaction. A salesclerk with high sales ability estimates the customer satisfaction level of the customer from the conversation with the customer, and continues the conversation or changes the content of the conversation based on the estimation. Thereby, the customer's willingness to purchase can be improved. Further, based on the recognized customer satisfaction, a customer service method for improving customer satisfaction is learned, which contributes to improvement of customer service ability.
However, such a method for recognizing customer satisfaction has to rely on the clerk's intuition and analysis ability, and depending on the clerk, the customer satisfaction cannot be recognized.

  The present invention relates to a customer service support device, a customer service support method, and a program capable of improving the customer service capability of a store clerk by making a store clerk recognize customer satisfaction in real time.

  The customer service support apparatus of the present invention includes a voice data acquisition unit that acquires voice data in a conversation between a store clerk and a customer, a voice data extraction unit that extracts customer voice data from the acquired voice data, and the extracted customer voice data A customer satisfaction calculation unit for calculating customer satisfaction, and a satisfaction notification unit for notifying the store clerk of the customer satisfaction calculated by the customer satisfaction calculation unit.

  The customer service support method of the present invention is a customer service support method for a customer service support device that supports customer service by a store clerk, and the customer service support device acquires a voice data acquisition step for acquiring voice data in a conversation between a store clerk and a customer. The voice data extraction process for extracting customers from the voice data, the customer satisfaction calculation process for calculating customer satisfaction based on the extracted customer voice data, and the customer satisfaction calculated by the customer satisfaction calculation process, And a satisfaction notifying step for notifying the store clerk.

  According to these configurations, it is possible to make the clerk recognize the customer satisfaction by calculating the customer satisfaction based on the voice data in the conversation between the clerk and the customer, and notifying the calculated customer satisfaction to the clerk. it can. Therefore, the customer service ability of the store clerk can be improved.

  In the customer service support apparatus described above, it is preferable that the satisfaction notifying unit notifies the customer satisfaction by outputting a voice indicating the customer satisfaction.

  In this case, the satisfaction notifying unit notifies the customer satisfaction by selectively outputting one voice pattern from a plurality of voice patterns according to the customer satisfaction calculated by the customer satisfaction calculating unit. It is preferable to do.

  In this case, it is preferable that the satisfaction notifying unit notifies the customer satisfaction by displaying a character string indicating the customer satisfaction.

  In this case, it is preferable that the satisfaction notifying unit notifies the customer satisfaction by vibration according to the customer satisfaction.

  According to these configurations, customer satisfaction can be easily notified with a simple configuration.

  In this case, it is preferable that the voice data acquisition unit and the satisfaction notification unit are configured by a wearable device attached to a store clerk.

  According to this configuration, the voice of the clerk can be efficiently acquired by configuring the voice data acquisition unit with the wearable device. Further, by configuring the satisfaction notification unit with a wearable device, the customer satisfaction can be notified only to the store clerk via the earphone or the like, so that the customer does not feel uncomfortable. Here, the “wearable device” means a device that can be worn on a store clerk's body or clothes.

  In this case, the customer satisfaction calculation unit, when a continuous speech section is set as one utterance section, an individual satisfaction calculation section that calculates customer satisfaction for each utterance section, and a clerk who is repeated without interruption for a predetermined time or more Alternatively, customer satisfaction for each conversation section, where a group of customer utterance sections is one conversation section, and a group of talk sections that are repeated alternately without interruption between the store clerk and the customer is a conversation section. And a section satisfaction calculation unit that calculates customer statistic of each as a conversation section, and the satisfaction notification section notifies the customer satisfaction of each conversation section at the end of the conversation section. Is preferred.

  According to this configuration, at the end of the conversation section where the talking section is not interrupted, the customer satisfaction is notified in units of the conversation section, so that the customer satisfaction is notified after the conversation is interrupted. Therefore, the conversation with the customer is not hindered by the notification of customer satisfaction. In addition, since the customer satisfaction is calculated with a section where the talking section is not interrupted as one unit, the store clerk can easily recognize which section of the customer satisfaction is notified.

  In this case, the customer satisfaction level has reached a predetermined threshold among the related keyword storage unit that stores the related keyword related to the predetermined voice keyword and the utterance section that is a continuous voice section included in the voice data. A voice identification unit for identifying a voice segment, a voice recognition unit for voice recognition of voice data of the voiced speech segment, and a voice for extracting a voice keyword included in the voice data of the voice segment based on the recognition result of the voice recognition unit A keyword extraction unit; a related keyword extraction unit that extracts a related keyword related to the extracted voice keyword from the related keyword storage unit; and a related keyword notification unit that notifies the clerk of the extracted related keyword. preferable.

  In this case, the related keyword storage unit preferably stores at least one of the product name of the product estimated by the voice keyword, the product type of the product, and a catch phrase as the related keyword.

  According to these configurations, it is possible to easily recognize what keywords are popular with customers. Moreover, since the voice keyword is not notified as it is and the related keyword related thereto is notified, for example, the keyword indicating the same thing can be integrated into the related keyword and notified. That is, since the store clerk can omit the thought of analyzing the keywords and integrating them into related keywords, the customer service efficiency can be improved. Further, by storing the product name as the related keyword, for example, when the abbreviation of the product or the product type is acquired as the voice keyword, the product name indicated by them can be integrated and notified. Further, by storing the product type as the related keyword, when the product name is acquired as the voice keyword, it can be integrated and notified to the product type to which the product name belongs. Furthermore, by storing a catch phrase as a related keyword, when a catch phrase of a similar expression is acquired, it can be integrated and notified to the one catch phrase.

  In this case, a topic keyword storage unit that stores topic keywords classified for each topic, and a speech recognition unit that recognizes speech data in units of the speech segment when a continuous speech segment is defined as one speech segment; Based on the recognition result of the voice recognition unit, the voice keyword extraction unit that extracts the voice keyword included in the voice data of the clerk and the customer and the topic keyword storage unit are referred to, and the voice keyword extracted for each utterance section is supported. A topic segment identifying unit that identifies a topic keyword and identifies a group of utterance segments to which the classification to which the topic keyword belongs does not change as a topic segment, and the satisfaction degree calculation unit is a customer satisfaction level for each utterance segment Individual satisfaction calculation unit that calculates statistic and section satisfaction calculation that calculates customer satisfaction statistics for each topic section as customer satisfaction for each topic section Includes a part, the satisfaction notifying unit, for each end of the topic section, it is preferable to notify the customer satisfaction topic section units.

  According to this configuration, it is possible to easily recognize which topic was popular or unpopular with the customer by notifying the customer satisfaction level in units of topic sections at the end of the topic section.

  The program of the present invention causes a computer to execute each step in the customer service support method.

  According to this configuration, each process in the customer service support method can be executed simply by installing the program on a computer.

1 is a system configuration diagram of a customer service support system according to a first embodiment. It is a control block diagram of the terminal for shop assistants. It is a control block diagram of a receipt printer. It is a control block diagram of a management server. It is explanatory drawing of a speech section, a talk section, a conversation section, and a customer service section. It is a functional block diagram of the customer service support system concerning a 1st embodiment. It is explanatory drawing of the database for management servers which concerns on 1st Embodiment. It is a figure which shows an example of an audio | voice data management table, a salesclerk utterance area management table, and a customer utterance area management table. It is a figure which shows an example (a) of the customer satisfaction preservation | save table which concerns on 1st Embodiment, and an example (b) of a customer satisfaction preservation | save table which concerns on 2nd Embodiment. It is a figure which shows an example of the table for goods keywords. It is a figure which shows an example of the table for non-commodity keywords. It is a figure which shows an example of a merchandise satisfaction keyword management table and a non-commodity satisfaction keyword management table. It is a flowchart which shows the audio | voice data preservation | save process which concerns on 1st Embodiment. It is a flowchart which shows the service area specific process which concerns on 1st Embodiment. It is a flowchart which shows the shop assistant talk area specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area B specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area A specific process which concerns on 1st Embodiment. It is a flowchart which shows the satisfaction notification process which concerns on 1st Embodiment. It is a flowchart which shows the keyword recording process which concerns on 1st Embodiment. It is a flowchart which shows the keyword process for every speech area which concerns on 1st Embodiment. It is a flowchart which shows the recommended goods notification process which concerns on 1st Embodiment. It is a flowchart which shows the favorable impression keyword notification process which concerns on 1st Embodiment. It is a functional block diagram of the customer service support system which concerns on 2nd Embodiment. It is explanatory drawing of the database for management servers which concerns on 2nd Embodiment. It is the flowchart which showed the topic area specific process which concerns on 2nd Embodiment. It is the flowchart which showed the topic area satisfaction notification process which concerns on 2nd Embodiment. It is the figure which showed an example of the audio | voice pattern table.

[First Embodiment]
Hereinafter, a customer service support device, a customer service support method, and a program according to the present invention will be described with reference to the accompanying drawings. In the following embodiments, the case where the customer service support device of the present invention is applied to the customer service support system SY is illustrated. The customer service support system SY is constructed to recognize customer emotions in stores and venues related to retail, restaurant and service industries, and to utilize the results for improving customer satisfaction and sales. Therefore, in the following embodiment, the case of performing emotion recognition of a customer who is serving a store clerk in a clothing store is illustrated.

  FIG. 1 is a system configuration diagram of a customer service support system SY according to the first embodiment. As shown in the figure, the customer service support system SY is provided at the body entrance sensor 1, the voice acquisition microphone 2, the voice instruction earphone 3, and the store clerk terminal 5 that are attached to the store clerk, and at the store entrance and in the store. An in-store camera 11 (only one is shown in the figure), a POS (Point Of Sales) terminal 12 and a receipt printer 13 installed in the checkout counter 14, and a management server 15 installed in the backyard of the store And a display terminal 16.

  The body-conducting sound sensor 1 is mounted on the head and neck of the store clerk, and detects the body-conducted sound that the store clerk's voice reaches the body surface through bones and meat. In the present embodiment, the voice acquired by the voice acquisition microphone 2 is used to identify whether it is that of a store clerk or a customer. The voice acquisition microphone 2 is attached to the clerk's uniform (near the chest) and acquires the clerk's and customer's voices. In place of the body-conducting sound sensor 1 and the voice acquisition microphone 2, microphones having directivity for the store clerk and the customer may be used. In other words, the clerk's voice and the customer's voice may be identified by using two microphones, one for obtaining the clerk's voice and the other for obtaining the customer's voice.

  The voice instruction earphone 3 is attached to a store clerk's ear, and outputs voice based on voice data for voice notification. Specifically, as voice data for voice notification, a message indicating customer satisfaction and a message notifying a recommended product or a good impression keyword are notified. For example, as a message indicating customer satisfaction, a message such as “customer satisfaction 50” is notified, and as a message indicating a recommended product or a good impression keyword, “the recommended product is a stretch slim fit shirt. "And" The keyword of good impression is a birthday party for children. "

  The clerk terminal 5 is attached to the clerk's uniform (belt or the like), acquires output data of the body conduction sensor 1 and the voice acquisition microphone 2 via a dedicated cable, and is connected to the voice instruction earphone 3 to the management server. The voice data for voice notification from 15 is output. In addition, it has a wireless communication function with the receipt printer 13, and exchanges information with the management server 15 via the receipt printer 13.

  The in-store camera 11 is provided on the ceiling of each place in the store and captures images of customers who have come to the store as well as salesclerks and customers who are serving customers. As the in-store camera 11, a CCD camera or a PTZ (Pan Tilt Zoom) camera can be used.

  The POS terminal 12 has a general register configuration, and performs accounting processing according to the POS application. Further, a product code is obtained from a bar code scanner or a keyboard (not shown), and receipt data for printing on the accounting receipt R (see FIG. 3) is generated with reference to the product master 18.

  The receipt printer 13 is connected to the POS terminal 12 via a dedicated cable, and prints receipt print data acquired from the POS terminal 12 on receipt paper. The receipt printer 13 has a wireless communication function with the clerk terminal 5 and a wired communication function with the management server 15. In this way, by inputting / outputting various information with the receipt printer 13 as the main (the receipt printer 13 filters the acquired various information and outputs necessary information), the POS backbone network (POS terminal 12). Network) is not affected. Further, when applying the present invention to an existing POS system, it is not necessary to change the POS backbone network itself.

  The management server 15 is connected to the receipt printer 13 via a network 19 such as an intranet, and exchanges various information with the clerk terminal 5 via the receipt printer 13. Further, based on various data such as voice data acquired from the clerk terminal 5, voice recognition, emotion recognition, and customer satisfaction are calculated. Also, voice data for voice notification based on the calculated customer satisfaction is generated.

  Next, the hardware configuration of the clerk terminal 5, the receipt printer 13, and the management server 15 will be described with reference to FIGS. FIG. 2 is a control block diagram of the clerk terminal 5. The clerk terminal 5 includes a wireless LAN antenna 21, a wireless LAN RF (Radio Frequency) unit 22, a wireless LAN modem unit 23, and a wireless LAN baseband unit 24 for realizing a wireless communication function with the receipt printer 13. . The wireless LAN baseband unit 24 stores a MAC address for identifying the clerk terminal 5. In addition, the store clerk terminal 5 includes an amplifier unit 28 and an A / D converter 29 for obtaining the detection result of the body-conducting sound sensor 1, and an amplifier unit 32 for obtaining the audio data acquired by the audio acquisition microphone 2. An A / D converter 33, an amplifier unit 35 for outputting voice data for voice notification to the voice instruction earphone 3, and a D / A converter 36 are provided. Further, the store clerk terminal 5 includes a control unit 25 that performs overall control of each unit, a memory 26 that stores various data including firmware, and a battery 34 that supplies power. Based on the data acquired from the A / D converter 29 and the A / D converter 33, the control unit 25 specifies a store clerk utterance section specifying function for specifying a clerk utterance section (continuous voice time zone), and an A / D converter. And a sound level determination function for determining a sound level based on the data acquired from 33.

  Next, FIG. 3 is a control block diagram of the receipt printer 13. The receipt printer 13 includes a wireless LAN antenna 41, a wireless LAN RF unit 42, a wireless LAN modem unit 43, and a wireless LAN baseband unit 44 for realizing a wireless communication function with the clerk terminal 5. The wireless LAN baseband unit 44 stores a MAC address for identifying the receipt printer 13. The receipt printer 13 includes an input I / F unit 45 to which receipt data is input from the POS terminal 12, a CG-ROM 46 that stores character patterns, a control unit 47 that performs overall control of each unit, a print head, a head A printing mechanism 48 including a driving mechanism and a receipt paper transport mechanism, and a wired LAN I / F unit 49 connected to the management server 15 via a wired LAN are provided.

  The control unit 47 includes a main processing unit 47a that performs analysis of receipt data including a predetermined command, generation of print data to be printed on the accounting receipt R, and a receipt data semantic analysis unit that is a configuration unique to the present embodiment. 47b. The receipt data semantic analysis unit 47b recognizes the terminal number of the POS terminal 12, the receipt number, the product code, the product name, the product unit price, the amount of money, the operator name, and the like from the receipt data, and can be interpreted by the management server 15 serving as the host system. To a predetermined data format (for example, XML format). In addition, what converted the recognition result of the receipt data into a predetermined data format is hereinafter referred to as “converted data”. In addition, the control unit 47 transmits the audio data received from the store clerk terminal 5 via the wireless LAN (audio data acquired from the wireless LAN baseband unit 44) to the management server 15 via the wired LAN I / F unit 49. Forward.

  Next, FIG. 4 is a control block diagram of the management server 15. The management server 15 acquires audio data and conversion data from the receipt printer 13 and also displays a variety of information on the display terminal 16 and the wired LAN I / F unit 51 for acquiring video data from the in-store camera 11. The display processing unit 52 includes a control unit 53 that acquires input data from an input device 55 such as a mouse and a keyboard and performs overall control of each unit, and a storage unit 54 that stores various types of information. The control unit 53 has a conversation section specifying function for specifying a conversation section (one unit section of a conversation between a store clerk and a customer) based on the acquired voice data, and a satisfaction calculation function for calculating a customer satisfaction per conversation section. And a satisfaction keyword recording function that records keywords (satisfaction keywords) with high customer satisfaction based on voice data of conversation sections with high customer satisfaction, and voices based on the calculated customer satisfaction and recorded satisfaction keywords. A voice data generation function for generating voice data for notification.

  Next, the definition of each section will be described with reference to FIG. As described above, a continuous voice section by the same person, that is, a one-phrase section in which no breath (breathing) or the like is entered is referred to as an “utterance section”. In the present embodiment, emotion recognition and voice recognition are performed for each utterance section. Further, as shown in FIG. 5A, a collection of clerk or customer utterance sections that are repeated without interruption for a predetermined time or longer is referred to as a “talking section”. That is, it refers to a set of one or more utterance sections whose interval is less than the predetermined time X (where X is a constant that satisfies X> 0). In the example shown in the figure, a store clerk talk section (hereinafter referred to as “store clerk talk section”) and two customer talk sections (hereinafter referred to as “customer talk section”) around the store clerk talk section. Both cases show a case of two utterance sections.

  Similarly, as shown in FIG. 5A, a group of talking sections in which the talking sections of the store clerk and the customer are alternately repeated without being interrupted for a predetermined time or longer are referred to as “conversation sections”. That is, it refers to a collection of one or more utterance sections whose interval is less than the predetermined time Y (where Y is a constant such that Y> X). In the present embodiment, a group of customer talk sections before and after the store staff talk section (that is, a collection of at least one and up to three talk sections) is expressed as “one conversation pattern” = “one conversation section”. Define.

  In addition, as shown in FIG. 5B, a collection of conversation sections that are repeated without interruption for a predetermined time or longer is referred to as a “service section”. That is, it refers to an aggregate of one or more conversation sections whose interval is less than the predetermined time Z (where Z is a constant such that Z> Y). The example of the figure shows a service section 1 composed of two conversation sections and a service section 2 composed of three conversation sections. Thus, the number of conversation sections included in the customer service section is arbitrary.

  Next, with reference to FIG. 6 and FIG. 7, the functional configuration of the customer service support system SY according to the first embodiment will be described. FIG. 6 is a block diagram of the customer service support system SY. The in-store camera 11 has a customer service imaging unit 111 as a main functional configuration. The customer service imaging unit 111 captures a customer service situation including a clerk and a customer. In the present embodiment, the customer service imaging unit 111 always performs imaging, and the video data is output to the management server 15 as needed.

  The body sound sensor 1 has an utterance detection unit 101 as a main functional configuration. The utterance detection unit 101 detects that the clerk uttered and the utterance section based on the body conduction sound. The voice acquisition microphone 2 has a voice acquisition unit (voice data acquisition unit) 102 as a main functional configuration. The voice acquisition unit 102 acquires the voice (voice signal) of the clerk and the customer. The voice instruction earphone 3 has a voice output unit (customer satisfaction notification unit, related keyword notification unit) 103 as a main functional configuration. The voice output unit 103 outputs voice based on voice data for voice notification. The clerk terminal 5 has an audio data communication unit 105 as a main functional configuration. The audio data communication unit 105 determines the presence or absence of audio using a power filter in the audio level determination function, and transmits audio data of a predetermined audio level or higher (for example, 1.5 v or higher after amplifier amplification) to the management server 15. . Further, the salesclerk utterance section specifying function specifies a salesclerk utterance section based on the detection result of the utterance detection unit 101 and the voice acquisition result of the voice acquisition unit 102, and notifies the management server 15 of the occurrence of the clerk utterance section. Furthermore, voice data for voice notification is received from the management server 15. Actually, the clerk terminal 5 and the management server 15 communicate with each other via the receipt printer 13. However, the receipt printer 13 simply passes information, and is not shown.

  The receipt printer 13 has a conversion data transmission unit 113 as a main functional configuration. The conversion data transmission unit 113 transmits the conversion data obtained by converting the receipt data output from the POS terminal 12 into the XML format to the management server 15.

  The management server 15 includes a video recording unit 151, a customer identification unit 152, a conversation recording unit 153, a voice recognition unit 154, a customer emotion recognition unit 155, a customer satisfaction calculation unit 156, a converted data reception unit 158, as main functional configurations. A satisfaction keyword recording unit 161, a recommended level calculation unit 162, a good impression level calculation unit 163, an audio data generation unit 164, and a management server database DB are provided.

  The video recording unit 151 acquires video data from the customer service imaging unit 111 and records it in the management server database DB. The customer identifying unit 152 identifies a customer based on the facial feature amount included in the video data. Specifically, customer identification information and customer face feature amounts are stored in advance in the management server database DB (see the customer information storage unit 81 in FIG. 7), and the captured customer face feature amounts (in-store camera) 11 is obtained by analyzing the imaging result of 11 and performing face detection, and normalizing the image of the face part), and facial features of many customers stored in the management server database DB. Collate and determine that the customer is the most similar among them. Note that when there are a plurality of store clerk in the store, it is necessary to associate the store clerk with the customer, so the store clerk may also be identified from the facial feature amount included in the video data. That is, the clerk identification information and the clerk face feature amount are stored in the management server database DB in advance, and the captured clerk face feature amount and each clerk face feature stored in the management server database DB are stored. The quantity is collated, and it is determined that the store clerk with the highest degree of similarity is serving customers.

  The conversation recording unit 153 records the conversation between the clerk and the customer, that is, the voice data transmitted from the voice data communication unit 105 (records it in the management server database DB). The voice recognition unit 154 recognizes the voices of the clerk and the customer included in the recorded data recorded by the conversation recording unit 153. The speech recognition unit 154 includes an acoustic analysis unit, an acoustic model, a language model, a word dictionary, and a text conversion unit (all not shown). The voice recognition unit 154 uses these to convert the acquired voice signal into text information and record it in the management server database DB.

  The customer emotion recognition unit 155 recognizes the customer's emotion based on the customer's voice included in the conversation. Specifically, emotion recognition is performed based on the amount of change in voice intensity, voice generation speed (number of mora per unit time, etc.), word-by-word intensity, volume, and voice spectrum. In this case, the customer emotion recognition unit 155 determines whether the voice included in the conversation is the voice of the clerk based on the detection result of the utterance detection unit 101 (the clerk utterance section specified by the voice data communication unit 105). Determine if it is voice. Further, the customer emotion recognition unit 155 performs emotion recognition for each customer utterance section included in the recording data of the conversation recording unit 153. Thus, highly accurate emotion data can be obtained by performing emotion recognition for each phrase. Further, an overlapping section in which the customer utterance section and the clerk utterance section overlap on the time axis is specified, and the overlapping section is set as an “emotional recognition non-target section”, and emotion recognition is performed on the customer utterance section excluding the overlapping section. Do. In this way, misrecognition is prevented by performing emotion recognition except for emotion recognition non-target sections (overlapping sections) in which customer voice and store clerk voice are mixed and accurate emotion recognition cannot be performed. Can do.

  The customer satisfaction calculation unit 156 calculates customer satisfaction based on the recognition result of the customer emotion recognition unit 155. Further, as the customer emotion recognition unit 155 performs emotion recognition for each utterance section, the customer satisfaction calculation unit 156 also calculates the customer satisfaction for each utterance section.

  The conversion data reception unit 158 receives the conversion data transmitted from the conversion data transmission unit 113 and records it in the management server database DB. The conversion data is used to obtain information related to sales results recorded as the customer service data. Therefore, from the information included in the conversion data, only information that can specify the presence / absence of sales and the sales amount, such as the receipt number and total amount, may be extracted and recorded as sales results, or the conversion data All may be recorded in the management server database DB.

  The satisfaction keyword recording unit 161 collates the speech recognition result (text information) of the conversation section in which the customer satisfaction has reached a predetermined threshold with the keyword table 89 in the management server database DB, and based on the collation result, A keyword with high customer satisfaction (satisfaction keyword) and related information are recorded in the management server database DB. Note that the satisfaction keyword is classified into a product keyword indicating a handled product and a non-product keyword other than that.

  The recommended level calculation unit 162 calculates the recommended level of each product keyword recorded by the satisfaction keyword recording unit 161 and records it in the management server database DB. The good impression level calculation unit 163 calculates the good impression level of each non-product keyword recorded by the satisfaction keyword recording unit 161 and records it in the management server database DB. The recommended level is a numerical value indicating the recommended degree of each product indicated by each product keyword. On the other hand, the good impression level is a numerical value indicating the degree of good impression each non-product keyword gives to a customer.

  The voice data generation unit 164, based on various data in the management server database DB, voice data for notifying customer satisfaction and keywords for notifying recommended products and good impression keywords. Voice data for notification is generated and transmitted to the clerk terminal 5.

  FIG. 7 is an explanatory diagram of the management server database DB according to the first embodiment. The management server database DB includes a customer information storage unit 81, a store clerk information storage unit 82, an audio data storage unit 83, a video data storage unit 84, an audio data management table 85, a store clerk utterance section management table 86, and a customer utterance section management table 87. , Customer satisfaction storage table 92, text information storage unit 88, keyword table (related keyword storage unit) 89, product type keyword storage unit 90, and satisfaction keyword management table (database) 91. The management server database DB may be provided for each store, or may collectively manage data of a plurality of stores.

  The customer information storage unit 81 associates customer identification information (customer ID and the like), customer face feature amounts, and customer data (including personal information such as name, address, telephone number, date of birth, and sex). Remember. The store clerk information storage unit 82 stores store clerk identification information (store clerk ID, etc.), the store clerk's facial feature quantity, and the MAC address of the store clerk terminal 5 in association with each other. In addition, the voice data storage unit 83 stores the voice data recorded by the conversation recording unit 153 at all times. In addition, the video data storage unit 84 stores video data always captured by the customer service imaging unit 111.

  The voice data management table 85 is a table in which the acquired voice data is recorded for each continuous voice section (hereinafter referred to as “continuous speech section”) without distinguishing between the store clerk and the customer (FIG. 8). (See (a))). Further, the store clerk utterance section management table 86 is a table in which store section utterance sections are recorded (see FIG. 8B). The customer utterance section management table 87 is a table in which customer utterance sections are recorded (see FIG. 8C).

  Next, the voice data management table 85, the store clerk utterance section management table 86, and the customer utterance section management table 87 will be described with reference to FIG. FIG. 8A shows an example of the audio data management table 85. The voice data management table 85 includes voice data No. assigned to each continuous utterance section (a section including at least one utterance section) which is a continuous voice section that does not distinguish between the clerk and the customer's voice. Recording start time corresponding to the start time of the continuous utterance section, recording end time corresponding to the end time of the continuous utterance section, voice data based on the customer's utterance, voice data based on the occurrence of the store clerk, or both The duplication flag indicating whether the voice data is based on the utterance of the voice and the storage address of the voice data are associated with each other. For example, voice data No. The voice data 201 is a continuous utterance section having a start time of 12:36:03 and an end time of 12:36:16, and the customer's voice and the clerk's voice overlap in at least some of the sections. It shows that.

  FIG. 8B is a diagram showing an example of the store clerk utterance section management table 86. The store clerk utterance section management table 86 includes a store clerk utterance number assigned to each utterance section. , The clerk utterance start time which is the start time of the utterance section, the clerk utterance end time which is the end time of the utterance section, and the talk No. indicating which talk section belongs to. And the overlap start time, which is the start time of the overlapping section with the customer utterance, and the overlap end time, which is the end time of the overlap section with the customer utterance. For example, clerk utterance No. 100 and the clerk utterance No. The utterance section 101 is considered as a series of talk sections because the interval is less than a predetermined time (for example, 3 seconds). Is added. In addition, clerk utterance No. 100 utterance segments indicate that the entire segment overlaps with the customer's occurrence.

  FIG. 8C is a diagram showing an example of the customer utterance section management table 87. The customer utterance section management table 87 includes a customer utterance No. assigned to each utterance section. , The customer utterance start time that is the start time of the utterance section, the customer utterance end time that is the end time of the utterance section, and the talk No. indicating which talk section it belongs to. And the overlap start time, which is the start time of the overlapping section with the clerk's utterance, and the overlap end time, which is the end time of the overlap section, with the clerk's utterance. For example, customer utterance No. 101 and customer utterance no. The utterance section 102 is regarded as a different talk section because the interval exceeds a predetermined time (for example, 3 seconds). Is added. In addition, customer utterance No. The 100 utterance sections indicate that 6 seconds out of the section length of 13 seconds overlap with the occurrence of the store clerk.

  Next, the customer satisfaction storage table 92, the keyword table 89, and the satisfaction keyword management table 91 will be described with reference to FIGS. FIG. 9A is a diagram showing an example of the customer satisfaction degree storage table 92. As shown in FIG. 9A, the customer satisfaction storage table 92 stores customer satisfaction for each customer and for each conversation section. The customer identification information, the customer's name, and the start of the conversation section are stored. Time and end time are stored in association with customer satisfaction.

  The keyword table 89 includes a product keyword table 89a for storing product keywords (product names) and a non-product keyword table 89b for storing non-product keywords. FIG. 10 is a diagram illustrating an example of the product keyword table 89a. The product keyword table 89a stores a product keyword (product name: related keyword) indicating a handled product and a plurality of product type keywords (product type: voice keyword) indicating a product type to which the handled product is associated with each other. is doing. In each product keyword, one or more product type keywords are registered for each item such as “item”, “size”, “color”, “shape”, and “processing method”. For example, for the product keyword “stretch slim fit shirt”, “item” is “shirt”, “long sleeve shirt”, “shirt”, and “size” is “S”, “M”, “L”, etc. , “OFF”, “WHITE”, “LIGHT”, “GRAY”, etc. as colors, “Slim”, “fit”, “narrow”, etc. as “shape”, “Easy care” ”,“ Shape memory ”,“ stretch ”, and the like are stored in association with each other as product type keywords. In each item type keyword for each item, an association level indicating the degree of association with the handling item is set. For example, “level 3” is set for “shirt” and “long sleeve shirt”, and “level 2” is set for “shirt”. In addition, a score is added to each item, and a coefficient is added to each related level. These are used when calculating the degree of matching described later.

  FIG. 11 is a diagram showing an example of the non-product keyword table 89b. The non-product keyword table 89b is a table in which non-product keywords are recorded for each category. Main categories such as “Human relations”, “Working / commuting related”, “Hobby related”, “Event related”, “Identity related”, “Location name related”, “Date related”, and their subcategories The non-product keyword table 89b is configured by a table for each main category. For example, the main category “Human relations” has a first subcategory such as “Family” and “Friends”, and the first subcategory “Family” is a second subcategory such as “Father”, “Mother”, “Brother”, etc. Has a category. Non-product keywords such as “father”, “dad”, “father”, “father” are registered in the category of “human relation”-“family”-“father”. Similarly, in other categories, there are subcategories, and a plurality of non-product keywords are registered.

  The text information storage unit 88 stores text information obtained as a speech recognition result by the speech recognition unit 154. The product type keyword storage unit 90 stores the extracted product type keyword in the keyword recording process described later.

  The satisfaction keyword management table 91 is generated for each customer (customer identification information), and includes a product satisfaction keyword management table 91a and a non-product satisfaction keyword management table 91b. FIG. 12A shows an example of the product satisfaction keyword management table 91a. The product satisfaction keyword management table 91a is a table that stores the product keywords recorded by the satisfaction keyword recording unit 161 in association with the related information. Strictly speaking, the product satisfaction keyword management table 91a integrates and records the same product keyword when the same product keyword is recorded a plurality of times, and includes a plurality of product keywords and items, recommended levels, Average value of matching degree over multiple recording processes, number of recordings, average value of customer satisfaction across multiple recording processes, date of last utterance, presence of purchased flag, presence of uttered flag, and utterance elapsed time Record. Note that the average value of the matching level over multiple recording processes, the average value of customer satisfaction over multiple recording processes, the date of the last utterance, the presence / absence of a purchased flag, the presence / absence of an uttered flag, and the elapsed utterance time Will be described later.

  FIG. 12B is a diagram illustrating an example of the non-commodity satisfaction keyword management table 91b. The non-product satisfaction keyword management table 91b stores the related information for each set of the non-product keyword of the first keyword and the non-product keyword of the second keyword, and records the same set of non-product keywords a plurality of times. In this case, it is a table that records in an integrated manner. The non-commodity satisfaction keyword management table 91b includes a non-product keyword of the first keyword, a non-product keyword of the second keyword, and a customer who has a good impression level, the number of times of recording, and a plurality of recording processes for the set of non-product keywords. The average value of satisfaction, the last utterance date, and the presence / absence of a spoken flag are stored.

  Next, the audio data storing process will be described with reference to the flowchart of FIG. As described above, the clerk terminal 5 and the management server 15 communicate with each other via the receipt printer 13, but the illustration is omitted because the receipt printer 13 simply passes information.

  When the clerk terminal 5 acquires the audio signal (audio) from the audio acquisition microphone 2 (S11) (audio data acquisition process), the clerk terminal 5 determines the audio level by the power filter of the audio level determination function (S12). Here, if the audio level is equal to or higher than the predetermined level, it is determined that the audio is “present”, and buffering of the audio data is started in the audio data storage area (not shown) in the memory 26 (S13). At this time, the recording start time is recorded in the audio data storage area.

  Subsequently, when no audio signal is received, the recording end time is determined and recorded in the audio data storage area, and the buffering is ended (S14). Thereafter, transmission of audio data is declared to the management server 15 (S15), and the audio data buffered in the audio data storage area is transmitted together with the recording start time and recording end time (S16).

  When the management server 15 (control unit 53) receives the voice data from the store clerk terminal 5 (S17), the voice data No. 5 uniquely determined in the voice data management table 85 (see FIG. 8A). Then, the recording start time and recording end time are registered (S18). Also, the audio data is stored in the audio data storage destination (predetermined folder) designated in the audio data management table 85 (S19).

  Next, with reference to the flowcharts of FIGS. 14 to 17, a series of flow of the service area identification process will be described. FIG. 14 is a flowchart showing the main process (service section identification process), and FIGS. 15 to 17 are flowcharts showing the subroutine. As shown in FIG. 14, the management server 15 (the control unit 53) first specifies the store clerk talk section (S21), then specifies the customer talk section B generated after the store clerk talk section (S22), and also stores the store clerk talk. The customer talk section A that occurred before the section is specified (S23). Through these steps S21 to S23, a conversation section is specified (S24, see FIG. 5A). Further, by repeating S21 to S24, the customer service section is specified (S25, see FIG. 5B).

  Next, with reference to the flowchart of FIG. 15, the clerk talk section specifying process corresponding to S21 of FIG. 14 will be described. When the store clerk terminal 5 acquires the detection data from the body conduction sound sensor 1 (S31), the detection level is determined by the power filter of the store clerk utterance section specifying function. The start time is determined and recorded in the memory 26 (S32). Subsequently, the voice level is determined by the power filter of the voice level determination function, and if it is equal to or higher than the predetermined level, it is determined that the detection data is based on the voice utterance (S33). Although not particularly shown in the flowchart, when the detection level or the audio level is less than a predetermined level, the process is terminated. It is determined that the detection data is voice utterance. When it is determined that the detection data is based on voice utterance, the clerk utterance section end time is determined from the silent section following the voice utterance and recorded in the memory 26. Thereafter, the management server 15 is notified of the occurrence of the clerk utterance section (S34). At this time, the clerk utterance section start time and the clerk utterance section end time recorded in the memory 26 are also transmitted.

  On the other hand, when the management server 15 (control unit 53) receives the notification of the occurrence of the store clerk utterance section from the store clerk terminal 5 (S35), the management server 15 (control unit 53) is uniquely defined in the store clerk utterance section management table 86 (see FIG. 8B). Sales clerk utterance data No. And the clerk talk number set for each clerk talk section. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereafter, it is determined whether or not a notification of occurrence of the next clerk utterance section is received within a predetermined time (S37). If received (S37: Yes), the clerk utterance data No. And the same clerk Talk No. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereby, the salesclerk utterance section generated before and the salesclerk utterance section generated this time can be defined as a series of talk sections. Note that, when the occurrence notification of the next clerk utterance section is not received within a predetermined time (S37: No), the process is terminated assuming that a series of talk sections by the clerk is over.

  Next, with reference to the flowchart of FIG. 16, the customer talk section B specifying process corresponding to S22 of FIG. 14 will be described. The management server 15 (the control unit 53) refers to the voice data management table 85 after the end of identification of a series of talk sections by the clerk, and the voice data is stored within a predetermined time from the clerk utterance end time of the last clerk utterance section. It is determined whether or not it exists (S41). Here, when there is no voice data (S41: No), it is determined that the customer talk section B does not exist, and the process ends. If the voice data exists (S41: Yes), the recording start time and the recording end time of the voice data are read from the voice data management table 85, and the uniquely defined customer utterance number. And customer talk No. determined for each customer talk section. Then, the customer utterance start time and customer utterance end time are registered in the customer utterance section management table 87 (S42). Thereafter, referring to the voice data management table 85, it is determined whether or not voice data exists within a predetermined time from the customer utterance end time of the last customer utterance section (S43), and if present (S43: Yes). Unique customer utterance No. And the same customer talk number as above. The customer utterance start time and customer utterance end time are registered (S42). Thereby, the customer utterance section generated before and the customer utterance section generated this time can be defined as a series of talk sections. If there is no voice data within a predetermined time from the customer utterance end time of the last customer utterance section (S43: No), it is considered that there is no subsequent talk section by the customer, and the process is terminated.

  Next, with reference to the flowchart of FIG. 17, the customer talk section A specifying process corresponding to S23 of FIG. 14 will be described. The management server 15 (control unit 53) refers to the customer utterance section management table 87 after the identification of the customer conversation section B is completed, and determines from the clerk utterance start time of the clerk utterance section specified in the processing of FIG. Before the time, customer utterance No. It is determined whether or not there is a customer utterance section given (S51). Here, when the customer utterance section exists (S51: Yes), it is considered that the customer's talk section specifying process has already ended, and the process is ended. In addition, the customer utterance No. Is not present (S51: No), the voice data management table 85 is referred to, and unprocessed voice data exists before a predetermined time from the clerk utterance start time of the clerk utterance section. Is determined (S52). Here, when there is no unprocessed voice data (S52: No), it is considered that the customer talk section A does not exist, and the process is terminated. If unprocessed voice data exists (S52: Yes), the recording start time and recording end time of the voice data are read, and the customer utterance No. And customer talk No. determined for each customer talk section. Then, the customer utterance start time and the customer utterance end time are registered in the customer utterance section management table 87 (S53).

  Thereafter, referring to the voice data management table 85, it is determined whether or not there is unprocessed voice data from the customer utterance start time of the registered customer utterance section to a predetermined time before (S54). S54: Yes), the recording start time and recording end time of the voice data are read, and the customer utterance No. uniquely determined is read. And the same customer talk number as above. Then, the customer utterance start time and customer utterance end time are registered (S54). Thereby, the customer utterance section registered previously and the customer utterance section specified this time can be defined as a series of talk sections. If there is no unprocessed voice data from the customer utterance start time of the registered customer utterance section to a predetermined time before (S54: No), it is considered that there is no previous talk section by the customer, The process ends.

  As described above, the voice data management table 85, the clerk utterance section management table 86, and the customer utterance section management table 87 are registered by the processing shown in FIGS. The section B can be specified. Here, the detection of the overlap section between the store clerk talk section and the customer talk section will be briefly described. After identifying each talking section, the management server 15 refers to the store clerk utterance section management table 86 and the customer utterance section management table 87 to detect an overlapping section and record it in each table 85, 86, 87. Specifically, first, referring to the clerk utterance section management table 86, the same talk No. The earliest clerk utterance start time and the latest clerk utterance end time are set as the start time and end time of the clerk talk section. Similarly, referring to the customer utterance section management table 87, the same talk No. The earliest customer utterance start time and the latest customer utterance end time are set as the start time and end time of the customer talk section. Then, it is determined whether or not there is an overlapping section between the store clerk talk section and the customer talk section. Time) and a duplication flag is set in the audio data management table 85 (corresponding to “customer / store” in the duplication flag column in FIG. 8A).

  Next, the satisfaction degree notification process will be described with reference to the flowchart of FIG. This satisfaction notification process is performed every time a conversation section is specified, that is, every time a conversation section ends, and notifies customer satisfaction in real time. As shown in FIG. 18, first, the management server 15 (control unit 53) extracts each customer utterance section included in the specified conversation section from the customer utterance section management table 87 (S61), and the voice data management table 85. Thus, the voice data of each customer utterance section is extracted (S62) (voice data extraction unit) (voice data extraction step).

  After extracting the voice data of each customer utterance section, the customer emotion recognition unit 155 performs emotion recognition for each utterance section based on the voice data for each customer utterance section (S63). Thereafter, the customer satisfaction calculation unit 156 calculates the customer satisfaction of each customer utterance section based on the recognition result of the customer emotion recognition unit 155 (S64) (individual satisfaction calculation unit). After calculating the customer satisfaction level of each customer utterance section, based on this, the customer satisfaction level of the conversation section is calculated (S65) (section satisfaction calculation step) (customer satisfaction calculation step). Specifically, the average value of the calculated customer satisfaction levels of each customer utterance section is calculated as the customer satisfaction level of the conversation section.

  When the customer satisfaction level of the conversation section is calculated, the calculated customer satisfaction level is recorded in the customer satisfaction storage table 92 together with the information of the conversation section (S66). Specifically, the calculated customer satisfaction is recorded in association with the customer identification information, the customer's name, and the start time and end time of the conversation section. The customer identification information is based on the result of identifying the customer who has a conversation in the conversation section by the customer identification unit 152, and the customer name is the customer name corresponding to the customer identification information. Extract from

  When the calculated customer satisfaction level of the conversation section is recorded, text data for satisfaction level notification based on the customer satisfaction level is generated (S67). Specifically, as text data for notifying satisfaction, a message for notifying customer satisfaction in the conversation section, for example, when the customer satisfaction is “50”, the text of the message “customer satisfaction“ 50 ”” Generate data. When the text data is generated, the voice data generation unit generates voice data for satisfaction notification that utters the message based on the text data (S68). If the voice data for satisfaction degree notification is generated, the voice data communication unit 105 transmits the voice data to the clerk terminal 5 (S69).

  The voice instruction earphone 3 that has received the satisfaction degree notification voice data via the clerk terminal 5 outputs the satisfaction degree notification voice data (S70), and the customer satisfaction level of the conversation section is given to the clerk. Notify (satisfaction notification process). This is the end of this process.

  Next, the keyword recording process will be described with reference to the flowchart of FIG. This keyword recording process is a process of updating the satisfaction keyword management table 91 in real time for each customer service section. That is, every time a customer service section is specified, a keyword recording process is performed for each service section. First, the management server 15 (control unit 53) extracts the customer satisfaction in each conversation section in the customer service section specified by the customer service section specifying process from the customer satisfaction storage table 92 (S81). When the customer satisfaction level of each conversation section is extracted, a conversation section in which the customer satisfaction level reaches a predetermined threshold (for example, 70) is extracted (S82) (section specifying unit). If the conversation section has not been extracted (S83: No), this process ends.

  When one or more conversation sections whose customer satisfaction level has reached a predetermined threshold are extracted (S83: Yes), the voice data management section 85 extracts voice data (section data) of all utterance sections included in each extracted conversation section. ) Is extracted (S84). Thereafter, the speech recognition unit 154 performs speech recognition on the extracted speech data of each utterance section, and the text information obtained as a speech recognition result is stored in the text information storage unit 88 (S85).

  After the text information for each utterance section is extracted and stored, keyword processing for each utterance section is performed based on the text information (S86). Here, the keyword processing (S86) for each utterance section will be described in detail with reference to FIG.

  As shown in FIG. 20, the management server 15 (control unit 53) first extracts one or more product type keywords included in the text information for each utterance section from the product type keywords registered in the product keyword table 89a. (S91) (voice keyword extraction unit). For example, product type keywords “button down” and “shirt” are extracted and stored in the product type keyword storage unit 90 based on voice data of a comment “I like a button-down shirt”. If no product type keyword is extracted (S92: No), the process proceeds to a non-product keyword extraction process (S99 to S102) described later.

  On the other hand, when one or more product type keywords are extracted (S92: Yes), one or more product keywords stored in the product keyword table 89a are extracted based on the extracted one or more product type keywords. (S93) (Related Keyword Extraction Unit) (Keyword Extraction Step). Specifically, in the product keyword table 89a, a product keyword (specific product name) stored in association with the one or more product type keywords is extracted. For example, when the product type keywords “button down” and “shirt” are extracted as the product type keywords, the product keyword “fine Oxford shirt” stored as the product type keywords is extracted. When a plurality of product type keywords belonging to the same item are extracted, any one product type keyword is set as a reference keyword based on the relation level and the number of extractions. Further, when there are a plurality of product keywords stored in association with the one or more product type keywords, a plurality of product keywords are extracted.

  When one or more product keywords are extracted, the matching degree of each product keyword is calculated (S94). The degree of matching is a numerical value indicating the degree of whether or not the set of the one or more product type keywords extracted indicates each product keyword. Specifically, for each extracted product type keyword, the score of the item to which each product type keyword belongs is multiplied by the coefficient at the relevant level of each product type keyword, and the multiplication value of all the product type keywords is obtained. The total numerical value is used as the matching degree (matching degree = Σ {scoring of item × related level coefficient}).

  Once the degree of matching of each extracted product keyword is calculated, it is determined whether or not the degree of matching of each product keyword has reached a predetermined threshold (S95). When the degree of matching of each extracted product keyword does not reach the predetermined threshold value (S96: No), the process proceeds to a non-product keyword extraction process (S99 to S102) described later. On the other hand, if any of the product keywords has reached the predetermined threshold (S96: Yes), the recommended level calculation unit 162 calculates the recommended level of the product keyword that has reached the predetermined threshold (S97). . Specifically, as shown in the following formula, the recommended level is determined based on the average value of the matching degree over a plurality of recording processes, the average value of customer satisfaction over the plurality of recording processes, and the number of recordings. (Recommended level = A · (average matching degree) × B · (average customer satisfaction−50) × C · (number of extractions) (0 ≦ (A, B, C) ≦ 1)). The “average value of matching degree over a plurality of recording processes” (average matching degree) here means the degree of matching calculated for the keyword by this keyword process and the keyword recorded in the past keyword process. In addition, the average value of each matching degree calculated in the above is shown, and “average value of customer satisfaction over a plurality of recording processes” (average customer satisfaction) is the keyword extracted in this keyword processing. Utterance interval The average value of the customer satisfaction and the customer satisfaction of each utterance section from which the keyword was extracted in the past keyword processing.If the keyword has never been recorded in the past, The matching value of this keyword processing and the customer satisfaction level of the utterance section are used as the above values as they are.

  Thereafter, the satisfaction keyword recording unit 161 records each product keyword as a satisfaction keyword in the product satisfaction keyword management table 91a in association with the related information (S98). Specifically, as related information, the item of each product keyword, the recommended level, the average value of the matching degree over a plurality of recording processes, the number of recordings, the average value of customer satisfaction over a plurality of recording processes, Record the date of the last utterance, the presence / absence of a purchased flag, the presence / absence of an uttered flag, and the elapsed utterance time. The presence / absence of the purchase flag indicates whether or not the product of the product keyword has been purchased, and is recorded based on the conversion data from the receipt printer 13 with reference to the conversion data. The last utterance date indicates the date of the last utterance, and the presence or absence of the utterance flag indicates whether or not the product keyword has been uttered by the clerk or customer on the current date (today's date), and the utterance elapsed time Indicates the elapsed time after the product keyword is uttered on the current date, and the voice data stored in the voice data storage unit 83 (strictly speaking, the voice data in the customer service section specified on the current date is voice-recognized. These are recorded on the basis of the text information obtained in the above. Thereby, this process is complete | finished.

  Next, the non-product keyword extraction process (S99 to 102) will be described. In the non-product keyword extraction process, first, the management server 15 (control unit 53) extracts non-product keywords included in the text information for each utterance section from the non-product keywords registered in the non-product keyword table 89b ( S99). For example, non-product keywords such as “Last week”, “Sapporo”, and “Ski resort” are extracted based on voice data of a statement “I went to a ski resort in Sapporo last week”. If no non-product keyword is extracted (S100: No), this process is terminated.

  On the other hand, when one or more non-product keywords are extracted (S100: Yes), the first keyword and the second keyword are determined from the extracted one or more non-product keywords (S101). Specifically, first, “personal relationships”, “work / school related”, and “hobby related” are collated in this order, and if it is determined that one non-product keyword falls into each main category, the determination is made. When this is done, this is determined as the first keyword.

  If the first keyword is determined, then the second keyword is determined. Specifically, non-product keywords other than the first keyword are queried in the order of “event-related”, “date-time related”, “identity-related”, “place-name-related”, and whether or not it falls under the main category To do. When it is determined that it corresponds, the corresponding non-product keyword is determined as the second keyword. Although not shown in this flowchart, if there is no non-product keyword determined as the first keyword or the second keyword, this process ends.

  When the first keyword and the second keyword are determined, the good impression level calculation unit 163 calculates the good impression level for the set of non-product keywords (the combination of the first keyword and the second keyword). More specifically, as shown in the following formula, calculation is made based on the average value of customer satisfaction over a plurality of recording processes and the number of times of recording (good impression level = B · (average customer satisfaction−50 ) × C · (number of extractions) (0 ≦ (B, C) ≦ 1)). Thereafter, the set of non-commodity keywords is recorded as a satisfaction keyword in the non-commodity satisfaction keyword management table 91b in association with the related information (S102). As related information, a good impression level, the number of times of recording, the average value of customer satisfaction over a plurality of times of recording, the date of last utterance, and the presence or absence of an uttered flag are recorded. The keyword recording process is terminated by this keyword process (S86) for each utterance section.

  Next, the recommended product notification process will be described with reference to the flowchart of FIG. The recommended product notification process is performed at the start of the service section (when the next voice data after the end of the service section is acquired). As shown in FIG. 19, the management server 15 (control unit 53) refers to the product satisfaction keyword management table 91a by the voice data generation unit 164 and generates text data for keyword notification (S111). Specifically, from the product satisfaction keyword management table 91a, a product keyword that does not have a purchase flag and an utterance flag and has the highest recommended level at the present time is extracted, and a message to which it is added, for example, “Recommended product is Create text data for the message "It is a stretch slim shirt." Next, based on the text data, keyword notification voice data for generating the message is generated (S112). When the voice data for keyword notification is generated, the voice data communication unit 105 transmits the voice data to the clerk terminal 5 (S113). The voice instruction earphone 3 that has received the keyword notification voice data via the clerk terminal 5 outputs the keyword notification voice data. Thereby, the recommended product is notified to the store clerk (S114). This is the end of this process.

  Next, the good impression keyword notification process will be described with reference to the flowchart of FIG. Like the recommended product notification process, the good impression keyword notification process is performed at the start of the service section (when the next voice data after the end of the service section is acquired). As shown in FIG. 22, the management server 15 (control unit 53) refers to the non-commodity satisfaction keyword management table 91b and generates text data for keyword notification (S116). Specifically, a set of non-product keywords having no utterance flag and the highest favorable impression level at the present time is extracted from the non-product satisfaction keyword management table 91b, and a message to which this is added, for example, “child” is the first. When one keyword, “Birthday Party” is the second keyword, text data of a message “The good impression keyword is“ Birthday Party ”of“ Children ”.” Is generated. Next, based on the text data, keyword notification voice data for generating the message is generated (S117). When the voice data is generated by the voice data communication unit 105, the voice data is transmitted to the clerk terminal 5 (S118). The voice instruction earphone 3 that has received the keyword notification voice data via the clerk terminal 5 outputs the keyword notification voice data (S119). This notifies the store clerk of the good impression keyword. This is the end of this process.

[Second Embodiment]
Next, with reference to FIGS. 23 to 26, only a different part of the customer service support system SY of the second embodiment will be described. In the customer service support system SY of the present embodiment, the management server 15 extracts a topic keyword for each conversation section in addition to the configuration of FIG. 4, and identifies a set of conversation sections in which the topic keyword does not change as a topic section. It has a topic section identification function and a topic section satisfaction notification function for notifying customer satisfaction in units of topic sections. Further, as shown in FIG. 23, the management server 15 includes a topic section specifying unit 203 that specifies a topic section in addition to the configuration of FIG. Furthermore, as shown in FIG. 24, the management server database DB functions as a section data storage unit 93 in addition to the functions of FIG. As shown in FIG. 9B, the customer satisfaction storage table 92 stores customer satisfaction in units of topic sections in addition to the data in FIG. 9A. The topic keyword is a keyword that becomes a topic in one conversation section, and the topic section is a set of continuous conversation sections in which the topic keyword does not change.

  Here, the topic section specifying process will be described with reference to the flowchart of FIG. Since the topic section specifying process is a process for specifying a topic section that is an aggregate of conversation sections, each time a conversation section is specified in the customer service section specifying process, a predetermined loop process (S122 to S137) is performed. It specifies the section.

  As shown in FIG. 25, when the conversation section is specified (S121: Yes), the management server 15 (control unit 53) performs loop processing (S122 to S137) with the specified conversation section as the target conversation section. Do. The conversation section targeted in the previous loop process is hereinafter referred to as the previous conversation section. In the loop processing, first, product keywords of each utterance section included in the target conversation section are extracted by the same method as in FIG. 20 (S91 to S98) (S122). Specifically, first, the voice data of each utterance section included in the conversation section is extracted by the voice data management table 85. When the voice data of each utterance section is extracted, the voice recognition unit 154 performs voice recognition of the voice data. Thereafter, the text information obtained as a speech recognition result of the speech recognition unit 154 is stored in the text information storage unit 88. When the text information is extracted and stored, one or more product type keywords (voice keywords) included in the text information are extracted from the product keyword table 89a (voice keyword extraction unit). Thereafter, product keywords are extracted based on the extracted one or more product type keywords (related keywords).

  Next, in the same method as in FIG. 20 (S99 to S102), a set of non-product keywords is extracted in each utterance section included in the conversation section (S123) (unit keyword extraction unit). Specifically, one or more non-product keywords included in the text information are extracted from the non-product keyword table 89b. After that, by determining the first keyword and the second keyword from the extracted non-product keywords, a set of non-product keywords is extracted.

  As a result, when any keyword (a combination of product keyword or non-product keyword) is extracted as the entire conversation section (S124: Yes), the combination of the product keyword and the non-product keyword in the entire conversation section The extracted numbers are compared (S125). When the number of extracted non-product keyword sets is large (S126: B), the set of non-product keywords with the largest number of extracted non-product keyword sets is determined as the topic keyword (S127). ). On the other hand, when the number of product keywords extracted is large (S126: A), each matching degree of one or more extracted product keywords is calculated (S128). If there is a product keyword whose matching degree exceeds a predetermined threshold among the extracted one or more product keywords (S129: Yes), the product keyword is determined as a topic keyword (S130).

  When no keywords (a combination of product keywords or non-product keywords) are extracted (S124: No), or when it is determined that there is no product keyword with a matching degree exceeding a predetermined threshold (S129: No) The topic keyword in the previous conversation section is determined as the topic keyword in the conversation section (S131).

  When the topic keyword of the target conversation section is determined, it is determined whether or not the topic keyword of the target conversation section is the same as the topic keyword of the previous conversation section (S132). When the topic keywords are the same (S133: Yes), the conversation section (strictly speaking, the start time and end time of the conversation section) is recorded in the section data storage unit 93 (S134), and this loop processing is terminated. (Returns to the conversation waiting state (S121)). On the other hand, if the topic keywords are not the same, the topic section specifying unit 203 specifies a set of one or more conversation sections stored in the section data storage unit 93 as one topic section (S135). Thereafter, all the conversation sections in the section data storage section 93 are deleted (S136), and the conversation section targeted this time is recorded in the section data storage section 93 (S137), and the loop processing is terminated (conversation section). Return to the specific waiting state (S121). As described above, the aggregate of conversation sections stored in the section data storage unit 93 is sequentially identified as the topic section.

  Next, the topic section satisfaction degree notification process will be described with reference to the flowchart of FIG. This topic section satisfaction degree notification process is performed every time a topic section is specified, that is, every time a topic section ends. As shown in FIG. 26, first, the management server 15 (the control unit 53) extracts the customer satisfaction of each conversation section included in the topic section using the customer satisfaction storage table 92 (S141). Next, based on the extracted customer satisfaction of each conversation section, the customer satisfaction of the topic section is calculated (S142) (section satisfaction calculation unit). Specifically, the average value of the customer satisfaction levels of each extracted conversation section is calculated as the customer satisfaction level of the topic section.

  When the customer satisfaction level of the topic section is calculated, text data for satisfaction notification based on the calculated customer satisfaction level is generated (S143). Specifically, as text data for notifying the satisfaction level, a message for notifying the customer satisfaction level of the topic section, for example, when the customer satisfaction level is “40”, “customer satisfaction level of the topic section“ 40 ””. Generate text data for the message. After the text data is generated, the voice data generation unit 164 generates voice data for satisfaction notification that utters the message based on the text data (S144). When the voice data for satisfaction level notification is generated, the voice data communication unit 105 transmits the voice data to the clerk terminal 5 (S145).

  The voice instruction earphone 3 that has received the satisfaction notification voice data via the salesclerk terminal 5 outputs the satisfaction notification voice data (S146), and the customer satisfaction of the topic section is given to the salesclerk. Notice. This is the end of this process.

  According to each embodiment described above, the customer satisfaction is calculated by calculating the customer satisfaction based on the voice data in the conversation between the store clerk and the customer and notifying the store clerk of the calculated customer satisfaction. be able to. Therefore, the customer service ability of the store clerk can be improved.

  Further, by outputting a voice indicating customer satisfaction, it is possible to easily notify customer satisfaction with a simple configuration.

  Furthermore, by configuring the voice data acquisition unit with the voice acquisition microphone 2 that is a wearable device, the voice of the clerk can be efficiently acquired. Further, by configuring the satisfaction notification unit with the voice instruction earphone 3 that is a wearable device, only the store clerk can be notified of the customer satisfaction, so that the customer does not feel uncomfortable. The term “wearable device” as used herein refers to a device that can be worn on a store clerk's body or clothes.

  Furthermore, the utterance section that reaches the predetermined threshold is specified, and the voice keyword (product type keyword) included in the voice data of the specified utterance section is extracted, and the related keyword (product keyword) related thereto is extracted. By notifying the store clerk, the store clerk can easily recognize what keywords are popular with customers. Further, since the voice keyword is not notified as it is, and the related keyword related thereto is notified, the voice keyword can be integrated into the related keyword and notified. That is, since the store clerk can omit the thought of analyzing the voice keyword and integrating it into the related keyword, the customer service efficiency can be improved. The abbreviation of the product may be stored in the keyword table 89, and the abbreviation of the product may be integrated into the product keyword and notified.

  In addition, according to the configuration of the first embodiment, the customer satisfaction level is notified after the conversation is interrupted because the customer satisfaction level in units of the conversation interval is notified at the end of the conversation interval where the conversation interval is not interrupted. . Therefore, the conversation with the customer is not hindered by the notification of customer satisfaction. In addition, since the customer satisfaction is calculated with a section where the talking section is not interrupted as one unit, the store clerk can easily recognize which section of the customer satisfaction is notified.

  In addition, according to the configuration of the second embodiment, it is possible to easily recognize which topic was popular or unpopular with the customer by notifying the customer satisfaction level in units of topic section at the end of the topic section. it can.

  In each of the above embodiments, the customer satisfaction level of the section is calculated and notified at the end of the conversation section or at the end of the topic section. However, at each end of the customer utterance section or at the end of the customer talk section The customer satisfaction degree of the section may be calculated and notified, or the customer satisfaction degree of the section may be calculated and notified at the end of the customer service section. Furthermore, at the end of the customer service section, a configuration for notifying customer satisfaction for each topic section or for each conversation section, or for a configuration for notifying customer satisfaction for each conversation section for each end of the topic section good.

  In each of the above embodiments, the product keyword is stored as the related keyword. However, the product type keyword (product type) is stored as the related keyword, and the product keyword is stored as the voice keyword. There may be. In such a case, when a product keyword is acquired as an audio keyword, it can be notified by being integrated with the product type keyword to which it belongs. For example, when a product keyword such as “stretch slim fit shirt” or “fine oxford shirt” is acquired, a “shirt” that is a product type keyword of the product type to which the product keyword belongs is notified. Furthermore, a configuration may be used in which catch phrases are stored as related keywords and a plurality of catch phrases of analogy expressions are stored as voice keywords. In such a case, when a catch phrase of a similar expression is acquired, it can be notified by being integrated with the catch phrase of the one. A catch phrase is an advertising phrase for customer service. For example, when a catch phrase having a similar expression such as “satisfying”, “satisfying”, “satisfying” is acquired, “satisfying” is notified.

  Further, in each of the above embodiments, the management server 15 generates voice data of a message notifying the customer satisfaction, and outputs the voice data by the voice instruction earphone 3 to notify the customer clerk of the customer satisfaction. However, a plurality of voice patterns are stored in advance, one voice pattern is selectively output from the plurality of voice patterns according to customer satisfaction, and customer satisfaction is notified. There may be. For example, as the customer satisfaction level, the customer satisfaction level is classified step by step, and a voice pattern table (see FIG. 27) for storing a beep sound pattern for each customer satisfaction level is provided. According to the calculated customer satisfaction level A beep sound pattern is output to notify customer satisfaction. Although not shown, as a beep sound pattern, a pattern “pi” is associated with the customer level 1, and a beep sound pattern “pi pi pi pi pi” is associated with the customer level 5. Remember.

  Further, in each of the above-described embodiments, the customer satisfaction is notified to the store clerk by voice output from the voice instruction earphone 3, but text data (character string) for satisfaction notification indicating the customer satisfaction is provided. It may be configured to display and notify the store clerk of customer satisfaction. For example, text data may be displayed by a head-mounted display worn by a store clerk, or text data may be displayed by a bracelet-shaped display device using electronic paper. In addition, if it is a business type such as the restaurant business, the text data may be displayed on the display screen of the handy terminal for ordering, or it may be a business type where customers are served through counters such as watches and jewelry stores. If so, a configuration may be adopted in which a small display is installed under the counter to display the text data.

  Furthermore, the structure which notifies customer satisfaction by the vibration according to customer satisfaction may be sufficient. For example, the customer satisfaction is notified by vibration according to the customer satisfaction by a vibration function or a separate vibrator mounted on the clerk terminal 5.

  In each of the above embodiments, the average value of the customer satisfaction in the utterance section is the customer satisfaction in the conversation section. However, the average value of the customer satisfaction is not limited to this, for example, the utterance section The maximum value, median value, minimum value, standard deviation, etc. of customer satisfaction may be used as the customer satisfaction level of the conversation section. The same applies to the case of calculating customer satisfaction in the topic section.

  3: voice instruction earphone, 15: management server, 53: control unit, 102: voice acquisition unit, 103: voice output unit, 154: voice recognition unit, 156: customer satisfaction calculation unit, 203: topic section identification unit, SY: Customer service support system

Claims (12)

An audio data acquisition unit for acquiring audio data in a conversation between a store clerk and a customer;
An audio data extraction unit for extracting the customer's audio data from the acquired audio data;
A customer satisfaction calculation unit for calculating customer satisfaction based on the extracted voice data of the customer;
A customer service support apparatus, comprising: a satisfaction notifying unit that notifies the store clerk of the customer satisfaction calculated by the customer satisfaction calculating unit.   The customer service support apparatus according to claim 1, wherein the satisfaction notifying unit notifies the customer satisfaction by outputting a voice indicating the customer satisfaction.   The satisfaction notification unit selectively outputs one voice pattern from a plurality of voice patterns according to the customer satisfaction calculated by the customer satisfaction calculation unit, thereby obtaining the customer satisfaction. The service support apparatus according to claim 2, wherein notification is provided.   The customer service support apparatus according to claim 1, wherein the satisfaction level notification unit notifies the customer satisfaction level by displaying a character string indicating the customer satisfaction level.   The customer service support device according to claim 1, wherein the satisfaction notifying unit notifies the customer satisfaction by vibration according to the customer satisfaction.   6. The customer service support device according to claim 1, wherein the voice data acquisition unit and the satisfaction notification unit are configured by a wearable device attached to the store clerk. The customer satisfaction calculation unit
When a continuous speech section is defined as one utterance section, an individual satisfaction calculation unit that calculates the customer satisfaction for each utterance section;
A set of the utterance sections of the store clerk or the customer that is repeated without interruption for a predetermined time or more, and a set of the conversation sections that the clerk of the store clerk and the customer repeats alternately without interruption for a predetermined time or more. When the body is one conversation section, a section satisfaction degree calculation unit that calculates the customer satisfaction statistic for each conversation section as a customer satisfaction degree for each conversation section, and
The customer service support apparatus according to claim 1, wherein the satisfaction level notification unit notifies the customer satisfaction level in units of the conversation section at each end of the conversation section. A related keyword storage unit for storing related keywords related to a predetermined voice keyword;
Among conversation sections that are continuous voice sections included in the voice data, a section identifying unit that identifies a conversation section in which the customer satisfaction has reached a predetermined threshold;
A voice recognition unit that recognizes voice data of the identified conversation section;
A speech keyword extraction unit that extracts speech keywords included in the speech data of the conversation section based on a recognition result of the speech recognition unit;
A related keyword extraction unit that extracts a related keyword related to the extracted voice keyword from the related keyword storage unit;
The service support apparatus according to claim 7, further comprising a related keyword notification unit that notifies the salesclerk of the extracted related keyword.   The related keyword storage unit stores at least one of a product name of a product estimated by the voice keyword, a product type of the product, and a catch phrase as the related keyword. 8. A customer service support device according to 8. A topic keyword storage unit for storing topic keywords classified by topic;
When a continuous speech segment is defined as one speech segment, a speech recognition unit that recognizes the speech data in speech units,
A speech keyword extraction unit that extracts speech keywords included in speech data of the clerk and the customer based on a recognition result of the speech recognition unit;
Referring to the topic keyword storage unit, the topic keyword corresponding to the voice keyword extracted for each utterance section is identified, and a set of the utterance sections in which the classification to which the topic keyword belongs does not change is defined as a topic section. A topic section identifying unit to identify,
The satisfaction calculation unit
An individual satisfaction calculation unit for calculating the customer satisfaction for each utterance section;
A section satisfaction calculation unit that calculates the customer satisfaction statistic for each topic section as the customer satisfaction of the topic section;
The customer service support device according to claim 1, wherein the satisfaction degree notification unit notifies the customer satisfaction degree in units of the topic section every time the topic section ends. A customer service support method for a customer service support device for supporting customer service by a store clerk,
The customer support device is a voice data acquisition step of acquiring voice data in a conversation between a store clerk and a customer;
A voice data extraction step of extracting the customer from the acquired voice data;
A customer satisfaction calculation step of calculating customer satisfaction based on the extracted voice data of the customer;
A customer service support method, comprising: performing a customer satisfaction level calculation step of notifying the customer clerk of the customer satisfaction level calculated by the customer satisfaction level calculation step.   The program for making a computer perform each process in the customer service support method of Claim 11.

Images