JP5477153B2 - Service data recording apparatus, service data recording method and program - Google Patents

Description

  The present invention relates to a customer service data recording apparatus, a customer service data recording method, and a program for recording customer service data for use as marketing data.

  In recent years, attempts have been made to collect customer data and sales data as marketing data for use in sales promotion. For example, Patent Document 1 proposes a marketing data collection / analysis technique that analyzes POS data to quantitatively grasp a consumer's co-sale tendency and reflects it in sales promotion.

  On the other hand, as a technique for estimating a conversation situation, an emotion recognition technique for recognizing an emotion based on a person's voice is known. For example, in Patent Document 2, emotion recognition is performed by associating changes in speech intensity, tempo, and intonation with respective emotional states such as “anger”, “sadness”, and “joy” from a speech signal input by a subject. It is carried out.

  By the way, in the field of marketing, it is said that increasing customer satisfaction at the expense of productivity and efficiency is better as a result of consumer repeaters. Also, retailers and other stores that emphasize hospitality have a close relationship between sales and customer satisfaction, so customer service skills training and customer impressions are improved to increase customer satisfaction. For training of smiles and greetings. In particular, conversation techniques, which are one of customer service skills, are regarded as important, and training focusing on the ratio of conversations with customers is conducted. Generally speaking, it is said that the ratio of the conversation between the store clerk and the customer is preferably larger (for example, 2: 8), and maintaining this will have a positive impact on customer satisfaction. It has been broken.

JP 2007-94592 A JP 2002-91482 A

  However, even if the conversation ratio that is preferable for customer service is known, there is no method for measuring the conversation ratio, so it is unclear whether or not each store clerk can practice it. Also, it is not clear how much the conversation ratio affects customer satisfaction. For this reason, there is a problem that even if the sales are improved by performing training focusing on the conversation ratio, the relevance cannot be proved. In the retail industry, there is a demand to collect information on the conversation ratio and customer satisfaction during the actual customer service as marketing data and use it for management strategy, but it has not been realized yet.

  In view of the above problems, the present invention provides a customer service data recording device, a customer service data recording method, and a program capable of recording customer service data for grasping the relationship between conversation ratio and customer satisfaction as one of marketing data. The purpose is to provide.

The customer service data recording device of the present invention includes a conversation acquisition unit that acquires a conversation between a clerk and a customer, a clerk talk section that is a voice section by a clerk from the acquired conversation, and a customer talk that is a voice section by a customer talk and spoke section extracting unit extracts a section, and a conversation ratio calculation unit for calculating data of a conversation ratio between the length of the clerk spoke length and customer talk period interval, based on the voice of the customer talk period, the customer and recognizing an emotion customer emotion recognition unit, on the basis of the recognition result of the customer emotion recognition, and customer satisfaction calculation unit for calculating data of customer satisfaction, calculated customer satisfaction and the calculated conversation ratio of data A service data recording unit that records service data associated with the data in a database;
Here, the conversation ratio is the ratio of the length of the clerk talk section to the length of the customer talk section, the ratio of the length of the clerk talk section to the sum of the length of the clerk talk section and the length of the customer talk section, or the customer talk section It is desirable that the ratio is the ratio of the length of the clerk talking section to the total length of the customer talking section.

  The customer service data recording method of the present invention includes a conversation acquisition step in which a computer acquires a conversation between a store clerk and a customer, a store clerk talk section that is a speech section by a store clerk from the acquired conversation, and a voice section by a customer talk. Talk section extraction step for extracting a certain customer talk section, ratio of the length of the clerk talk section and the customer talk section, or the length of the clerk talk section or the customer talk section length of the clerk talk section and the customer talk section Based on the recognition result of the conversation ratio calculation step for calculating the conversation ratio that is the ratio to the total of the length, the customer emotion recognition step for recognizing the customer's emotion based on the voice of the customer talk section, and the customer emotion recognition step, Customer satisfaction calculation step to calculate customer satisfaction and conversation based on the calculated conversation ratio And rate data, associating the satisfaction data based on customer satisfaction, and executes and a customer data recording step of recording in the database as customer data.

According to these structures, since the customer service data which linked | related conversation ratio data and satisfaction data are recorded, this can be utilized as marketing data. In addition, the customer service data can estimate how much the conversation ratio affects customer satisfaction, and can prove the results of conversation training. Further, by using the present invention individually, it can be used to improve conversation skills (conversation techniques) between individuals. The conversation ratio is as follows: (1) The length of the clerk talk section is La and the length of the customer talk section is Lb, the ratio of La and Lb, and (2) the ratio of La or Lb to (La + Lb) It can be calculated as either.
The voices of the store clerk and the customer are not necessarily acquired by one conversation acquisition unit (such as a microphone), and may be acquired individually by two conversation acquisition units. In this case, the talk section extraction unit distinguishes the voices of the store clerk and the customer by the acquired conversation acquisition section, and extracts each talk section.

  The customer service data recording device described above further includes an utterance detection unit that is attached to a store clerk and detects that the store clerk uttered, and the talking section extraction unit is included in the conversation based on the detection result of the utterance detection unit. It is preferable to determine whether the voice to be played is a store clerk's voice or a customer's voice, and to extract each talking section based on the determination result.

  In the customer service data recording method described above, the computer is attached to a store clerk and connected to an utterance detection unit that detects that the store clerk uttered, and the talking section extraction step is based on the detection result of the utterance detection unit. Thus, it is preferable to determine whether the voice included in the conversation is a store clerk voice or a customer voice, and extract each talking section based on the discrimination result.

According to these configurations, by using the utterance detection unit, it is possible to accurately determine the voices of the store clerk and the customer, and thus more accurately calculate the conversation ratio and the customer satisfaction.
As the utterance detection unit, for example, a body conduction sensor that detects body conduction sound that a person's voice reaches the body surface through bones and meat can be cited. In this case, it is preferable that the body sound sensor is mounted on the body surface of the head and neck.

  In the customer service data recording apparatus described above, when the continuous speech section without breathing is one utterance section and the group of clerk or customer utterance sections repeated without interruption for a predetermined time or more is one talk section, the conversation ratio It is preferable that the calculation unit calculates the total length of the speech sections included in one talk section as the length of each talk section.

  According to this configuration, the conversation ratio can be calculated based on the total length of the utterance sections of the store clerk and the customer. That is, when the speech section is interrupted due to breathing or the like, the length of the salesclerk talk section and the length of the customer talk section can be specified excluding the interval.

  In the customer service data recording device described above, when the conversation section of the store clerk and the customer is alternately repeated without interruption for a predetermined time or more as one conversation section, the conversation ratio calculation unit is included in the conversation section Based on one or more talking sections, the conversation ratio is calculated for each conversation section, and the customer satisfaction calculation unit calculates the customer for each conversation section based on the customer satisfaction of each customer talking section included in the conversation section. It is preferable to calculate the degree of satisfaction.

  According to this configuration, since the conversation ratio is calculated for each conversation section that is an aggregate of a series of talk sections, it is possible to calculate a conversation ratio with higher reliability as compared to the case where the conversation ratio is calculated for each unit time. Moreover, since the customer satisfaction is calculated in the same section as the conversation ratio, the correlation between the two can be estimated more accurately.

  In the customer service data recording apparatus described above, the customer emotion recognition unit performs emotion recognition in units of utterance intervals, and a customer satisfaction calculation unit calculates customer satisfaction in units of utterance intervals, The degree is preferably an average value of customer satisfaction levels for each utterance section included in the customer talk section.

  According to this configuration, since emotion recognition is performed in units of utterance sections, emotion recognition can be performed with higher accuracy than in the case where emotion recognition is performed in units of talk sections or conversation sections.

  In the customer service data recording apparatus described above, when a set of conversation sections that are repeated without interruption for a predetermined time or more is defined as one customer service section, the conversation ratio calculation unit calculates the average conversation ratio of each conversation section included in the customer service section. The customer satisfaction calculation unit calculates the average value of customer satisfaction of each conversation section included in the customer service section as customer satisfaction in the customer service section, and calculates customer service data. The recording unit records the conversation ratio in the customer service section and the conversation ratio in each conversation section as the conversation ratio data, and the customer satisfaction in the customer service section and the customer satisfaction in each conversation section as the satisfaction data, Is preferably recorded.

  According to this configuration, the conversation ratio in each conversation section is recorded as the conversation ratio data, and the customer satisfaction in each conversation section is recorded as the satisfaction degree data. Change and customer emotion change. Moreover, since the conversation ratio and customer satisfaction in each customer service section are recorded as customer service data, it is possible to easily grasp the overall evaluation of each customer service.

  The customer service data recording apparatus described above further includes a screen display unit that displays a viewer screen for confirming customer service data, and the screen display unit is configured by selecting or inputting person identification information for identifying a store clerk and / or a customer. It is preferable that the customer service data including the person identification information is extracted and displayed on the viewer screen.

  According to this configuration, desired customer service data can be confirmed on the viewer screen by selecting or inputting person identification information for identifying at least one of a store clerk and a customer as a search condition.

  In the customer service data recording apparatus described above, the screen display unit graphs the transition of the conversation ratio in each conversation section and the transition of customer satisfaction in each conversation section on the common time axis in the service section. It is preferable to display the superimposed graph on the viewer screen.

  According to this configuration, it is possible to grasp at a glance a change in the conversation situation within one customer service, a change in the customer's emotion, and the correlation between them in the overlay graph.

  The program of the present invention causes a computer to execute each step in the service data recording method described above.

  By using this program, it is possible to realize a customer data recording method capable of recording customer data for grasping the relationship between the conversation ratio and customer satisfaction as one of the marketing data.

1 is a system configuration diagram of a customer service support system according to a first embodiment. It is a control block diagram of the terminal for shop assistants. It is a control block diagram of a receipt printer. It is a control block diagram of a management server. It is explanatory drawing of a speech section, a talk section, a conversation section, and a customer service section. It is a functional block diagram of the customer service support system concerning a 1st embodiment. It is explanatory drawing of the database for management servers which concerns on 1st Embodiment. It is a figure which shows the calculation algorithm of a conversation ratio. It is a figure which shows an example of an audio | voice data management table, a salesclerk utterance area management table, and a customer utterance area management table. It is a flowchart which shows the audio | voice data preservation | save process which concerns on 1st Embodiment. It is a flowchart which shows the service area specific process which concerns on 1st Embodiment. It is a flowchart which shows the shop assistant talk area specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area B specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area A specific process which concerns on 1st Embodiment. It is a figure which shows the measuring method of the frequency | count of duplication. It is a figure which shows an example of a viewer screen (conversation ratio list). It is a figure which shows an example of a viewer screen (conversation ratio-sales performance correlation diagram). It is a figure which shows the calculation algorithm of a customer service score, a conversation ratio determination table, and a duplication count determination table. It is a functional block diagram of the customer service support system which concerns on 2nd Embodiment. It is explanatory drawing of the database for management servers which concerns on 2nd Embodiment. It is a figure which shows the calculation algorithm of customer satisfaction. It is a figure which shows an example of a viewer screen (satisfaction-conversation ratio list). It is a figure which shows an example of a viewer screen (satisfaction-conversation ratio overlay graph). It is a functional block diagram of the customer service assistance system which concerns on 3rd Embodiment. It is explanatory drawing of the database for management servers which concerns on 3rd Embodiment. It is a figure which shows an example of switching detection data, and an image figure of a customer service switching period. It is a figure which shows an example of a service conversation area specific result, and an image figure of a service conversation section. It is explanatory drawing of the service area identification pattern A. It is explanatory drawing of the service area specific pattern B. It is explanatory drawing of the customer service area specific pattern C. FIG. It is explanatory drawing of a service area setting.

[First Embodiment]
Hereinafter, a conversation ratio calculation device, a service data recording device, a conversation ratio calculation method, and a program according to the present invention will be described with reference to the accompanying drawings. In each embodiment shown below, the conversation ratio calculation apparatus of the present invention is exemplified for the case where it is applied to the customer service support system SY. The customer service support system SY acquires voices of people (customers, visitors) in stores and venues related to retail, restaurants and service industries, and uses the results to improve customer satisfaction and customer service skills. It was built accordingly. Thus, in each of the following embodiments, a case where the customer service support system SY is introduced into a retail store such as an apparel store or a jewelry store is illustrated.

  FIG. 1 is a system configuration diagram of a customer service support system SY1 according to the first embodiment. As shown in the figure, the customer service support system SY1 includes a body conduction sound sensor 1, a voice acquisition microphone 2 and a clerk terminal 5 attached to a clerk, and in-store cameras 11 ( In the figure, only one unit is shown), a POS (Point Of Sales) terminal 12 and a receipt printer 13 installed at the checkout counter 14, a management server 15 and a display terminal 16 installed in the backyard of the store. Become. In addition, the computer in a claim comprises each apparatus other than the body-conduction sound sensor 1.

  The body-conducting sound sensor 1 is mounted on the head and neck of the store clerk, and detects the body-conducted sound that the store clerk's voice travels through the bones and meat and reaches the body surface. In the present embodiment, the voice acquired by the voice acquisition microphone 2 is used to identify whether it is that of a store clerk (person A) or a customer (person B). The voice acquisition microphone 2 is attached to the clerk's uniform (near the chest) and acquires the clerk's and customer's voices. In place of the body-conducting sound sensor 1 and the voice acquisition microphone 2, microphones having directivity for the store clerk and the customer may be used. In other words, the clerk's voice and the customer's voice may be identified by using two microphones, one for obtaining the clerk's voice and the other for obtaining the customer's voice.

  The store clerk terminal 5 is attached to a store clerk's uniform (belt or the like), and is connected to the body conduction sound sensor 1 and the voice acquisition microphone 2 via a dedicated cable. In addition, it has a wireless communication function with the receipt printer 13, and exchanges information with the management server 15 via the receipt printer 13. The in-store camera 11 is provided on the ceiling and walls of various places in the store, and images customers who have come to the store as well as clerk and customers who are serving customers. As the in-store camera 11, a CCD camera or a PTZ (Pan Tilt Zoom) camera can be used.

  The POS terminal 12 has a general register configuration, and performs accounting processing according to the POS application. Further, a product code is obtained from a bar code scanner or a keyboard (not shown), and receipt data for printing on the accounting receipt R (see FIG. 3) is generated with reference to the product master 18. The product master 18 may be provided in a POS server (not shown) connected to the POS terminal 12.

  The receipt printer 13 is connected to the POS terminal 12 via a dedicated cable, and prints receipt print data acquired from the POS terminal 12 on receipt paper. The receipt printer 13 has a wireless communication function with the clerk terminal 5 and a wired communication function with the management server 15. In this way, by inputting / outputting various information with the receipt printer 13 as the main (the receipt printer 13 filters the acquired various information and outputs necessary information), the POS backbone network (POS terminal 12). Network) is not affected. Further, when applying the present invention to an existing POS system, it is not necessary to change the POS backbone network itself.

  The management server 15 is connected to the receipt printer 13 via a network 19 such as an intranet, and exchanges various information with the clerk terminal 5 via the receipt printer 13. Further, based on the voice data acquired from the clerk terminal 5, the voice data is recorded, the conversation ratio is calculated, and the number of times of duplication is measured. Further, the management server 15 displays the calculated conversation ratio and the viewer screen D (see FIG. 16 and the like) for confirming the recorded voice data on the display screen 16 a of the display terminal 16.

  Next, the hardware configuration of the clerk terminal 5, the receipt printer 13, and the management server 15 will be described with reference to FIGS. FIG. 2 is a control block diagram of the clerk terminal 5. The clerk terminal 5 includes a wireless LAN antenna 21, a wireless LAN RF (Radio Frequency) unit 22, a wireless LAN modem unit 23, and a wireless LAN baseband unit 24 for realizing a wireless communication function with the receipt printer 13. . The wireless LAN baseband unit 24 stores a MAC address for identifying the clerk terminal 5. In addition, the store clerk terminal 5 includes an amplifier unit 28 and an A / D converter 29 for acquiring the detection result of the body conduction sound sensor 1, and an amplifier unit 32 and A for acquiring audio data from the audio acquisition microphone 2. / D converter 33.

  The clerk terminal 5 includes a control unit 25 that performs overall control of each unit, a memory 26 that stores various data including the firmware (including voice data from the voice acquisition microphone 2), and a clerk terminal 5. A battery 34 for supplying power. The control unit 25, based on the detection data and the voice data acquired from the A / D converter 29 and the A / D converter 33, a store clerk utterance section specifying function for specifying the clerk's utterance section (continuous voice time zone), A sound level determination function for determining a sound level based on the sound data.

  FIG. 3 is a control block diagram of the receipt printer 13. The receipt printer 13 includes a wireless LAN antenna 41, a wireless LAN RF unit 42, a wireless LAN modem unit 43, and a wireless LAN baseband unit 44 for realizing a wireless communication function with the clerk terminal 5. The wireless LAN baseband unit 44 stores a MAC address for identifying the receipt printer 13. The receipt printer 13 includes an input interface unit 45 for receiving receipt data from the POS terminal 12, a CG-ROM 46 for storing character patterns, a control unit 47 for performing overall control of each unit, a print head, and a head drive mechanism. And a printing mechanism 48 including a receipt paper transport mechanism, and a wired LAN interface unit 49 connected to the management server 15 via a wired LAN.

  The control unit 47 includes a main processing unit 47a that performs analysis of receipt data including a predetermined command, generation of print data to be printed on the accounting receipt R, and a receipt data semantic analysis unit that is a configuration unique to the present embodiment. 47b. The receipt data semantic analysis unit 47b recognizes the terminal number of the POS terminal 12, the receipt number, the product code, the product name, the product unit price, the amount of money, the operator name, and the like from the receipt data, and can be interpreted by the management server 15 serving as the host system. To a predetermined data format (for example, XML format). In addition, what converted the recognition result of the receipt data into a predetermined data format is hereinafter referred to as “converted data”. In addition, the control unit 47 transfers the voice data received from the clerk terminal 5 via the wireless LAN (voice data acquired from the wireless LAN baseband unit 44) to the management server 15 via the wired LAN interface unit 49. To do.

  FIG. 4 is a control block diagram of the management server 15. The management server 15 acquires audio data and conversion data from the receipt printer 13 and also displays a wired LAN interface unit 51 for acquiring video data from the in-store camera 11 and a display for displaying various information on the display terminal 16. The processing unit 52 includes a control unit 53 that acquires input data from an input device 55 such as a mouse or a keyboard and performs overall control of each unit, and a storage unit 54 that stores various types of information. Based on the acquired audio data, the control unit 53 determines a customer service section specifying function for specifying a customer service section (a time zone during which the store clerk is serving customers), and a conversation ratio between the customer clerk and the customer per customer service section. A conversation ratio calculation function to be calculated, an overlap count measurement function that measures the number of overlaps of conversation per service section, and a viewer that controls display of the viewer screen D (see FIG. 16 and the like) based on information input from the input device 55 And a screen display control function. The storage unit 54 is realized by a hard disk, and stores various control programs such as a voice extraction program and a conversation ratio calculation program, and also functions as a management server database DB1 described later.

  Next, the definition of the voice section will be described with reference to FIG. First, a section of speech continuously uttered by the same person (a store clerk or a customer), that is, a section of one phrase that does not contain a breath or the like is referred to as an “utterance section”. In the present embodiment, emotion recognition and voice recognition are performed for each utterance section. Further, as shown in FIG. 5A, a collection of clerk or customer utterance sections that are repeated without interruption for a predetermined time or longer is referred to as a “talking section”. That is, it refers to a set of one or more utterance sections whose interval is less than the predetermined time X (where X is a constant that satisfies X> 0). In the example shown in the figure, a store clerk talk section (hereinafter referred to as “store clerk talk section”) and two customer talk sections (hereinafter referred to as “customer talk section”) around the store clerk talk section. The case where all are composed of two utterance sections is shown.

  Similarly, as shown in FIG. 5A, a group of talking sections in which the talking sections of the store clerk and the customer are alternately repeated without being interrupted for a predetermined time or longer are referred to as “conversation sections”. That is, it refers to an aggregate of one or more talking sections whose interval is less than the predetermined time Y (where Y is a constant satisfying Y ≧ X). In the present embodiment, a group of customer talk sections before and after the store staff talk section (that is, a collection of at least one and up to three talk sections) is expressed as “one conversation pattern” = “one conversation section”. Define.

  In addition, as shown in FIG. 5B, a collection of conversation sections that are repeated without interruption for a predetermined time or longer is referred to as a “service section”. That is, it refers to an aggregate of one or more conversation sections whose interval is less than the predetermined time Z (where Z is a constant such that Z> Y). The example of the figure shows a service section 1 composed of two conversation sections and a service section 2 composed of three conversation sections. Thus, the number of conversation sections included in the customer service section is arbitrary. Note that it is preferable that the value of Z is obviously large compared to the values of X and Y, such as several minutes to several tens of minutes.

  Next, with reference to FIG. 6 and FIG. 7, the functional configuration of the customer service support system SY1 according to the first embodiment will be described. FIG. 6 is a block diagram of the customer service support system SY1. The in-store camera 11 has a customer service imaging unit 111 as a main functional configuration. The customer service imaging unit 111 captures a customer service situation including a clerk and a customer. In the present embodiment, the customer service imaging unit 111 always performs imaging, and the video data is output to the management server 15 as needed.

  The body sound sensor 1 has an utterance detection unit 101 as a main functional configuration. The utterance detection unit 101 detects that the clerk uttered and the utterance section based on the body conduction sound. The voice acquisition microphone 2 has a conversation acquisition unit 102 as a main functional configuration. The conversation acquisition unit 102 acquires sound (voice signal) based on the conversation between the store clerk and the customer. The clerk terminal 5 has an audio data transmission unit 105 as a main functional configuration. The audio data transmission unit 105 determines the presence / absence of audio using a power filter in the audio level determination function, and transmits audio data of a predetermined audio level or higher (for example, 1.5 v or higher after amplifier amplification) to the management server 15. . Further, the store clerk utterance section is specified based on the detection result of the utterance detection unit 101 and the voice acquisition result of the conversation acquisition unit 102 (store clerk utterance section specifying function), and the management server 15 is notified of the occurrence of the clerk utterance section. Note that the clerk terminal 5 and the management server 15 communicate via the receipt printer 13.

  The receipt printer 13 has a conversion data transmission unit 113 as a main functional configuration. The conversion data transmission unit 113 transmits the conversion data obtained by converting the receipt data output from the POS terminal 12 into the XML format to the management server 15.

  The management server 15 includes a video recording unit 151, a person identification unit 152, a voice data recording unit 153, a voice extraction unit 154, a conversation ratio calculation unit 155, a duplication count measurement unit 156, a converted data acquisition unit 157, as main functional configurations. A service data recording unit 158, a screen display unit 159, and a management server database DB1 are provided.

  The video recording unit 151 acquires video data from the customer service imaging unit 111 and records it in the management server database DB1. The person identifying unit 152 identifies a store clerk and a customer based on the facial feature amount included in the video data. For example, for a clerk, the clerk identification information and the clerk's facial feature quantity are associated in advance and stored in the management server database DB1 (see the clerk information storage unit 82 in FIG. 7). In addition, the face detection is performed by analyzing the imaging result of the in-store camera 11, the face feature amount calculated by normalizing the detected image of the face portion, and the face feature of the clerk stored in the management server database DB1. The amount is collated, and it is determined that the salesclerk has the highest similarity among them. Similarly, for the customer, the customer identification information and the facial feature amount of the customer are associated with each other and stored in the management server database DB1 (see the customer information storage unit 81 in FIG. 7), and the calculated customer face is stored. The feature quantity is compared with the face feature quantities of many customers stored in the management server database DB1, and it is determined that the customer has the highest similarity among them. Note that the clerk identification information and customer identification information, which are the identification results of the person identification unit 152, are linked and stored in the customer service data storage unit 88 described later.

  The voice data recording unit 153 records the conversation between the store clerk and the customer, that is, the voice data transmitted from the voice data transmission unit 105 (records in the management server database DB1). The voice extraction unit 154 extracts a clerk voice and a customer voice from the acquired conversation (voice data). Specifically, based on the detection result of the utterance detection unit 101, it is determined whether the voice included in the conversation is a store clerk voice or a customer voice, and each voice is extracted based on the determination result. Each voice is extracted in units of utterance sections or talk sections. Details will be described later.

  The conversation ratio calculation unit 155 indicates the conversation ratio calculation function of the control unit 53, and calculates the conversation ratio between the store clerk and the customer. Specifically, the conversation ratio is calculated for each conversation section, and the conversation ratio (average conversation ratio) for each service section is calculated based on the calculation result for each conversation section. The calculated conversation ratio is stored in the customer service data storage unit 88 as part of the customer service data. The conversation ratio calculation algorithm will be described later.

  The overlap number measurement unit 156 indicates the overlap number measurement function of the control unit 53, and measures the number of overlaps, which is the number of times that the clerk's voice and the customer's voice are overlapped in the customer service section (number of overlap sections). . The measured number of overlaps is stored in the customer service data storage unit 88 as part of the customer service data. An overlapping section detection method and an overlapping section measurement method will also be described later.

  The conversion data acquisition unit 157 acquires the conversion data transmitted from the conversion data transmission unit 113 of the receipt printer 13 and records it in the management server database DB1. The converted data is used to obtain sales information recorded as a part of the customer service data. The conversion data is recorded in the information included in the conversion data, such as customer identification information (membership number, etc.), receipt number (sales No.), total amount, etc. that can specify the presence or absence of sales and the sales amount. May be extracted and recorded as sales information, or all of the conversion data may be recorded in the management server database DB1.

  The customer service data recording unit 158 associates and manages customer service data including the clerk identification information and customer identification information, which are identification results of the person identification unit 152, and the calculation result of the conversation ratio calculation unit 155, for each customer service section. Record in the server database DB1. Note that the customer identification information and the clerk identification information are specified from the face feature amount as described above. Further, the clerk identification information and the MAC address of the clerk terminal 5 are stored in association with each other (see the clerk information storage unit 82 in FIG. 7), and the video data acquired by the management server 15 is associated with the audio data. Can be done.

  The screen display unit 159 displays a viewer screen D (see FIG. 16 and the like) for confirming the recorded customer service data on the display screen 16a. The viewer screen D will be described in detail later with a specific example.

  FIG. 7 is an explanatory diagram of the management server database DB1 according to the first embodiment. The management server database DB1 includes a customer information storage unit 81, a clerk information storage unit 82, an audio data storage unit 83, a video data storage unit 84, an audio data management table 85, a clerk utterance section management table 86, and a customer utterance section management table 87. And functions as a customer service data storage unit 88. The management server database DB1 may be provided for each store, or may be shared by a plurality of stores.

  The customer information storage unit 81 associates customer identification information (customer ID and the like), customer face feature amounts, and customer data (including personal information such as name, address, telephone number, date of birth, and sex). And remember. The store clerk information storage unit 82 stores store clerk identification information (store clerk ID and the like), the store clerk's facial feature quantity, and the MAC address of the store clerk terminal 5 in association with each other. Also, the voice data storage unit 83 stores the voice data always recorded by the voice data recording unit 153 together with a time stamp. In addition, the video data storage unit 84 stores video data always captured by the customer service imaging unit 111 together with a time stamp.

  The voice data management table 85 is a table in which the acquired voice data is recorded for each continuous voice section (hereinafter referred to as “continuous speech section”) without distinguishing between the store clerk and the customer (FIG. 9). (See (a))). Further, the store clerk utterance section management table 86 is a table in which store section utterance sections are recorded (see FIG. 9B). The customer utterance section management table 87 is a table in which customer utterance sections are recorded (see FIG. 9C).

  The customer service data storage unit 88 corresponds to the customer identification information and the clerk identification information, which are identification results of the person identification unit 152, and the voice data for the customer service section among the voice data stored in the voice data storage unit 83. Among the recorded data and the video data stored in the video data storage unit 84, the imaging data corresponding to the video data for the service section, and the conversation ratio of the service section, which is the calculation result of the conversation ratio calculation unit 155, The number of times of overlap of the customer service section, which is a measurement result of the overlap number measuring unit 156, the sales information indicating the presence or absence of sales by the customer service (generated within the customer service section or within a predetermined time after the end of the customer service section), and the sales amount, A customer service date and a customer service date and time indicating a customer service start time and a customer service end time are stored in association with each other. As for the sales information, the customer's facial feature amount calculated from the customer's video imaged by the in-store camera 11 installed at the checkout location and a large number of customer's information stored in the management server database DB1 in advance. It is possible to identify the customer service data to be associated with the customer identification information of the customer having the highest similarity from the facial feature amounts. It is also possible to specify customer service data to be associated with the customer identification information included in the conversion data transmitted from the receipt printer 13. When the converted data includes store clerk identification information (such as operator name or store clerk number), it is preferably linked to customer service data that matches both customer identification information and store clerk identification information.

  Next, various conversation ratio calculation algorithms (calculation formulas) will be described with reference to FIG. As shown in FIG. 8A, as the conversation ratio in the conversation section, three patterns of “store clerk: customer relative conversation ratio”, “store clerk conversation ratio”, and “customer conversation ratio” can be calculated. . For example, “relative conversation ratio of clerk: customer” is expressed as follows: La is the total length of each clerk talk section included in the conversation section, and Lb is the total length of each customer talk section included in the conversation section. La: Refers to the value of Lb. “Conversation ratio of store clerk” indicates a value of La / (La + Lb), and conversation ratio of a customer indicates a value of “Lb / (La + Lb)”. Here, “the length of the talking section” refers to the length from the start time to the end time of the talking section.

  Note that La may be defined as the total length of each clerk utterance section included in the conversation section. That is, as shown in FIG. 5A, the talk section may include the interval X, but La may be defined as a length obtained by subtracting the length of the interval. For example, in the case of the customer talk section A shown in FIG. 5A, La is the sum of the length from the start time to the end time of the utterance section 1 and the length from the start time to the end time of the utterance section 2. . The same applies to Lb.

  On the other hand, as shown in FIG. 8B, the conversation ratio in the customer service section can be calculated by the average value of the conversation ratios in the respective conversation sections included in the customer service section. Note that various statistics such as the maximum value, the minimum value, and the intermediate value may be calculated as the conversation ratio in the customer service section instead of the average value of the conversation ratio in each conversation section. Further, the conversation ratio in the customer service section also depends on which pattern is adopted as the conversation ratio in the conversation section (see FIG. 8A), and the “relative conversation ratio of the clerk: customer”, “conversation ratio of the clerk” and Three patterns of “customer conversation ratio” can be calculated.

  Moreover, you may use the algorithm of FIG.8 (c) as a modification of the calculation method of the conversation ratio in a service area. In this case, the “relative conversation ratio of store clerk: customer” is the sum of the lengths of the talker sections included in the customer service section is ΣLa, and the total length of the customer talk sections included in the customer service section is ΣLb. , ΣLa: refers to the value of ΣLb. Further, “the clerk conversation ratio” indicates a value of ΣLa / (ΣLa + ΣLb), and “customer conversation ratio” indicates a value of ΣLb / (ΣLa + ΣLb).

  Note that ΣLa may be defined as the total length of each clerk utterance section included in the customer service section in the same manner as La is defined as the total length of each clerk utterance section included in the conversation section. . The same applies to ΣLb.

  Next, the voice data management table 85, the clerk utterance section management table 86, and the customer utterance section management table 87 will be described with reference to FIG. FIG. 9A shows an example of the audio data management table 85. The voice data management table 85 includes “voice data No.” continuously assigned to each continuous utterance section (section including at least one utterance section) which is a continuous voice section that does not distinguish between the clerk and the customer's voice. “Recording start time” corresponding to the start time of the utterance section and “Recording end time” corresponding to the end time of the continuous utterance section and voice data based on the customer's utterance or voice data based on the clerk's utterance, or The “duplicate flag” indicating whether the voice data is based on both utterances and the “voice data storage destination address” for storing the voice data are associated with each other. For example, the voice data of “voice data No .: 201” is a continuous utterance section having a start time of 12:36:03 and an end time of 12:36:16. Indicates that the clerk ’s voice and the clerk ’s voice overlap.

  FIG. 9B is a diagram showing an example of the store clerk utterance section management table 86. The clerk utterance section management table 86 includes a “clerk utterance number” assigned to each clerk utterance section, a “clerk utterance start time” that is a start time of the clerk utterance section, and a “clerk utterance section end time” that is the end time of the clerk utterance section. End of the overlapping section of "Speech end time", "Speaking No." indicating which store clerk talk section belongs to, "Overlapping start time" which is the start time of overlapping section with customer utterance, and customer utterance The “duplication end time”, which is the time, is associated with each other. For example, the utterance sections of “Clerk utterance No .: 100” and “Clerk utterance No .: 101” are regarded as a series of talk sections because the interval is less than a predetermined time X (for example, 3 seconds), and are the same. Talk No. Is added. Further, the utterance section of “clerk utterance number: 100” indicates that the entire section overlaps with the customer's utterance.

  FIG. 9C is a diagram illustrating an example of the customer utterance section management table 87. The customer utterance section management table 87 includes “customer utterance No.” given to each customer utterance section, “customer utterance start time” which is the start time of the customer utterance section, and “customer utterance section end time”. End of the overlapping section of the “speech end time”, “speaking No.” indicating which customer speaking section it belongs to, “duplication start time” which is the start time of the overlapping section with the clerk utterance, and the clerk utterance The “duplication end time”, which is the time, is associated with each other. For example, the utterance sections of “customer utterance No .: 101” and “customer utterance No .: 102” are regarded as different talk sections because the intervals exceed a predetermined time X (for example, 3 seconds). Talk No. Is granted. Further, the utterance section of “customer utterance No .: 100” indicates that 6 seconds out of the section length of 13 seconds overlaps with the clerk's utterance.

  Next, the audio data storing process will be described with reference to the flowchart of FIG. As described above, the clerk terminal 5 and the management server 15 communicate with each other via the receipt printer 13, but the illustration is omitted because the receipt printer 13 simply passes information.

  When the store clerk terminal 5 (control unit 25) acquires an audio signal (audio) from the audio acquisition microphone 2 (S11), the audio level is determined by the power filter of the audio level determination function (S12). Here, if the audio level is equal to or higher than the predetermined level, it is determined that the audio is “present”, and buffering of audio data is started in an audio data storage area (not shown) in the memory 26 (S13). At this time, the recording start time is recorded in the audio data storage area.

  Subsequently, when no audio signal is received, the recording end time is determined and recorded in the audio data storage area, and the buffering is ended (S14). Thereafter, transmission of audio data is declared to the management server 15 (S15), and the audio data buffered in the audio data storage area is transmitted together with the recording start time and recording end time (S16).

  When the management server 15 (control unit 53) receives the voice data from the store clerk terminal 5 (S17), the voice data No. uniquely determined in the voice data management table 85 (see FIG. 9A) is obtained. Then, the recording start time and recording end time are registered (S18). Also, the audio data is stored in the audio data storage destination (predetermined folder) designated in the audio data management table 85 (S19).

  Next, with reference to the flowcharts of FIGS. 11 to 14, a series of flow of the service area identification process will be described. FIG. 11 is a flowchart showing the main process (service section identification process), and FIGS. 12 to 14 are flowcharts showing the subroutine. As shown in FIG. 11, the management server 15 (control unit 53) first identifies a store clerk talk section (S21), then identifies a customer talk section B generated after the store clerk talk section (S22), and also stores the store clerk talk. The customer talk section A that occurred before the section is specified (S23). Thereafter, a conversation section is specified based on the conversation section specified by the steps S21 to S23 (see S24, FIG. 5A). Further, by repeating the steps S21 to S24, the customer service section is specified (S25, see FIG. 5B).

  Next, with reference to the flowchart of FIG. 12, the clerk talk section specifying process corresponding to S21 of FIG. 11 will be described. When the clerk terminal 5 acquires the detection data from the body conduction sound sensor 1 (S31), the clerk terminal section determines the detection level by the power filter of the clerk utterance section specifying function. The time is determined and recorded in the memory 26 (S32). Subsequently, when the detection level is determined by the power filter of the store clerk utterance section specifying function and the state below the predetermined level is maintained for a certain period of time (when no signal section occurs), the detection level is above the predetermined level. The last detection time is determined as the clerk utterance section end time and recorded in the memory 26 (S33). Thereafter, the management server 15 is notified of the occurrence of the clerk utterance section (S34). At this time, the clerk utterance start time and the clerk utterance end time recorded in the memory 26 are also transmitted.

  On the other hand, when the management server 15 (control unit 53) receives the notification of the occurrence of the clerk utterance section from the clerk terminal 5 (S35), it is uniquely determined in the clerk utterance section management table 86 (see FIG. 9B). Sales clerk utterance data No. And the clerk talk number set for each clerk talk section. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereafter, it is determined whether or not a notification of occurrence of the next clerk utterance section is received within a predetermined time (S37). If received (S37: Yes), the clerk utterance data No. And the same clerk Talk No. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereby, the salesclerk utterance section generated before and the salesclerk utterance section generated this time can be defined as a series of talk sections. Note that, when the occurrence notification of the next clerk utterance section is not received within a predetermined time (S37: No), the process is terminated assuming that a series of talk sections by the clerk is over.

  Next, with reference to the flowchart of FIG. 13, the customer talk section B specifying process corresponding to S22 of FIG. 11 will be described. The management server 15 (the control unit 53) refers to the voice data management table 85 after the end of identification of a series of talk sections by the clerk, and the voice data is stored within a predetermined time from the clerk utterance end time of the last clerk utterance section. It is determined whether or not it exists (S41). Here, when there is no voice data (S41: No), it is determined that the customer talk section B does not exist, and the process ends. If the voice data exists (S41: Yes), the recording start time and the recording end time of the voice data are read from the voice data management table 85, and the uniquely defined customer utterance number. And customer talk No. determined for each customer talk section. Then, the customer utterance start time and customer utterance end time are registered in the customer utterance section management table 87 (S42).

  Thereafter, referring to the voice data management table 85, it is determined whether or not voice data exists within a predetermined time from the customer utterance end time of the last customer utterance section (S43), and if present (S43: Yes). Unique customer utterance No. And the same customer talk number as above. The customer utterance start time and customer utterance end time are registered (S42). Thereby, the customer utterance section generated before and the customer utterance section generated this time can be defined as a series of talk sections. If there is no voice data within a predetermined time from the customer utterance end time of the last customer utterance section (S43: No), the process is terminated assuming that there is no subsequent talk section by the customer.

  Next, with reference to the flowchart of FIG. 14, the customer talk section A specifying process corresponding to S23 of FIG. 11 will be described. The management server 15 refers to the voice data management table 85 after the identification of the customer talk section B is completed, and unprocessed voice data based on the customer's utterance from the clerk utterance start time of the clerk utterance section to a predetermined time before. Whether or not exists is determined (S51). Here, when there is no unprocessed audio data (S51: No), it is determined that the customer talk section A does not exist, and the process is terminated. If unprocessed voice data exists (S51: Yes), the recording start time and the recording end time of the voice data are read, and the uniquely defined customer utterance number. And customer talk No. determined for each customer talk section. The customer utterance start time and customer utterance end time are registered in the customer utterance section management table 87 (S52).

  Thereafter, referring to the voice data management table 85, it is determined whether or not unprocessed voice data exists before a predetermined time from the customer utterance start time of the registered customer utterance section (S53). S53: Yes), the recording start time and the recording end time of the voice data are read, and the customer utterance No. uniquely determined is read. And the same customer talk number as above. Then, the customer utterance start time and customer utterance end time are registered (S52). Thereby, the customer utterance section registered previously and the customer utterance section specified this time can be defined as a series of talk sections. If there is no unprocessed voice data within a predetermined time before the customer utterance start time of the registered customer utterance section (S53: No), the processing is ended assuming that there is no previous talk section by the customer. To do.

  As described above, registration into the voice data management table 85, the clerk utterance section management table 86, and the customer utterance section management table 87 is performed by the processes shown in FIGS. And the customer talk section B can be specified. Here, the detection of the overlap section between the store clerk talk section and the customer talk section will be briefly described. After specifying each talking section, the management server 15 refers to the store clerk utterance section management table 86 and the customer utterance section management table 87, detects an overlapping section, and records it in each table 85, 86, 87. Specifically, first, referring to the clerk utterance section management table 86, the same talk No. The earliest clerk utterance start time and the latest clerk utterance end time are set as the start time and end time of the clerk talk section. Similarly, referring to the customer utterance section management table 87, the same talk No. The earliest customer utterance start time and the latest customer utterance end time are set as the start time and end time of the customer talk section. Then, it is determined whether or not there is an overlapping section between the clerk talk section and the customer talk section. If there is a duplicate section, the clerk utterance section management table 86 and the customer utterance section management table 87 indicate the overlapping sections (duplication start time and duplication end time). Time) and a duplication flag is set in the audio data management table 85 (corresponding to “customer / store” in the duplication flag column of FIG. 9A).

  Next, a method for measuring the number of times of duplication will be described with reference to FIG. In the example shown in FIG. 15A, there are four overlapping sections in which the store clerk talk section and the customer talk section overlap. However, since the second overlapping section is a short utterance section (talking section of a predetermined time or less), it is not subject to overlap measurement. Therefore, in the example of the figure, three times of overlap measurement sections (1) to (3) are measured as the number of overlaps.

  Note that since the overlap measurement sections (2) and (3) are sections included in one talk section, this may be measured as one overlap measurement section. In this case, in the example shown in FIG. 15A, the number of times of duplication is measured as two.

  In addition, all overlapping sections of the talking section having a predetermined time or less may be subjected to overlap measurement. In this case, in the example shown in FIG. 15A, the number of times of duplication is measured as four.

  Further, as in the example shown in FIG. 15B, only the sections that are overlapped by talking from the store clerk may be subject to overlap measurement. In the example of the figure, there are four overlapping sections in which the store clerk talk section and the customer talk section overlap. However, since the second and fourth overlapping sections are overlapping sections due to a talk from the customer, they are not subject to overlapping measurement. That is, when a customer's talk starts in a state where the store clerk talk section is started in advance, even if an overlap section occurs, it is not subject to overlap measurement. Therefore, in the example of the same figure, two times of the overlap measurement sections (1) and (2) are measured as the number of overlaps. In this way, by setting only the section that overlaps due to the conversation from the store clerk as the object of overlap measurement, the store clerk's customer service method can accurately determine whether or not it is good.

  Next, a viewer screen D for confirming customer service data will be described with reference to FIGS. 16 and 17. The viewer screen D is displayed on the display screen 16a of the display terminal 16. FIG. 16 is a diagram showing a viewer screen D01 of the conversation ratio list. The viewer screen D01 includes an inquiry condition selection area E11 for selecting an inquiry condition, a list display area E12 for displaying a list of conversation ratios, and a reproduction operation area E13 for performing an operation for reproducing recorded data included in the customer service data. And are included.

  In the inquiry condition selection area E11, “Store”, “Date”, and “Clerk (person identification information)” can be selected (input). The customer service data that meets the conditions selected (input) here is displayed in the list display area E12. Note that it is possible to select (input) “customer (person identification information)” instead of “store clerk”, and display the customer service data associated with the customer in the list display area E12. Further, both “clerk” and “customer” can be selected (input), and customer service data satisfying these and conditions or conditions may be displayed in the list display area E12.

  The list display area E12 includes, for each customer service data, “store”, “date”, “store clerk”, “customer service section No.”, “customer service start”, “customer service end”, “relative conversation ratio”, “duplication number”. , “Customer”, “sales amount”, and “sales number” are related and displayed in a table format. “Service section No.” is an identification number automatically assigned to each service section, and “service start” and “service end” indicate the start time and end time of the service section. “Relative conversation ratio” and “number of duplications” refer to the relative conversation ratio and the number of duplications in the customer service section. “Customer” displays the name of the customer who is the object of the customer service, and “Sales Amount” and “Sales No.” are the total amount of the receipt R and the receipt No. obtained from the conversion data. Point to. In the list display area E12, any one of the customer service data (rows) can be selected (see the frame 211), and the recorded data included in the selected customer service data can be reproduced in the reproduction operation area E13. .

  The reproduction operation area E13 displays a button group 212 for performing a reproduction operation of recording data and imaging data in the service section, a progress bar 213 indicating a reproduction position, and a volume adjustment slider 214. Although not particularly illustrated, the management server 15 includes a reproduction unit (including an audio output unit such as a speaker) for reproducing the recording data and the imaging data in accordance with the operation of the reproduction operation area E13. The progress bar 213 has a time scale in minutes in the horizontal axis direction. In addition, the progress bar 213 distinguishes and displays the clerk talk section, the customer talk section, the overlap section, and the non-conversation section that does not correspond to any of these sections. In the figure, these can be identified by different hatches and blanks, but any display form can be used as long as the user can identify each section such as color-coded display, mark-added display, and text-added display. The scale of the progress bar 213 may be a time unit instead of a minute unit. Further, the unit of the scale and the scale interval may be varied according to the length of the recorded data so that the reproduction status of the entire recorded data in the customer service data can be grasped.

  FIG. 17 is a diagram showing a viewer screen D02 of a conversation ratio-sales performance correlation diagram. The viewer screen D02 includes an inquiry condition selection area E21 for selecting an inquiry condition, and a correlation diagram display area E22 for displaying a correlation diagram between the conversation ratio and the sales performance (sales information).

  The inquiry condition selection area E21 can select (input) “store”, “date section”, and “store clerk (person identification information)”. In addition, “clerk” can select “ALL (all clerk)”. The correlation diagram display area E22 is displayed based on the customer service data that matches the condition selected (input) here.

  The correlation diagram display area E22 is a scatter diagram in which the horizontal axis represents the customer conversation ratio (unit:%) and the vertical axis represents the customer unit price (unit: yen). Here, on the basis of all customer service data received on March 5 by all the store clerk, “points” that are the intersections of the average conversation ratio and sales information (customer unit price) are plotted. Thereby, the user can grasp at a glance how much the sales ratio is improved for the store. For example, in the example shown in the figure, it can be estimated that by increasing the customer conversation ratio to about 70%, the sales can be improved.

  As described above, according to the customer service support system SY1 according to the first embodiment, by calculating the conversation ratio between the store clerk and the customer, this can be collected and used for management strategy, or used for customer service skill training. can do. In addition, since the clerk identification information and the customer identification information are associated with the customer service data, it is possible to specify which store clerk is serving the customer for which customer. Thereby, appropriate customer service training can be performed for each store clerk. Furthermore, since sales information is associated with customer service data, the correlation between the conversation ratio and sales can be collected as marketing data.

  In addition, since the conversation ratio is calculated for each customer service section and displayed on the viewer screen D, whether or not each customer service is an overall favorable customer service (for example, the length of the clerk voice: the length of the customer voice is Whether or not the value is close to 2: 8 can be grasped. Further, since the recorded data is associated with the customer service data, for example, the recorded data associated with a preferable conversation ratio can be extracted and reproduced to be used as a service training material. That is, since the customer service situation of a salesclerk with high conversation skills can be efficiently shown to other salesclerks, it is useful for horizontal development of conversation skills.

  In addition, since the number of times of duplication is associated with the customer service data, it can be estimated whether or not the customer service is preferable from the number of times of duplication as well as the conversation ratio. For example, when the number of times of duplication is large, even if the conversation ratio is a preferable value, it can be determined that the customer service is not preferable.

  As a modification of the first embodiment, the number of customer service points based on the conversation ratio and the number of overlaps may be calculated and displayed on the viewer screen D. Hereinafter, the modification will be described with reference to FIG. As shown in FIG. 18A, the number of customer service points is calculated using “conversation ratio level” and “duplication count level” as parameters. Further, the “conversation ratio level” and the “duplication number level” are multiplied by weights P1 and P2, respectively, and the total value thereof becomes the number of customer service points. In general, the weighting is 0 ≦ P2 ≦ P1 ≦ 1, and P1 is larger than P2. In other words, the number of customer service points is calculated with a higher weight for the “conversation ratio level” than for the “duplication number level”. However, it is preferable that the user can arbitrarily set depending on the situation of the store used.

  Here, the “conversation ratio level” is higher as the conversation ratio between the clerk and the customer is closer to 2: 8. That is, as shown in FIG. 18B, the “conversation ratio level” is a numerical value of “0 to 3” depending on the customer conversation ratio. Further, the “duplication number level” is higher as the number of duplication is smaller. That is, as shown in FIG. 18C, the “duplication number level” is a numerical value of “0 to 3” depending on the number of duplications.

  Thus, by calculating the number of customer service points, it is possible to objectively determine the customer service. Further, by recording the determination result and the number of customer service points as part of the customer service data and displaying them on the viewer screen D, managers such as store managers and managers can check the customer service results at a glance.

  It should be noted that at each end of the conversation section or each end of the customer service section, a determination result as to whether or not the customer service is favorable and / or the number of customer service points may be notified to the target store clerk. In this case, the management server 15 determines the customer service and calculates the customer service score, and receives the information from the voice instruction earphone (not shown) attached to the store clerk via the receipt printer 13 and the store clerk terminal 5. It is preferable to notify. According to this configuration, since it is possible to grasp whether or not the store clerk is performing a favorable customer service during the customer service, an improvement in the customer service method can be expected.

  In the above embodiment, a group of customer talk sections before and after the store staff talk section (that is, a collection of at least one and up to three talk sections) is defined as “one conversation section”. The restriction on the number of talking sections included in “one conversation section” may be eliminated. That is, a group of talking sections in which the talking sections of the store clerk and the customer are alternately repeated without interruption for a predetermined time (interval Y) or more may be set as “one conversation section”.

  Moreover, although said embodiment illustrated the case where the conversation ratio of a salesclerk and a customer was calculated, the relationship between persons is not limited to this. That is, it may be a company boss and subordinate, a couple, a couple, or friends.

  In the above embodiment, the conversation ratio is calculated for each conversation section or customer service section. However, the conversation ratio within a predetermined time may be calculated. For example, the conversation ratio may be calculated based on the clerk and customer voices spoken during a predetermined time of 10 minutes. Moreover, you may calculate the conversation ratio in the daily work of a store clerk.

  In the above embodiment, the conversation ratio is calculated for each conversation section or for each customer service section. However, simply based on the conversation section of the adjacent store clerk and the customer (based on the ratio of the two conversation sections), The conversation ratio may be calculated.

  In the above embodiment, the person identifying unit 152 identifies the customer using the face recognition technique, but other methods may be employed. For example, each customer possesses a membership card with a built-in wireless IC (RFID: Radio Frequency Identification, etc.), and this is read with an IC reader installed at the entrance of the store to identify the customer, thereby identifying the customer You may do it. Further, the store clerk may be obliged to have a store clerk card in which a wireless IC is incorporated, and the store clerk may be identified by reading the store clerk card. By this method, when the read clerk card and the membership card are read at the same time, it is possible to associate the two as if the clerk is serving the customer. Further, the customer may be identified by reading the membership card (magnetic card) on which the magnetic information is recorded with a magnetic card reader attached to the POS terminal 12. In this case, the store clerk in charge of customer service can associate the customer with the store clerk by having his / her store clerk card read at the same time. The magnetic card reader may be directly connected to the management server 15. Furthermore, voice recognition technology may be used instead of face recognition technology. In this case, the customer information storage unit 81 and the store clerk information storage unit 82 need to store a voiceprint instead of the face feature amount.

  In the above embodiment, the imaging result of the in-store camera 11 is directly transmitted to the management server 15 via the wired LAN, but may be transmitted to the management server 15 via the receipt printer 13. Conversely, the clerk terminal 5 is configured to transmit audio data to the management server 15 via the receipt printer 13, but may be directly transmitted from the clerk terminal 5 to the management server 15. Each function of the management server 15 may be realized by a POS system or a WWW server.

[Second Embodiment]
Next, with reference to FIG. 19 thru | or FIG. 23, the customer service data recording device of this invention, the customer service data recording method, and a program are demonstrated as 2nd Embodiment. The customer service support system SY2 according to the present embodiment records customer service data in which conversation ratio data and satisfaction data are associated, and uses this as marketing data. Therefore, only differences from the first embodiment will be described. In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to 1st Embodiment is applied similarly about this embodiment.

  FIG. 19 is a functional block diagram of the customer service support system SY2 according to the second embodiment. In the management server 15 according to the present embodiment, compared to the first embodiment, a talk section extraction unit 161, a customer emotion recognition unit 162, and a customer satisfaction calculation unit 163 are added.

  The talk section extraction unit 161 corresponds to the voice extraction unit 154 of the first embodiment, and extracts a store clerk talk section and a customer talk section from the acquired conversation (voice data).

  The customer emotion recognition unit 162 applies the voice strength, the voice generation speed (number of mora per unit time, etc.), the strength for each word to the customer talking section extracted from the recorded data (voice data corresponding to the customer service section), Emotion recognition is performed based on the amount of change such as volume and voice spectrum. Specifically, emotion recognition is performed for each customer utterance section included in the recorded data. Thus, highly accurate emotion data can be obtained by performing emotion recognition for each phrase. Further, as shown in FIG. 15), the customer emotion recognition unit 162 identifies an overlapping section in which the customer talk section and the store clerk talk section overlap on the time axis, and identifies the overlap section as an “emotion recognition non-target section”. Then, emotion recognition is performed on the customer talk section excluding the overlap section. In this way, misrecognition can be prevented by performing emotion recognition excluding overlapping sections in which customer voice and store clerk voice are mixed and accurate emotion recognition cannot be performed.

  The customer satisfaction calculation unit 163 calculates customer satisfaction based on the recognition result of the customer emotion recognition unit 162. Further, as the customer emotion recognition unit 162 performs emotion recognition for each utterance section, the customer satisfaction calculation unit 163 also calculates the customer satisfaction for each utterance section. A specific method for calculating customer satisfaction will be described later.

  The customer service data recording unit 158 of the present embodiment associates the conversation ratio data based on the conversation ratio calculated by the conversation ratio calculation unit 155 and the satisfaction data based on the customer satisfaction calculated by the customer satisfaction calculation unit 163, Recorded in the management server database DB2 as part of the customer service data.

  FIG. 20 is an explanatory diagram of the management server database DB2 according to the second embodiment. The management server database DB2 is different from the first embodiment in the storage contents of the customer service data storage unit 91. The customer service data storage unit 91 of the present embodiment includes customer identification information, store clerk identification information, recording data, imaging data, sales information, customer service date, conversation ratio data based on the calculation result of the conversation ratio calculation unit 155, and customer satisfaction. The satisfaction degree data based on the calculation result of the degree calculation unit 163 is stored in association with each other. Here, the conversation ratio data refers to the conversation ratio in the customer service section and the conversation ratio in each conversation section. The satisfaction data refers to customer satisfaction in the customer service section and customer satisfaction in each conversation section.

  Next, a customer satisfaction calculation algorithm (calculation formula) will be described with reference to FIG. As shown in the figure, the customer satisfaction is calculated in the order of the utterance section, the conversation section, and the customer service section. First, as shown in FIG. 21A, the satisfaction level for each utterance section is calculated by the formula ““ satisfaction level for each utterance section ”=“ joy data ”+“ laughing data ”× A”. Here, “joy data” is emotion data of “joy” (for example, numerical values from 0 to 50, the same applies hereinafter), “laugh data” is emotion data of “laughter”, and A is a constant satisfying 0 ≦ A ≦ 1. Means. The calculation algorithm is derived from the idea that the degree of satisfaction of a person is based on the psychological state of “pleasant” and the multiplication result of the psychological strength.

  Further, as shown in FIG. 21 (b), the actual satisfaction level for each utterance section is calculated using the formula "" Real satisfaction level for each utterance section "=" Satisfaction level for each utterance section "-" Satisfaction level for each utterance section "× C" That is, “(“ joy data ”+“ laugh data ”× A) − (“ anger data ”+“ sad data ”× B) × C” is calculated. Here, “anger data” means “anger” emotion data, “sad data” means “sadness” emotion data, B means a constant satisfying 0 ≦ B ≦ 1, and C means a constant satisfying 0 ≦ C ≦ 1. To do. As described above, by using emotion data of “anger” and “sadness” in addition to “joy” and “laughter”, it is possible to calculate a more reliable satisfaction considering complex emotions. . The calculation algorithm is based on the idea that the person's dissatisfaction level is based on the psychological state of “uncomfortable” and the multiplication result of the psychological strength, and the psychological state where the actual satisfaction level is “pleasant” and “unpleasant”. It is derived from the idea of being based on.

  Further, as shown in FIG. 21C, the satisfaction level for each conversation section is ““ satisfaction level for each conversation section ”= an average value of“ actual satisfaction level for each utterance section ”of each customer utterance section included in the conversation section”. Obtained from. In addition, as shown in FIG. 21 (d), the satisfaction level by customer service section is obtained from ““ Satisfaction level by customer service section ”= average value of“ satisfaction level by conversation section ”of each conversation section included in the service section. It is done.

  Next, the viewer screen D according to the second embodiment will be described with reference to FIGS. FIG. 22 is a diagram showing a viewer screen D03 of the satisfaction level-conversation ratio list. The viewer screen D03 is displayed by, for example, a button operation (not shown) for displaying the “satisfaction-conversation ratio list” from the viewer screen D01 shown in FIG. Section No. indicating A display area E31 and a list display area E32 for displaying a list in which the customer satisfaction and the customer conversation ratio are associated are included. The list display area E32 includes “time” indicating the start time and end time of the service section, “conversation section No.” of each conversation section included in the service section, “customer satisfaction” of each conversation section, and each conversation. The “customer conversation ratio” of the section is related and displayed in a table format. Here, “customer satisfaction” of each conversation section refers to the satisfaction degree by conversation section shown in FIG.

  FIG. 23 is a diagram showing a viewer screen D04 of the satisfaction-conversation ratio overlay graph. The viewer screen D04 is displayed by, for example, a button operation (not shown) for displaying the “satisfaction-conversation ratio superimposed graph” from the viewer screens D01 and D03 shown in FIG. 16 or FIG. A customer service data display area E41 that displays part of the information included in the customer service data, and a graph display area E42 that graphs the relationship between customer satisfaction and the conversation ratio are included.

  The customer service data display area E41 includes “date”, “store clerk”, “customer”, “customer service time”, “sales number”, “sales amount”, “average customer conversation ratio”, “average customer satisfaction” and “Service section No.” is displayed. Here, “customer service time” refers to a customer service start time and a customer service end time. Further, the “average customer satisfaction” refers to the satisfaction by customer service section shown in FIG.

  The graph display area E42 includes a first line graph (shown by a solid line and a black circle) whose horizontal axis is “conversation section No.” and whose vertical axis is “customer satisfaction”, and whose horizontal axis is “conversation section No.”. Then, a superposition graph in which a second line graph (shown by broken lines and white circles) with the vertical axis as “conversation ratio” is overlaid is displayed. Here, the “conversation section No.” on the horizontal axis is arranged in time series. Instead of “conversation section No.”, the horizontal axis may be time (time). In addition, it is assumed that the “customer satisfaction” includes emotion data and values of constants A, B, and C as represented by numerical values of 0 to 100. The “conversation ratio” indicates the customer conversation ratio in unit%, and is represented by 0 to 100%. In this way, by displaying the transition of customer satisfaction in each conversation section and the transition of the conversation ratio in each conversation section within the customer service section in a graph on the common time axis, the user can receive one customer service. It is possible to grasp at a glance changes in conversation status, customer emotions, and their correlations.

  As described above, according to the customer service support system SY2 according to the second embodiment, the conversation ratio data and the satisfaction level data are associated with each other and recorded as the customer service data, so that it can be used as marketing data. In addition, the customer service data can estimate how much the conversation ratio affects customer satisfaction, and can prove the results of conversation training. In addition, the conversation ratio in each conversation section is recorded as conversation ratio data, and the customer satisfaction in each conversation section is recorded as satisfaction data, and these are displayed on the viewer screens D03 and D04, thereby providing conversation within one customer service. You can see changes in situations and customer sentiment. Furthermore, since the average conversation ratio and average customer satisfaction in each customer service section are recorded and displayed as customer service data (see E41 in FIG. 23), it is possible to easily grasp the overall evaluation of each customer service.

  In the above embodiment, the conversation ratio between the store clerk and the customer and the customer satisfaction are calculated for the purpose of customer service management, but they may be used by individuals. As a result, the collected conversation ratio data and satisfaction data can be used to improve conversation skills (conversation techniques) between individuals.

[Third Embodiment]
Next, with reference to FIG. 24 to FIG. 31, a customer service support apparatus, a customer service support method, and a program according to the present invention will be described as a third embodiment. The customer service support system SY3 according to this embodiment is characterized in that a customer service section is specified for each customer serving as a customer service based on a monitoring result of a monitoring unit that monitors a store clerk or a customer. Therefore, only differences from the above embodiments will be described. In the present embodiment, the same components as those in the above-described embodiments are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to said each embodiment is applied similarly about this embodiment.

  FIG. 24 is a functional block diagram of the customer service support system SY3 according to the third embodiment. In the present embodiment, the voice acquisition microphone 2 functions as a monitoring unit. That is, the monitoring unit includes a conversation acquisition unit 102 (other examples of the monitoring unit will be described later), and the conversation acquisition unit 102 acquires a conversation between a salesclerk and a customer.

  In addition, the management server 15 according to the present embodiment is different from the second embodiment in that the customer service switching detection unit 171, the switching detection data recording unit 172, the customer service switching segment identification unit 173, the customer service conversation segment identification unit 174, and the customer service segment identification. A part 175 is added.

  The customer service switching detection unit 171 detects the switching of the target customer who is the customer service target of the store clerk based on the monitoring result of the monitoring unit, that is, the voice of the customer included in the conversation acquired by the conversation acquisition unit 102. In the present embodiment, voice print authentication is periodically performed on the customer's voice, and the change of the target customer is detected based on a change in the voice print authentication result. Note that it is also possible to discriminate voice feature amounts (voice pitch, speed, etc.) other than the voiceprint from the customer's voice, and detect the change of the target customer based on the change in the feature amount.

  The switching detection data recording unit 172 associates the clerk identification information for identifying the clerk and the detection time (time stamp) of the customer service switching detection unit 171, and records them in the management server database DB 3 as switching detection data.

  Based on the recorded switching detection data, the service switching section identification unit 173 sets the Nth detection time (where N is an integer satisfying N ≧ 1) as the section start time and the N + 1th detection time based on the recorded switching detection data. The customer service switching section used as the section end time is specified. That is, the “customer service switching section” is a section specified based on the switching detection data.

  The service conversation section identification unit 174 identifies a service conversation section based on the conversation section extracted by the conversation section extraction unit 161. The “service section” according to the first embodiment and the second embodiment corresponds to the “service conversation section” according to the present embodiment. As described above, since the conversation section is a collection of conversation sections that are alternately repeated without interruption for a predetermined time or more by the store clerk and the customer, a conversation section that is repeated without interruption for a predetermined time or more is defined as 1 Specify as a customer service section. That is, the “customer service conversation section” is a section specified based on the recording data included in the customer service data.

  Further, the customer service section specifying unit 175 specifies the customer service section based on the customer service switching section specified by the customer service switching section specifying unit 173 and the customer service conversation section specified by the customer service conversation section specifying unit 174. Specifically, the customer service section is specified by performing an AND operation or an OR operation on the customer service conversation section and the customer service switching section. The customer service section identification unit 175 associates arbitrary switching detection data and recorded data with the store clerk identification information, and performs a comparison operation. A method for identifying a service section by the service section identifying unit 175 will be described in detail later.

  With the above configuration, the conversation ratio calculation unit 155 of the present embodiment calculates the conversation ratio in the service section specified by the service section specification unit 175. In addition, the customer service data recording unit 158 of the present embodiment corresponds to recorded data that is voice data corresponding to the customer service section specified by the customer service section specifying unit 175, or a customer service section specified by the customer service section specifying unit 175. Imaging data that is video data is recorded as customer service data. Further, the screen display unit 159 of the present embodiment, based on the user's operation, the customer service switching section specified by the customer service switching section specifying unit 173, the customer service conversation section specified by the customer service conversation section specifying unit 174, and the customer service section The customer service section specified by the specifying unit 175 is displayed on the viewer screen D (see, for example, FIGS. 26 to 30). Further, the start time and end time of the customer service section can be corrected on the viewer screen D based on the user's operation.

  FIG. 25 is an explanatory diagram of the management server database DB3 according to the third embodiment. The management server database DB3 further functions as a switching detection data storage unit 93 in addition to the units shown in the first and second embodiments. The switch detection data storage unit 93 stores the switch detection data recorded by the switch detection data recording unit 172. Further, the customer service data storage unit 94 of the present embodiment includes customer identification information, store clerk identification information, recording data corresponding to voice data for the customer service section, imaging data corresponding to video data for the customer service section, and conversation ratio in the customer service section. In addition, customer service section data is stored. The customer service section data refers to data including a section start time and a section end time of the service section.

  Next, the switching detection data and the switching detection period will be described with reference to FIG. As shown in FIG. 26A, the switching detection data is information in which “salesperson identification information”, “date”, and “switching detection time” are associated with each other. The “switching detection time” refers to the time when the customer service switching detection unit 171 detects the switching of the target customer. In the present embodiment, voice print authentication is periodically performed on the customer's voice, and when the voice print authentication result is changed (when a voice print of a different customer is recognized), it is determined that the target customer has been switched.

  FIG. 26B is an image diagram showing the customer service switching section on the time axis. The customer service switching section is defined as a new section generated every time switching is detected, and therefore, as shown in the figure, there is no gap between adjacent sections, and the section is a continuous section.

  Next, the customer service conversation section will be described with reference to FIG. FIG. 27A is a diagram illustrating an example of the service conversation section identification result. The service conversation section is specified by the service section specifying method described in the first embodiment. Here, “clerk identification information”, “date”, “customer identification information”, and “customer conversation section” are shown as specific results. Further, it is assumed that the reference length of the interval of the service conversation section (interval Z in FIG. 5B) is 1 minute 30 seconds.

  FIG. 27B is an image diagram showing the customer service section on the time axis. Since the customer service section is defined as a collection of conversation sections that are repeated without interruption for a predetermined time or longer, a gap occurs between adjacent sections as shown in FIG.

  Next, with reference to FIG. 28 to FIG. 30, a method for specifying the customer service section will be described. Here, as a method for specifying the service section, three patterns (service section specification patterns A to C) are illustrated here. FIG. 28 is a diagram illustrating a service section identification pattern A. The customer service section specifying pattern A specifies a customer service section based on the customer service conversation section. However, when a plurality of consecutive customer service conversation sections are included in one customer service switching section (a relationship between the customer service conversation sections (1) and (2) and the customer service switching section (1)), the start of the plurality of customer service conversation sections From the time to the end time is defined as one customer service section (a customer service section (1)). In addition, when the customer service switching section is interrupted in the middle of the customer service conversation section (the relationship between the customer service conversation section (3) and the customer service switching sections (2) and (3)), the customer service section is divided according to the time when the customer service switching section is interrupted. To do. That is, in the example of the figure, the service period (2) is defined as the service period (2) from the start time of the service conversation section (3) to the end time of the service switching period (2) (service conversation section). The service section (3) is defined from the start time of (3) to the end time of the customer service section (3). In addition, when the section length of the customer service section specified by these specifying methods is less than a predetermined time, it is preferable not to specify (ignore) that section as the customer service section.

  As described above, the customer service section identification pattern A can accurately identify the customer service section for each customer in the configuration in which the change of the target customer is detected based on the change of the customer's voiceprint as in the present embodiment. For example, if a customer service section is specified only by the customer service switch section (customer service switch section = customer service section), the target customer has already been switched, but it is erroneously determined that the previous customer service continues because there is no customer utterance. (For example, in the case where there is no service conversation section (2) in the service switching section (1), a time corresponding to the service conversation section (2) is added to the service conversation section (1)). . Therefore, by specifying the customer service section on the basis of the customer service conversation section, the error of the customer service section end time can be eliminated.

  Also, if the customer service section is specified only by the customer service conversation section (customer service conversation section = customer service section), it is erroneously determined that the customer is a different customer service because the conversation is interrupted for a predetermined time or more, even though the customer is the same target customer. (For example, the relationship between the service conversation sections (1) and (2) and the service switching section (1)). On the other hand, since the conversation has not been interrupted for a predetermined time or more even though the target customer is switched, it may be erroneously determined that the customer is the same customer service (for example, the customer service conversation section (3) and the customer service switching section). (Relationship between (2) and (3)). Therefore, it is possible to specify an accurate customer service section by comparing and calculating not only one of the customer service conversation section and the customer service switching section, but also the service section.

  Note that the customer service switching detection unit 171 of this embodiment periodically performs voiceprint authentication, and determines that the target customer has been switched when the voiceprint authentication result changes. However, the appearance of the same voiceprint within a predetermined time When the rate is equal to or lower than the predetermined value, the switching of the target customer may be detected. According to this configuration, since the target customer is not always alone, such as when the customer service target is a family member, by detecting the switching of the target customer according to the appearance rate of the same voice print within a predetermined time, It is possible to accurately identify the customer service section for each target customer. For example, it is conceivable that the change of the target customer is not detected while the voice print of the same person can be recognized once or more per minute. In addition, instead of detecting the change of the target customer according to the appearance rate of the same voice print within a predetermined time, if the same voice print is not detected continuously for a predetermined time or more, it may be determined that the target customer has been switched.

  Subsequently, the customer service section specifying pattern B will be described with reference to FIG. The customer service section identification pattern B extracts a section where the customer service conversation section and the customer service switching section overlap (performs a comparison operation under the “and” condition) to identify the customer service section. For example, the customer service sections (1) and (2) are sections included in the customer service switching section (1), and thus are the same sections as the customer service conversation sections (1) and (2). Moreover, since the customer service switching section (2) is included in the customer service conversation section (3), the customer service switching section (2) becomes the customer service section (3). Furthermore, the customer service conversation section (3) and the customer service switching section (3) are compared, and the overlapping portion is extracted as the service section (4). In this example as well, it is preferable not to identify a customer service section whose section length is less than a predetermined time.

  Subsequently, the customer service section specifying pattern C will be described with reference to FIG. The customer service section specifying pattern C specifies a customer service section based on the customer service switching section. For example, the customer service section (1) is the same section as the customer service switching section (1). However, when the customer service conversation section is longer than the customer service switching section (a relationship between the customer service conversation section (3) and the customer service switching section (2)), the end time of the customer service switching section is not set as the service section switching time. That is, in the example of the figure, the total of the customer service switching sections (2) and (3) is defined as the service section (2) (the service section (2) start time is the service switching section (2) start time). And the end time of the service section (2) is the end time of the service switching section (3)). In this example as well, it is preferable not to identify a customer service section whose section length is less than a predetermined time.

  Next, with reference to FIG. 31, the customer service section setting (modified example of this embodiment) will be described. In the present embodiment, the monitoring means is the voice acquisition microphone 2 and it is determined that the target customer has been switched when a change in the voiceprint of the customer is detected. That is, the same figure (a-1) was employ | adopted as a monitoring means (monitoring content). In this case, it is described that the service section specifying pattern A is preferably used as the service section specifying method ((b-1) in the figure), but other specific patterns may be used. That is, the customer service section specifying pattern B (FIG. (B-2)) and the service section specifying pattern C (FIG. (B-3)) may be employed. Further, as other service area identification methods, a service switching section = a service section (FIG. (B-4)) and a service conversation section = a service section (FIG. (B-5)) are adopted. You may do it.

  Further, as shown in (a-2) in the figure, as a monitoring content, a clerk keyword may be monitored. In this case, the customer service switching detection unit 171 performs voice recognition on the clerk's voice, and detects a target keyword by detecting a predetermined keyword by the voice recognition. The management server 15 is assumed to include a speech recognition unit including an acoustic analysis unit, an acoustic model, a language model, a word dictionary, and a text conversion unit. Note that the voice recognition unit preferably recognizes the clerk's voice included in the recording data for each utterance section. According to this configuration, it is possible to easily detect the change of the target customer by detecting the predetermined keyword. For example, the time at which “welcome” indicating the start of the customer service section is detected may be set as the switching detection time. In addition, the time at which “waiting for another visit”, “thank you for your purchase”, and “please wait for a while” indicating the end of the customer service section may be used as the switching detection time. In addition, a keyword to be uttered at the end of customer service is determined for each store, and the time when the keyword is detected may be set as the switching detection time. As a keyword in this case, it is preferable to adopt a keyword that is not used during normal customer service, such as “End” and “Bye Bye”. Furthermore, the change of the target customer may be detected more accurately by detecting both the start keyword indicating the start of the customer service section and the end keyword indicating the end of the service section. That is, it may be regarded as the start of the customer service switching section by detecting the keyword “I welcome you”, and may be regarded as the end of the customer service switching section by detecting the keyword “Waiting for another visit”. In this case, unlike the example of the above embodiment, a gap is generated between adjacent customer service switching sections.

  In addition, as shown in FIG. 3A-3, the in-store camera 11 may be used as monitoring means to monitor the behavior of the store clerk. In this case, the customer service imaging unit 111 that captures the customer service situation of the clerk and the customer functions as a monitoring unit, and the customer service switching detection unit 171 detects the switching of the target customer based on the video imaged by the customer service imaging unit 111. . Specifically, a store clerk is identified by image recognition of a video, and a change in the target customer is detected by detecting a predetermined behavior of the store clerk. The in-store camera 11 may be attached to a ceiling or a desk, or a small camera may be attached to a store clerk's clothes or body instead of the in-store camera 11. Further, as the predetermined action, a naturally occurring action such as a bow performed at the end of customer service is conceivable, and the detection time of the bow in this case may be set as the switching detection time. In addition, for each store, an action (signal) to be performed at the end of customer service may be determined, and the change of the target customer may be detected by detecting the action. As an action in this case, it is preferable to adopt an action that is not normally performed at the time of customer service, such as “perform a V sign toward the camera” or “move to a predetermined place”. Further, by detecting both a start action indicating the start of the customer service section and an end action indicating the end of the customer service section, the switching of the target customer may be detected more accurately. That is, it may be regarded as the start of the customer service switching section by detecting the action of “V sign toward the camera” and the end of the customer service switching section by detecting the action of “bow”. Also in this case, a gap is generated between adjacent customer service switching sections.

  Further, as shown in FIG. 4A-4, an angle sensor (not shown) may be used as monitoring means to monitor the operation of the store clerk. In this case, an operation detection unit (not shown) that is attached to the store clerk and detects the operation of the store clerk functions as a monitoring unit, and the customer service switching detection unit 171 switches target customers based on the detection result of the operation detection unit. Is detected. The motion detection unit is preferably attached to the upper half of the store clerk. Further, a gravity sensor or a gyro sensor may be employed instead of the angle sensor. Moreover, it is preferable that an operation | movement detection part outputs the detection result to the terminal 5 for salesclerks, and the output result is transmitted to the management server 15 from the terminal 5 for salesclerks. In this case, the customer service switching detection unit 171 detects the switching of the target customer by detecting the motion of the store clerk being tilted by the motion detection unit. According to this configuration, since it is possible to determine that the bow performed at the end of customer service has been performed by the operation of tilting the upper body of the store clerk, switching of the target customer can be accurately detected by detecting the operation.

  In addition, as the operation of the store clerk, in addition to the naturally occurring operation as described above, an operation to be performed at the end of the customer service is determined in advance for each store, and the change of the target customer is detected by detecting the operation. good. As an operation in this case, for example, an operation that is not normally performed at the time of customer service such as “touching a clerk card” or “tapping a pocket” is conceivable. Further, as the operation detection unit, another sensor such as a contact sensor or an infrared sensor may be employed. In addition to the sensor, various operation means such as a touch button operated by a store clerk may be used as the motion detection unit. Furthermore, the switching of the target customer may be detected more accurately by detecting both the start operation indicating the start of the service section and the end operation indicating the end of the service section. In other words, it may be regarded as the start of the customer service switching section by detecting the operation of “touching the clerk card”, and may be regarded as the end of the customer service switching section by detecting the operation of “bowing”. Also in this case, a gap is generated between adjacent customer service switching sections.

  As described above, the user can select an arbitrary candidate from a plurality of options as the monitoring means and the customer service section specifying method by using the input device 55 provided in the management server 15 or the like. Further, the combination of the monitoring means and the customer service section specifying method is arbitrary, and can be changed according to the use situation and the needs of the user.

  As described above, according to the customer service support system SY3 according to the third embodiment, a change in the target customer serving as a customer is detected from the monitoring result of at least one of the customer and the customer, and the detection time is identified as the customer clerk. Since it is associated with information and recorded as switching detection data, the customer service switching section can be specified from the switching detection data. In addition, since the switching detection data is recorded, it can be aggregated as marketing data, which can be used to improve customer service skills of each store clerk.

  In addition, since the customer service switching section specified based on the switching detection data and the customer service conversation section specified based on the recording data are compared and specified to determine the customer service section, the customer service section for each customer to be serviced is accurately determined. Can be specified. Further, by specifying an accurate customer service section, a highly reliable conversation ratio can be calculated. Further, since the calculated conversation ratio is recorded as a part of the customer service data, it can be used as a teaching material for the customer service method, and it can be determined whether or not each customer (customer service data) is a good customer service. Thereby, for example, the customer service situation considered to be a good customer service based on the conversation ratio can be used for horizontal development of customer service skills, such as showing each store clerk.

  In the above embodiment, the conversation ratio in the customer service section specified by the customer service section specifying unit 175 is calculated. However, the customer satisfaction in the customer service section is calculated, and the conversation ratio data and the satisfaction data based on these are calculated. You may record as correlation and customer service data. That is, it is good also as a structure which combined 2nd Embodiment and 3rd Embodiment.

  While the first to third embodiments have been described above, it is possible to provide each process of the customer service support systems SY1, SY2, and SY3 shown in each of these embodiments as a program. Further, the program can be provided by being stored in various recording media (CD-ROM, flash memory, etc.). That is, a program for causing a computer to function as each component of the customer service support system SY1, SY2, SY3 and a recording medium on which the program is recorded are also included in the scope of rights of the present invention. In addition, the system configuration and processing steps of the customer service support systems SY1, SY2, and SY3 can be appropriately changed without departing from the gist of the present invention, regardless of the above-described embodiments.

  DESCRIPTION OF SYMBOLS 1 ... Body-conduction sound sensor 2 ... Sound acquisition microphone 5 ... Clerk terminal 6 ... Viewer screen 11 ... In-store camera 12 ... POS terminal 13 ... Receipt printer 14 ... Cash register counter 15 ... Management server 16 ... Display terminal 16a ... Display screen 18 ... Product master 19 ... Network 212 ... Button group 213 ... Progress bar 214 ... Volume control slider D ... Viewer screen DB ... Management server database E11 ... Inquiry condition selection area E12 ... List display area E13 ... Reproduction operation area E21 ... Inquiry condition selection Area E22 ... Correlation diagram display area E32 ... List display area E41 ... Customer service data display area E42 ... Graph display area R ... Accounting receipt SY ... Customer service support system

Claims (9)

A conversation acquisition unit for acquiring a conversation between a store clerk and a customer;
From the acquired conversation, a talk section extraction unit that extracts a store clerk talk section that is a voice section by the store staff talk and a customer talk section that is a voice section by the customer talk,
A conversation ratio calculation unit for calculating data of a conversation ratio between the length of the length of the clerk spoke section and the customer talking period,
A customer emotion recognition unit for recognizing customer emotion based on the voice of the customer talk section;
A customer satisfaction calculation unit for calculating customer satisfaction data based on the recognition result of the customer emotion recognition unit;
And customer data recording unit for recording the calculated customer data associated with the data of the customer satisfaction that data and calculation of the conversation ratio database,
Customer data recording apparatus characterized by Ru with a.   The conversation ratio is a ratio of the length of the clerk talk section and the length of the customer talk section, a ratio of the length of the clerk talk section to the sum of the length of the clerk talk section and the length of the customer talk section, or The customer service data recording apparatus according to claim 1, wherein the customer service section is a ratio of a length of the customer talk section to a sum of a length of the store staff talk section and a length of the customer talk section. 1 utterance period a voice section continuous with no breath, when the clerk or aggregates of the speech segment of the customer are repeated without interruption over a predetermined time first spoke section, and,
The conversation ratio calculation unit, as the clerk spoke length or the customer talk period interval, hospitality according to 請 Motomeko 1 or 2 for calculating the sum of the length of the speech segment contained in one of said spoke sections Data recording device. When the said spoken section and the customer talk to seat section and the conversation section 1 a collection of the spoken section are alternately repeated without interruption over a predetermined time of the clerk,
The conversation ratio calculation unit based on the talk period and the customer talk section of the clerk included in the conversation section, customer data recording apparatus according to claim 3 you calculate a conversation ratio of the conversation interval. The customer emotion recognition unit performs emotion recognition in units of the utterance interval,
The customer satisfaction calculation unit calculates the data of customer satisfaction by the speech segment units, customer satisfaction of the customer talk period, the data of customer satisfaction of the speech segment units included in the customer spoken section customer data recording apparatus according to 請 Motomeko 4 is an average value. The aggregate of the conversation interval repeated without interruption over a predetermined time when the first reception section,
The conversation ratio calculation unit, an average value of a conversation ratio of the conversation sections included in the customer section, calculated as a conversation ratio of the customer segment,
Before SL customer data recording unit, as the conversation ratio data, customer data recording apparatus according a conversation ratio of the customer segment, the 請 Motomeko 4 or 5 that records and a conversation ratio of the conversation interval. For example Bei the screen display unit that displays a bi Yua screen to check the customer data,
The screen display unit, the selection or input of the personal identification information for identifying the clerk and / or the customer, to extract the customer data including the personal identification information, to 請 Motomeko 1 displayed on the viewer screen 7. The customer service data recording device according to any one of items 6. And the conversation acquiring the conversation of store employees and customers,
A conversation section extraction step for extracting, from the acquired conversation, a store clerk talk section that is a speech section by the store staff talk and a customer talk section that is a speech section by the customer talk;
A conversation ratio calculation step of calculating the data of the conversation ratio between the length of the clerk spoke length and the customer talk period interval,
A customer emotion recognition step for recognizing customer emotion based on the voice of the customer talk section;
A customer satisfaction calculation step for calculating customer satisfaction data based on the recognition result of the customer emotion recognition step;
And calculated the meeting spoke ratio data, the customer data recording step of recording the customer data in a database that associates the and calculated the customer satisfaction data,
The service data recording method characterized by performing. The program for making a computer perform each step in the customer service data recording method of Claim 8 .

Images