JP2011237965A - Conversation ratio calculation device, customer service data recording device, conversation ratio calculation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a conversation ratio calculation device, a customer service data recording device, a conversation ratio calculation method and program capable of calculating an accurate conversation ratio.SOLUTION: A conversation ratio calculation device, a customer service data recording device, a conversation ratio calculation method and program equipped with; a conversation acquisition part 102 which acquires a conversation between a clerk and a customer; an utterance detection part 101 which is fixed by the clerk and detects a utterance of the clerk; a voice extraction part 154 which extracts a clerk's voice as a voice of the clerk and a customer's voice as a voice of the customer from an acquired conversation based on a detection result of the utterance detection part 101; a conversation ratio calculation part 155 which calculates a conversation ratio as a ratio of the length of the clerk's voice to the customer's voice, or a ratio of the length of the clerk's voice or the customer's voice to the total length of the clerk's voice and the customer's voice.

Description

  The present invention relates to a conversation ratio calculation apparatus, a customer service data recording apparatus, a conversation ratio calculation method, and a program for acquiring a voice of a person and estimating a conversation state.

  Conventionally, as a technique for estimating a conversation state, an emotion recognition technique for recognizing an emotion based on a person's voice is known. For example, in Patent Document 1, emotion recognition is performed by associating changes in speech intensity, tempo, and intonation with each emotional state such as “anger”, “sadness”, and “joy” from the speech signal input by the subject. It is carried out.

  On the other hand, in stores such as retail stores, attempts are being made to collect customer data and sales data as marketing data for use in sales promotion. For example, Patent Document 2 proposes a marketing data collection / analysis technique that analyzes POS data to quantitatively grasp the tendency of simultaneous sales by consumers and reflects it in sales promotion.

  By the way, in the field of marketing, it is said that increasing customer satisfaction at the expense of productivity and efficiency is better as a result of consumer repeaters. Also, retailers and other stores that emphasize hospitality have a close relationship between sales and customer satisfaction, so customer service skills training and customer impressions are improved to increase customer satisfaction. For training of smiles and greetings. In particular, conversation techniques, which are one of customer service skills, are regarded as important, and training focusing on the ratio of conversations with customers is conducted. Generally speaking, it is said that the ratio of the conversation between the store clerk and the customer is preferably larger (for example, 2: 8), and maintaining this will have a positive impact on customer satisfaction. It has been broken.

JP 2002-91482 A JP 2007-94592 A

  However, even if the conversation ratio that is preferable for customer service is known, there is no method for measuring the conversation ratio, so it is unclear whether or not each store clerk can practice it. In other words, there is a problem that even if sales are improved by performing training focusing on the conversation ratio, the relevance cannot be proved. Further, in the retail industry, there is a request to collect information on the conversation ratio during the actual customer service as marketing data and use it for the management strategy, but it has not been realized yet.

  In view of the above problems, an object of the present invention is to provide a conversation ratio calculation device, a customer service data recording device, a conversation ratio calculation method, and a program capable of calculating an accurate conversation ratio and recording the conversation ratio. .

  The conversation ratio calculation apparatus according to the present invention includes a conversation acquisition unit that acquires a conversation between a person A and a person B, and an A sound that is a sound of the person A and a B sound that is a sound of the person B from the acquired conversation. A speech ratio for calculating a speech ratio that is a ratio of the length of the A voice and the B voice, or a ratio of the length of the A voice or the B voice to the total length of the A voice and the B voice. And a calculating unit.

  The conversation ratio calculation method of the present invention includes a conversation acquisition step in which a computer acquires a conversation between a person A and a person B, and an A sound that is the sound of the person A and a B sound that is the sound of the person B from the acquired conversation. And a conversation ratio that is a ratio of the length of the A voice and the B voice or a ratio of the length of the A voice or the B voice to the total length of the A voice and the B voice. Performing a conversation ratio calculating step.

According to these configurations, the conversation ratio between the person A and the person B can be calculated by extracting the A sound that is the sound of the person A and the B sound that is the sound of the person B from the conversation. In this case, the conversation ratio is either (1) the ratio of La and Lb when the length of the A voice is La and the length of the B voice is Lb, or (2) the ratio of La or Lb to (La + Lb). Can be calculated as As a result, information related to the conversation ratio can be collected as marketing data and used for management strategies, or used for training in customer service skills. Further, by using the present invention individually, it can be used to improve conversation skills (conversation techniques) between individuals.
The person A and the person B are not necessarily acquired by one conversation acquisition unit (such as a microphone), and may be acquired individually by two conversation acquisition units. In this case, the voice extraction unit distinguishes between the A voice and the B voice by the acquired conversation acquisition unit, and extracts each voice.

  In the conversation ratio calculation apparatus described above, the speech ratio calculation unit further includes an utterance detection unit that is attached to the person A and detects that the person A uttered, and the voice extraction unit performs conversation based on the detection result of the utterance detection unit. It is preferable to determine whether the included sound is the sound of the person A or the sound of the person B, and extract each sound based on the determination result.

  In the conversation ratio calculation method described above, the computer is attached to the person A and connected to an utterance detection unit that detects that the person A uttered, and in the voice extraction step, the detection result of the utterance detection unit is displayed. Based on this, it is preferable to determine whether the voice included in the conversation is the voice of the person A or the voice of the person B, and extract each voice based on the determination result.

According to these configurations, by using the utterance detection unit, it is possible to accurately discriminate between the voice of the person A and the voice of the person B, and thus more accurately calculate the conversation ratio.
As the utterance detection unit, for example, a body conduction sensor that detects body conduction sound that a person's voice reaches the body surface through bones and meat can be cited. In this case, it is preferable that the body sound sensor is mounted on the body surface of the head and neck.

  In the conversation ratio calculation apparatus described above, it is preferable that the person A and the person B indicate a store clerk and a customer who are serving customers, and the A sound and the B sound indicate a store clerk sound and a customer sound.

  According to this configuration, it is possible to collect information on the conversation ratio during the actual customer service from the conversation between the store clerk and the customer.

  In the conversation ratio calculation apparatus described above, when a continuous speech section without breathing is defined as one utterance section, and a set of clerk or customer utterance sections repeated without interruption for a predetermined time or more is defined as one talk section, the clerk voice Is the length of the talking section by the store clerk, and the length of the customer voice is preferably the length of the talking section by the customer.

  According to this configuration, the conversation ratio can be calculated based on the length of the talk section between the store clerk and the customer. In addition, since the talk section is a collection of clerk or customer utterance sections that are repeated without interruption for a predetermined time or more, even if the utterance section is interrupted due to breathing, etc. The customer's talking section (the length of the clerk voice and the length of the customer voice) can be specified.

  In the conversation ratio calculation apparatus described above, when a continuous speech section without breathing is defined as one utterance section, and a set of clerk or customer utterance sections repeated without interruption for a predetermined time or more is defined as one talk section, the clerk voice Is preferably the total length of each utterance section included in the talk section by the store clerk, and the length of the customer voice is preferably the total length of each utterance section included in the talk section by the customer. .

  According to this configuration, the conversation ratio can be calculated based on the total length of the utterance sections of the store clerk and the customer. That is, when the speech section is interrupted due to breathing or the like, the length of the clerk voice and the length of the customer voice can be specified excluding the interval.

  In the conversation ratio calculation apparatus described above, when the conversation section of the store clerk and the customer is alternately repeated without interruption for a predetermined time or more as one conversation section, the conversation ratio calculation unit is included in the conversation section. It is preferable to calculate a conversation ratio for each conversation section based on one or more talk sections.

  According to this configuration, since the conversation ratio is calculated for each conversation section that is an aggregate of a series of talk sections, it is possible to calculate a conversation ratio with higher reliability as compared to the case where the conversation ratio is calculated for each unit time.

  In the conversation ratio calculation device described above, when a set of conversation sections that are repeated without interruption for a predetermined time or more is defined as one customer service section, the conversation ratio calculation unit calculates the average conversation ratio of each conversation section included in the customer service section. The value is preferably calculated as the conversation ratio in the customer service section.

  In the conversation ratio calculation apparatus described above, a set of conversation sections in which the conversation sections of the store clerk and the customer are alternately repeated without interruption for a predetermined time or more is one conversation section, a collection of conversation sections that are repeated without interruption for a predetermined time or more. Is a customer service section, the conversation ratio calculation unit ΣLa (where ΣLa is ΣLa ≧ Σ), which is the sum of the length of each talk section or the total length of each speech section included in the customer service section ΣLb (where ΣLb is a variable that satisfies ΣLb ≧ 0), and ΣLa and ΣLb Or the ratio of ΣLa or ΣLb to (ΣLa + ΣLb) is preferably calculated as the conversation ratio in the customer service section.

  According to these configurations, in order to calculate the conversation ratio for each customer service section, whether or not each customer service is a comprehensive customer service (for example, the length of the clerk voice: the customer voice length is 2: 8). It is possible to grasp whether or not the value is close. In this case, the conversation ratio in the customer service section is (1) the average value of the conversation ratio of each conversation section included in the customer service section, and (2) the sum of the lengths of each talk section by the store clerk included in the customer service section is ΣLa, The ratio of ΣLa and ΣLb or the ratio of ΣLa or ΣLb to (ΣLa + ΣLb) when the total length of each talking section by the customer is ΣLb, (3) the length of each utterance section by the store clerk included in the customer service section ΣLa and ΣLb, or the ratio of ΣLa or ΣLb to (ΣLa + ΣLb), where ΣLa is the total length of utterances by the customer.

  The customer service data recording device of the present invention is identified by each unit in the conversation ratio calculation device described above, a person identifying unit that identifies a clerk and a customer who are parties to the conversation acquired by the conversation acquiring unit, and a person identifying unit A sales information acquisition unit that acquires sales information that is sales results for a customer of the store clerk, a store clerk and a customer identified by the person identification unit, a conversation ratio in the customer service section calculated by the conversation rate calculation unit, sales information, And a customer service data recording unit for recording the data as customer service data.

  According to this configuration, by recording the conversation ratio as customer service data, it can be used as marketing data. In addition, since the identified store clerk and customer are associated with the customer service data, it is possible to specify which store clerk is serving the customer for which customer the calculated conversation ratio. Thereby, appropriate customer service training can be performed for each store clerk. Furthermore, since sales information is associated with the customer service data, the correlation between the conversation ratio and the sales can be grasped.

  The customer service data recording device described above further includes a voice data recording unit that records voice data corresponding to a customer service section of the conversation acquired by the conversation acquisition unit, wherein the customer data recording unit includes a store clerk and a customer, a conversation ratio, and sales. It is preferable that the information is associated with the recording data of the voice data recording unit and recorded as customer service data.

  According to this configuration, since the recorded data is associated with the customer service data, for example, the recorded data associated with a preferable conversation ratio can be extracted and played back, and used as a service training material. That is, since the customer service situation of a salesclerk with high conversation skills can be efficiently shown to other salesclerks, it is useful for horizontal development of conversation skills.

  In the customer service data recording apparatus described above, the customer service data recording unit further includes an overlap count measuring unit that measures the number of times of overlap between the store clerk's voice and the customer's voice in the customer service section. It is preferable to associate the number of duplications with the conversation ratio and the sales information and record it as customer service data.

According to this configuration, since the number of times of duplication is associated with the customer service data, it is possible to infer whether or not the customer service is preferable from the number of times of duplication as well as the conversation ratio. In general, it is considered that if the voice of a store clerk is covered with the voice of a customer, it becomes a factor that lowers customer satisfaction. For this reason, when the number of times of duplication is large, even if the conversation ratio is a preferable value, it can be determined that the customer service is not preferable.
The customer service score may be calculated based on the conversation ratio and the number of times of duplication. In this case, it is preferable to increase the number of customer service points as the ratio between the length of the clerk voice and the length of the customer speech is closer to 2: 8, and to increase the customer service score as the number of overlaps decreases. Further, it may be determined whether or not the customer service is preferable according to the number of customer service points, and the determination result and the customer service score may be recorded as a part of the customer service data.
Further, at the end of the conversation section or the customer service section, the determination result as to whether or not the customer service is favorable and the customer service score may be notified to the target store clerk. According to this structure, since it can be grasped | ascertained whether the store clerk is performing the preferable customer service during customer service, improvement of the customer service method can be expected.

  The customer service data recording apparatus described above further includes a screen display unit that displays a viewer screen for confirming customer service data, wherein the screen display unit selects person identification information for identifying a clerk and / or a customer or It is preferable that the customer service data including the person identification information is extracted by input and displayed on the viewer screen.

  According to this configuration, desired customer service data can be confirmed on the viewer screen by selecting or inputting person identification information for identifying at least one of a store clerk and a customer as a search condition.

  In the customer service data recording apparatus described above, the screen display unit preferably displays a playback operation area for performing a playback operation of the recorded data included in the extracted customer service data on the viewer screen.

According to this configuration, the recorded data can be confirmed using the playback operation area displayed on the viewer screen.
In addition, the customer service imaging part which images a customer service situation may be provided, and the imaging data may be confirmed together with the recording data. In this case, the imaging data is recorded by being included in the customer service data. In addition, associating the imaging data with other data can use the identification information of the clerk and customer identified from the imaging data using the face recognition technology.

  In the customer service data recording apparatus described above, the playback operation area includes a progress bar indicating a playback position, and the progress bar includes a talk section by a store clerk, a talk section by a customer, an overlapping section in which both sections overlap, Are preferably displayed separately.

According to this configuration, the conversation ratio and the number of times of duplication can be estimated at a glance from the display of the progress bar. As a result, the store clerk can easily reproduce only the talking section or only the overlapping section.
Note that “display with distinction” refers to display so that the user can identify each section, such as color-coded display, mark addition display, and text addition display.

  In the customer service data recording apparatus described above, it is preferable that the screen display unit displays a correlation diagram illustrating a correlation between a conversation ratio included in the customer service data and sales information on the viewer screen.

According to this configuration, the correlation between the conversation ratio and the sales information can be grasped at a glance from the correlation diagram displayed on the viewer screen. That is, from the correlation diagram, it is possible to grasp how much the sales ratio is improved for the target store clerk and / or the target customer.
Note that the correlation diagram refers to a scatter diagram, a dot diagram, and the like.

  A program according to the present invention causes a computer to execute each step in the conversation ratio calculation method described above.

  By using this program, a conversation ratio calculation method capable of calculating an accurate conversation ratio can be realized.

1 is a system configuration diagram of a customer service support system according to a first embodiment. It is a control block diagram of the terminal for shop assistants. It is a control block diagram of a receipt printer. It is a control block diagram of a management server. It is explanatory drawing of a speech section, a talk section, a conversation section, and a customer service section. It is a functional block diagram of the customer service support system concerning a 1st embodiment. It is explanatory drawing of the database for management servers which concerns on 1st Embodiment. It is a figure which shows the calculation algorithm of a conversation ratio. It is a figure which shows an example of an audio | voice data management table, a salesclerk utterance area management table, and a customer utterance area management table. It is a flowchart which shows the audio | voice data preservation | save process which concerns on 1st Embodiment. It is a flowchart which shows the service area specific process which concerns on 1st Embodiment. It is a flowchart which shows the shop assistant talk area specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area B specific process which concerns on 1st Embodiment. It is a flowchart which shows the customer talk area A specific process which concerns on 1st Embodiment. It is a figure which shows the measuring method of the frequency | count of duplication. It is a figure which shows an example of a viewer screen (conversation ratio list). It is a figure which shows an example of a viewer screen (conversation ratio-sales performance correlation diagram). It is a figure which shows the calculation algorithm of a customer service score, a conversation ratio determination table, and a duplication count determination table. It is a functional block diagram of the customer service support system which concerns on 2nd Embodiment. It is explanatory drawing of the database for management servers which concerns on 2nd Embodiment. It is a figure which shows the calculation algorithm of customer satisfaction. It is a figure which shows an example of a viewer screen (satisfaction-conversation ratio list). It is a figure which shows an example of a viewer screen (satisfaction-conversation ratio overlay graph). It is a functional block diagram of the customer service assistance system which concerns on 3rd Embodiment. It is explanatory drawing of the database for management servers which concerns on 3rd Embodiment. It is a figure which shows an example of switching detection data, and an image figure of a customer service switching period. It is a figure which shows an example of a service conversation area specific result, and an image figure of a service conversation section. It is explanatory drawing of the service area identification pattern A. It is explanatory drawing of the service area specific pattern B. It is explanatory drawing of the customer service area specific pattern C. FIG. It is explanatory drawing of a service area setting.

[First Embodiment]
Hereinafter, a conversation ratio calculation device, a service data recording device, a conversation ratio calculation method, and a program according to the present invention will be described with reference to the accompanying drawings. In each embodiment shown below, the conversation ratio calculation apparatus of the present invention is exemplified for the case where it is applied to the customer service support system SY. The customer service support system SY acquires voices of people (customers, visitors) in stores and venues related to retail, restaurants and service industries, and uses the results to improve customer satisfaction and customer service skills. It was built accordingly. Thus, in each of the following embodiments, a case where the customer service support system SY is introduced into a retail store such as an apparel store or a jewelry store is illustrated.

  FIG. 1 is a system configuration diagram of a customer service support system SY1 according to the first embodiment. As shown in the figure, the customer service support system SY1 includes a body conduction sound sensor 1, a voice acquisition microphone 2 and a clerk terminal 5 attached to a clerk, and in-store cameras 11 ( In the figure, only one unit is shown), a POS (Point Of Sales) terminal 12 and a receipt printer 13 installed at the checkout counter 14, a management server 15 and a display terminal 16 installed in the backyard of the store. Become. In addition, the computer in a claim consists of each apparatus other than the body-conduction sound sensor 1. FIG.

  The body-conducting sound sensor 1 is mounted on the head and neck of the store clerk, and detects the body-conducted sound that the store clerk's voice reaches the body surface through bones and meat. In the present embodiment, the voice acquired by the voice acquisition microphone 2 is used to identify whether it is that of a store clerk (person A) or a customer (person B). The voice acquisition microphone 2 is attached to the clerk's uniform (near the chest) and acquires the clerk's and customer's voices. In place of the body-conducting sound sensor 1 and the voice acquisition microphone 2, microphones having directivity for the store clerk and the customer may be used. In other words, the clerk's voice and the customer's voice may be identified by using two microphones, one for obtaining the clerk's voice and the other for obtaining the customer's voice.

  The store clerk terminal 5 is attached to a store clerk's uniform (belt or the like), and is connected to the body conduction sound sensor 1 and the voice acquisition microphone 2 via a dedicated cable. In addition, it has a wireless communication function with the receipt printer 13, and exchanges information with the management server 15 via the receipt printer 13. The in-store camera 11 is provided on the ceiling and walls of various places in the store, and images customers who have come to the store as well as clerk and customers who are serving customers. As the in-store camera 11, a CCD camera or a PTZ (Pan Tilt Zoom) camera can be used.

  The POS terminal 12 has a general register configuration, and performs accounting processing according to the POS application. Further, a product code is obtained from a bar code scanner or a keyboard (not shown), and receipt data for printing on the accounting receipt R (see FIG. 3) is generated with reference to the product master 18. The product master 18 may be provided in a POS server (not shown) connected to the POS terminal 12.

  The receipt printer 13 is connected to the POS terminal 12 via a dedicated cable, and prints receipt print data acquired from the POS terminal 12 on receipt paper. The receipt printer 13 has a wireless communication function with the clerk terminal 5 and a wired communication function with the management server 15. In this way, by inputting / outputting various information with the receipt printer 13 as the main (the receipt printer 13 filters the acquired various information and outputs necessary information), the POS backbone network (POS terminal 12). Network) is not affected. Further, when applying the present invention to an existing POS system, it is not necessary to change the POS backbone network itself.

  The management server 15 is connected to the receipt printer 13 via a network 19 such as an intranet, and exchanges various information with the clerk terminal 5 via the receipt printer 13. Further, based on the voice data acquired from the clerk terminal 5, the voice data is recorded, the conversation ratio is calculated, and the number of times of duplication is measured. Further, the management server 15 displays the calculated conversation ratio and the viewer screen D (see FIG. 16 and the like) for confirming the recorded voice data on the display screen 16 a of the display terminal 16.

  Next, the hardware configuration of the clerk terminal 5, the receipt printer 13, and the management server 15 will be described with reference to FIGS. FIG. 2 is a control block diagram of the clerk terminal 5. The clerk terminal 5 includes a wireless LAN antenna 21, a wireless LAN RF (Radio Frequency) unit 22, a wireless LAN modem unit 23, and a wireless LAN baseband unit 24 for realizing a wireless communication function with the receipt printer 13. . The wireless LAN baseband unit 24 stores a MAC address for identifying the clerk terminal 5. In addition, the store clerk terminal 5 includes an amplifier unit 28 and an A / D converter 29 for acquiring the detection result of the body conduction sound sensor 1, and an amplifier unit 32 and A for acquiring audio data from the audio acquisition microphone 2. / D converter 33.

  The clerk terminal 5 includes a control unit 25 that performs overall control of each unit, a memory 26 that stores various data including the firmware (including voice data from the voice acquisition microphone 2), and a clerk terminal 5. A battery 34 for supplying power. The control unit 25, based on the detection data and the voice data acquired from the A / D converter 29 and the A / D converter 33, a store clerk utterance section specifying function for specifying the clerk's utterance section (continuous voice time zone), A sound level determination function for determining a sound level based on the sound data.

  FIG. 3 is a control block diagram of the receipt printer 13. The receipt printer 13 includes a wireless LAN antenna 41, a wireless LAN RF unit 42, a wireless LAN modem unit 43, and a wireless LAN baseband unit 44 for realizing a wireless communication function with the clerk terminal 5. The wireless LAN baseband unit 44 stores a MAC address for identifying the receipt printer 13. The receipt printer 13 includes an input interface unit 45 for receiving receipt data from the POS terminal 12, a CG-ROM 46 for storing character patterns, a control unit 47 for performing overall control of each unit, a print head, and a head drive mechanism. And a printing mechanism 48 including a receipt paper transport mechanism, and a wired LAN interface unit 49 connected to the management server 15 via a wired LAN.

  The control unit 47 includes a main processing unit 47a that performs analysis of receipt data including a predetermined command, generation of print data to be printed on the accounting receipt R, and a receipt data semantic analysis unit that is a configuration unique to the present embodiment. 47b. The receipt data semantic analysis unit 47b recognizes the terminal number of the POS terminal 12, the receipt number, the product code, the product name, the product unit price, the amount of money, the operator name, and the like from the receipt data, and can be interpreted by the management server 15 serving as the host system. To a predetermined data format (for example, XML format). In addition, what converted the recognition result of the receipt data into a predetermined data format is hereinafter referred to as “converted data”. In addition, the control unit 47 transfers the voice data received from the clerk terminal 5 via the wireless LAN (voice data acquired from the wireless LAN baseband unit 44) to the management server 15 via the wired LAN interface unit 49. To do.

  FIG. 4 is a control block diagram of the management server 15. The management server 15 acquires audio data and conversion data from the receipt printer 13 and also displays a wired LAN interface unit 51 for acquiring video data from the in-store camera 11 and a display for displaying various information on the display terminal 16. The processing unit 52 includes a control unit 53 that acquires input data from an input device 55 such as a mouse or a keyboard and performs overall control of each unit, and a storage unit 54 that stores various types of information. Based on the acquired audio data, the control unit 53 determines a customer service section specifying function for specifying a customer service section (a time zone during which the store clerk is serving customers), and a conversation ratio between the customer clerk and the customer per customer service section. A conversation ratio calculation function to be calculated, an overlap count measurement function that measures the number of overlaps of conversation per service section, and a viewer that controls display of the viewer screen D (see FIG. 16 and the like) based on information input from the input device 55 And a screen display control function. The storage unit 54 is realized by a hard disk, and stores various control programs such as a voice extraction program and a conversation ratio calculation program, and also functions as a management server database DB1 described later.

  Next, the definition of the voice section will be described with reference to FIG. First, a section of speech continuously uttered by the same person (a store clerk or a customer), that is, a section of one phrase that does not contain a breath or the like is referred to as an “utterance section”. In the present embodiment, emotion recognition and voice recognition are performed for each utterance section. Further, as shown in FIG. 5A, a collection of clerk or customer utterance sections that are repeated without interruption for a predetermined time or longer is referred to as a “talking section”. That is, it refers to a set of one or more utterance sections whose interval is less than the predetermined time X (where X is a constant that satisfies X> 0). In the example shown in the figure, a store clerk talk section (hereinafter referred to as “store clerk talk section”) and two customer talk sections (hereinafter referred to as “customer talk section”) around the store clerk talk section. The case where all are composed of two utterance sections is shown.

  Similarly, as shown in FIG. 5A, a group of talking sections in which the talking sections of the store clerk and the customer are alternately repeated without being interrupted for a predetermined time or longer are referred to as “conversation sections”. That is, it refers to an aggregate of one or more talking sections whose interval is less than the predetermined time Y (where Y is a constant satisfying Y ≧ X). In the present embodiment, a group of customer talk sections before and after the store staff talk section (that is, a collection of at least one and up to three talk sections) is expressed as “one conversation pattern” = “one conversation section”. Define.

  In addition, as shown in FIG. 5B, a collection of conversation sections that are repeated without interruption for a predetermined time or longer is referred to as a “service section”. That is, it refers to an aggregate of one or more conversation sections whose interval is less than the predetermined time Z (where Z is a constant such that Z> Y). The example of the figure shows a service section 1 composed of two conversation sections and a service section 2 composed of three conversation sections. Thus, the number of conversation sections included in the customer service section is arbitrary. Note that it is preferable that the value of Z is obviously large compared to the values of X and Y, such as several minutes to several tens of minutes.

  Next, with reference to FIG. 6 and FIG. 7, the functional configuration of the customer service support system SY1 according to the first embodiment will be described. FIG. 6 is a block diagram of the customer service support system SY1. The in-store camera 11 has a customer service imaging unit 111 as a main functional configuration. The customer service imaging unit 111 captures a customer service situation including a clerk and a customer. In the present embodiment, the customer service imaging unit 111 always performs imaging, and the video data is output to the management server 15 as needed.

  The body sound sensor 1 has an utterance detection unit 101 as a main functional configuration. The utterance detection unit 101 detects that the clerk uttered and the utterance section based on the body conduction sound. The voice acquisition microphone 2 has a conversation acquisition unit 102 as a main functional configuration. The conversation acquisition unit 102 acquires sound (voice signal) based on the conversation between the store clerk and the customer. The clerk terminal 5 has an audio data transmission unit 105 as a main functional configuration. The audio data transmission unit 105 determines the presence / absence of audio using a power filter in the audio level determination function, and transmits audio data of a predetermined audio level or higher (for example, 1.5 v or higher after amplifier amplification) to the management server 15. . Further, the store clerk utterance section is specified based on the detection result of the utterance detection unit 101 and the voice acquisition result of the conversation acquisition unit 102 (store clerk utterance section specifying function), and the management server 15 is notified of the occurrence of the clerk utterance section. Note that the clerk terminal 5 and the management server 15 communicate via the receipt printer 13.

  The receipt printer 13 has a conversion data transmission unit 113 as a main functional configuration. The conversion data transmission unit 113 transmits the conversion data obtained by converting the receipt data output from the POS terminal 12 into the XML format to the management server 15.

  The management server 15 includes a video recording unit 151, a person identification unit 152, a voice data recording unit 153, a voice extraction unit 154, a conversation ratio calculation unit 155, a duplication count measurement unit 156, a converted data acquisition unit 157, as main functional configurations. A service data recording unit 158, a screen display unit 159, and a management server database DB1 are provided.

  The video recording unit 151 acquires video data from the customer service imaging unit 111 and records it in the management server database DB1. The person identifying unit 152 identifies a store clerk and a customer based on the facial feature amount included in the video data. For example, for a clerk, the clerk identification information and the clerk's facial feature quantity are associated in advance and stored in the management server database DB1 (see the clerk information storage unit 82 in FIG. 7). In addition, the face detection is performed by analyzing the imaging result of the in-store camera 11, the face feature amount calculated by normalizing the detected image of the face portion, and the face feature of the clerk stored in the management server database DB1. The amount is collated, and it is determined that the salesclerk has the highest similarity among them. Similarly, for the customer, the customer identification information and the facial feature amount of the customer are associated with each other and stored in the management server database DB1 (see the customer information storage unit 81 in FIG. 7), and the calculated customer face is stored. The feature quantity is compared with the face feature quantities of many customers stored in the management server database DB1, and it is determined that the customer has the highest similarity among them. Note that the clerk identification information and customer identification information, which are the identification results of the person identification unit 152, are linked and stored in the customer service data storage unit 88 described later.

  The voice data recording unit 153 records the conversation between the store clerk and the customer, that is, the voice data transmitted from the voice data transmission unit 105 (records in the management server database DB1). The voice extraction unit 154 extracts a clerk voice and a customer voice from the acquired conversation (voice data). Specifically, based on the detection result of the utterance detection unit 101, it is determined whether the voice included in the conversation is a store clerk voice or a customer voice, and each voice is extracted based on the determination result. Each voice is extracted in units of utterance sections or talk sections. Details will be described later.

  The conversation ratio calculation unit 155 indicates the conversation ratio calculation function of the control unit 53, and calculates the conversation ratio between the store clerk and the customer. Specifically, the conversation ratio is calculated for each conversation section, and the conversation ratio (average conversation ratio) for each service section is calculated based on the calculation result for each conversation section. The calculated conversation ratio is stored in the customer service data storage unit 88 as part of the customer service data. The conversation ratio calculation algorithm will be described later.

  The overlap number measurement unit 156 indicates the overlap number measurement function of the control unit 53, and measures the number of overlaps, which is the number of times that the clerk's voice and the customer's voice are overlapped in the customer service section (number of overlap sections). . The measured number of overlaps is stored in the customer service data storage unit 88 as part of the customer service data. An overlapping section detection method and an overlapping section measurement method will also be described later.

  The conversion data acquisition unit 157 acquires the conversion data transmitted from the conversion data transmission unit 113 of the receipt printer 13 and records it in the management server database DB1. The converted data is used to obtain sales information recorded as a part of the customer service data. The conversion data is recorded in the information included in the conversion data, such as customer identification information (membership number, etc.), receipt number (sales No.), total amount, etc. that can specify the presence or absence of sales and the sales amount. May be extracted and recorded as sales information, or all of the conversion data may be recorded in the management server database DB1.

  The customer service data recording unit 158 associates and manages customer service data including the clerk identification information and customer identification information, which are identification results of the person identification unit 152, and the calculation result of the conversation ratio calculation unit 155, for each customer service section. Record in the server database DB1. Note that the customer identification information and the clerk identification information are specified from the face feature amount as described above. Further, the clerk identification information and the MAC address of the clerk terminal 5 are stored in association with each other (see the clerk information storage unit 82 in FIG. 7), and the video data acquired by the management server 15 is associated with the audio data. Can be done.

  The screen display unit 159 displays a viewer screen D (see FIG. 16 and the like) for confirming the recorded customer service data on the display screen 16a. The viewer screen D will be described in detail later with a specific example.

  FIG. 7 is an explanatory diagram of the management server database DB1 according to the first embodiment. The management server database DB1 includes a customer information storage unit 81, a clerk information storage unit 82, an audio data storage unit 83, a video data storage unit 84, an audio data management table 85, a clerk utterance section management table 86, and a customer utterance section management table 87. And functions as a customer service data storage unit 88. The management server database DB1 may be provided for each store, or may be shared by a plurality of stores.

  The customer information storage unit 81 associates customer identification information (customer ID and the like), customer face feature amounts, and customer data (including personal information such as name, address, telephone number, date of birth, and sex). Remember. The store clerk information storage unit 82 stores store clerk identification information (store clerk ID, etc.), the store clerk's facial feature quantity, and the MAC address of the store clerk terminal 5 in association with each other. Also, the voice data storage unit 83 stores the voice data always recorded by the voice data recording unit 153 together with a time stamp. In addition, the video data storage unit 84 stores video data always captured by the customer service imaging unit 111 together with a time stamp.

  The voice data management table 85 is a table in which the acquired voice data is recorded for each continuous voice section (hereinafter referred to as “continuous speech section”) without distinguishing between the store clerk and the customer (FIG. 9). (See (a))). Further, the store clerk utterance section management table 86 is a table in which store section utterance sections are recorded (see FIG. 9B). The customer utterance section management table 87 is a table in which customer utterance sections are recorded (see FIG. 9C).

  The customer service data storage unit 88 corresponds to the customer identification information and the clerk identification information, which are identification results of the person identification unit 152, and the voice data for the customer service section among the voice data stored in the voice data storage unit 83. Among the recorded data and the video data stored in the video data storage unit 84, the imaging data corresponding to the video data for the service section, and the conversation ratio of the service section, which is the calculation result of the conversation ratio calculation unit 155, The number of times of overlap of the customer service section, which is a measurement result of the overlap number measuring unit 156, the sales information indicating the presence or absence of sales by the customer service (generated within the customer service section or within a predetermined time after the end of the customer service section), and the sales amount, A customer service date and a customer service date and time indicating a customer service start time and a customer service end time are stored in association with each other. As for the sales information, the customer's facial feature amount calculated from the customer's video imaged by the in-store camera 11 installed at the checkout location and a large number of customer's information stored in the management server database DB1 in advance. It is possible to identify the customer service data to be associated with the customer identification information of the customer having the highest similarity from the facial feature amounts. It is also possible to specify customer service data to be associated with the customer identification information included in the conversion data transmitted from the receipt printer 13. When the converted data includes store clerk identification information (such as operator name or store clerk number), it is preferably linked to customer service data that matches both customer identification information and store clerk identification information.

  Next, various conversation ratio calculation algorithms (calculation formulas) will be described with reference to FIG. As shown in FIG. 8A, as the conversation ratio in the conversation section, three patterns of “store clerk: customer relative conversation ratio”, “store clerk conversation ratio”, and “customer conversation ratio” can be calculated. . For example, “relative conversation ratio of clerk: customer” is expressed as follows: La is the total length of each clerk talk section included in the conversation section, and Lb is the total length of each customer talk section included in the conversation section. La: Refers to the value of Lb. “Conversation ratio of store clerk” indicates a value of La / (La + Lb), and conversation ratio of a customer indicates a value of “Lb / (La + Lb)”. Here, “the length of the talking section” refers to the length from the start time to the end time of the talking section.

  Note that La may be defined as the total length of each clerk utterance section included in the conversation section. That is, as shown in FIG. 5A, the talk section may include the interval X, but La may be defined as a length obtained by subtracting the length of the interval. For example, in the case of the customer talk section A shown in FIG. 5A, La is the sum of the length from the start time to the end time of the utterance section 1 and the length from the start time to the end time of the utterance section 2. . The same applies to Lb.

  On the other hand, as shown in FIG. 8B, the conversation ratio in the customer service section can be calculated by the average value of the conversation ratios in the respective conversation sections included in the customer service section. Note that various statistics such as the maximum value, the minimum value, and the intermediate value may be calculated as the conversation ratio in the customer service section instead of the average value of the conversation ratio in each conversation section. Further, the conversation ratio in the customer service section also depends on which pattern is adopted as the conversation ratio in the conversation section (see FIG. 8A), and the “relative conversation ratio of the clerk: customer”, “conversation ratio of the clerk” and Three patterns of “customer conversation ratio” can be calculated.

  Moreover, you may use the algorithm of FIG.8 (c) as a modification of the calculation method of the conversation ratio in a service area. In this case, the “relative conversation ratio of store clerk: customer” is the sum of the lengths of the talker sections included in the customer service section is ΣLa, and the total length of the customer talk sections included in the customer service section is ΣLb. , ΣLa: refers to the value of ΣLb. Further, “the clerk conversation ratio” indicates a value of ΣLa / (ΣLa + ΣLb), and “customer conversation ratio” indicates a value of ΣLb / (ΣLa + ΣLb).

  Note that ΣLa may be defined as the total length of each clerk utterance section included in the customer service section in the same manner as La is defined as the total length of each clerk utterance section included in the conversation section. . The same applies to ΣLb.

  Next, the voice data management table 85, the clerk utterance section management table 86, and the customer utterance section management table 87 will be described with reference to FIG. FIG. 9A shows an example of the audio data management table 85. The voice data management table 85 includes “voice data No.” continuously assigned to each continuous utterance section (section including at least one utterance section) which is a continuous voice section that does not distinguish between the clerk and the customer's voice. “Recording start time” corresponding to the start time of the utterance section and “Recording end time” corresponding to the end time of the continuous utterance section and voice data based on the customer's utterance or voice data based on the clerk's utterance, or The “duplicate flag” indicating whether the voice data is based on both utterances and the “voice data storage destination address” for storing the voice data are associated with each other. For example, the voice data of “voice data No .: 201” is a continuous utterance section having a start time of 12:36:03 and an end time of 12:36:16. Indicates that the clerk ’s voice and the clerk ’s voice overlap.

  FIG. 9B is a diagram showing an example of the store clerk utterance section management table 86. The clerk utterance section management table 86 includes a “clerk utterance number” assigned to each clerk utterance section, a “clerk utterance start time” that is the start time of the clerk utterance section, and a “clerk clerk utterance section end time” End of the overlapping section of "Speech end time", "Speaking No." indicating which store clerk talk section belongs to, "Overlapping start time" which is the start time of overlapping section with customer utterance, and customer utterance The “duplication end time”, which is the time, is associated with each other. For example, the utterance sections of “Clerk utterance No .: 100” and “Clerk utterance No .: 101” are regarded as a series of talk sections because the interval is less than a predetermined time X (for example, 3 seconds), and are the same. Talk No. Is added. Further, the utterance section of “clerk utterance number: 100” indicates that the entire section overlaps with the customer's utterance.

  FIG. 9C is a diagram illustrating an example of the customer utterance section management table 87. The customer utterance section management table 87 includes “customer utterance No.” given to each customer utterance section, “customer utterance start time” which is the start time of the customer utterance section, and “customer utterance section end time”. End of the overlapping section of the “speech end time”, “speaking No.” indicating which customer speaking section it belongs to, “duplication start time” which is the start time of the overlapping section with the clerk utterance, and the clerk utterance The “duplication end time”, which is the time, is associated with each other. For example, the utterance sections of “customer utterance No .: 101” and “customer utterance No .: 102” are regarded as different talk sections because the intervals exceed a predetermined time X (for example, 3 seconds). Talk No. Is granted. Further, the utterance section of “customer utterance No .: 100” indicates that 6 seconds out of the section length of 13 seconds overlaps with the clerk's utterance.

  Next, the audio data storing process will be described with reference to the flowchart of FIG. As described above, the clerk terminal 5 and the management server 15 communicate with each other via the receipt printer 13, but the illustration is omitted because the receipt printer 13 simply passes information.

  When the store clerk terminal 5 (control unit 25) acquires an audio signal (audio) from the audio acquisition microphone 2 (S11), the audio level is determined by the power filter of the audio level determination function (S12). Here, if the audio level is equal to or higher than the predetermined level, it is determined that the audio is “present”, and buffering of the audio data is started in the audio data storage area (not shown) in the memory 26 (S13). At this time, the recording start time is recorded in the audio data storage area.

  Subsequently, when no audio signal is received, the recording end time is determined and recorded in the audio data storage area, and the buffering is ended (S14). Thereafter, transmission of audio data is declared to the management server 15 (S15), and the audio data buffered in the audio data storage area is transmitted together with the recording start time and recording end time (S16).

  When the management server 15 (control unit 53) receives the voice data from the store clerk terminal 5 (S17), the voice data No. uniquely determined in the voice data management table 85 (see FIG. 9A) is obtained. Then, the recording start time and recording end time are registered (S18). Also, the audio data is stored in the audio data storage destination (predetermined folder) designated in the audio data management table 85 (S19).

  Next, with reference to the flowcharts of FIGS. 11 to 14, a series of flow of the service area identification process will be described. FIG. 11 is a flowchart showing the main process (service section identification process), and FIGS. 12 to 14 are flowcharts showing the subroutine. As shown in FIG. 11, the management server 15 (control unit 53) first specifies a store clerk talk section (S21), then specifies a customer talk section B that occurs after the store clerk talk section (S22), and also stores the store clerk talk. The customer talk section A that occurred before the section is specified (S23). Thereafter, a conversation section is specified based on the conversation section specified by the steps S21 to S23 (see S24, FIG. 5A). Further, by repeating the steps S21 to S24, the customer service section is specified (S25, see FIG. 5B).

  Next, with reference to the flowchart of FIG. 12, the clerk talk section specifying process corresponding to S21 of FIG. 11 will be described. When the clerk terminal 5 acquires the detection data from the body conduction sound sensor 1 (S31), the clerk terminal section determines the detection level by the power filter of the clerk utterance section specifying function. The time is determined and recorded in the memory 26 (S32). Subsequently, when the detection level is determined by the power filter of the store clerk utterance section specifying function and the state below the predetermined level is maintained for a certain period of time (when no signal section occurs), the detection level is above the predetermined level. The last detection time is determined as the clerk utterance section end time and recorded in the memory 26 (S33). Thereafter, the management server 15 is notified of the occurrence of the clerk utterance section (S34). At this time, the clerk utterance start time and the clerk utterance end time recorded in the memory 26 are also transmitted.

  On the other hand, when the management server 15 (control unit 53) receives the notification of the occurrence of the clerk utterance section from the clerk terminal 5 (S35), it is uniquely determined in the clerk utterance section management table 86 (see FIG. 9B). Sales clerk utterance data No. And the clerk talk number set for each clerk talk section. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereafter, it is determined whether or not a notification of occurrence of the next clerk utterance section is received within a predetermined time (S37). If received (S37: Yes), the clerk utterance data No. And the same clerk Talk No. Then, the clerk utterance start time and the clerk utterance end time are registered (S36). Thereby, the salesclerk utterance section generated before and the salesclerk utterance section generated this time can be defined as a series of talk sections. Note that, when the occurrence notification of the next clerk utterance section is not received within a predetermined time (S37: No), the process is terminated assuming that a series of talk sections by the clerk is over.

  Next, with reference to the flowchart of FIG. 13, the customer talk section B specifying process corresponding to S22 of FIG. 11 will be described. The management server 15 (the control unit 53) refers to the voice data management table 85 after the end of identification of a series of talk sections by the clerk, and the voice data is stored within a predetermined time from the clerk utterance end time of the last clerk utterance section. It is determined whether or not it exists (S41). Here, when there is no voice data (S41: No), it is determined that the customer talk section B does not exist, and the process ends. If the voice data exists (S41: Yes), the recording start time and the recording end time of the voice data are read from the voice data management table 85, and the uniquely defined customer utterance number. And customer talk No. determined for each customer talk section. Then, the customer utterance start time and customer utterance end time are registered in the customer utterance section management table 87 (S42).

  Thereafter, referring to the voice data management table 85, it is determined whether or not voice data exists within a predetermined time from the customer utterance end time of the last customer utterance section (S43), and if present (S43: Yes). Unique customer utterance No. And the same customer talk number as above. The customer utterance start time and customer utterance end time are registered (S42). Thereby, the customer utterance section generated before and the customer utterance section generated this time can be defined as a series of talk sections. If there is no voice data within a predetermined time from the customer utterance end time of the last customer utterance section (S43: No), the process is terminated assuming that there is no subsequent talk section by the customer.

  Next, with reference to the flowchart of FIG. 14, the customer talk section A specifying process corresponding to S23 of FIG. 11 will be described. The management server 15 refers to the voice data management table 85 after the identification of the customer talk section B is completed, and unprocessed voice data based on the customer's utterance from the clerk utterance start time of the clerk utterance section to a predetermined time before. Whether or not exists is determined (S51). Here, when there is no unprocessed audio data (S51: No), it is determined that the customer talk section A does not exist, and the process is terminated. If unprocessed voice data exists (S51: Yes), the recording start time and the recording end time of the voice data are read, and the uniquely defined customer utterance number. And customer talk No. determined for each customer talk section. The customer utterance start time and customer utterance end time are registered in the customer utterance section management table 87 (S52).

  Thereafter, referring to the voice data management table 85, it is determined whether or not unprocessed voice data exists before a predetermined time from the customer utterance start time of the registered customer utterance section (S53). S53: Yes), the recording start time and the recording end time of the voice data are read, and the customer utterance No. uniquely determined is read. And the same customer talk number as above. Then, the customer utterance start time and customer utterance end time are registered (S52). Thereby, the customer utterance section registered previously and the customer utterance section specified this time can be defined as a series of talk sections. If there is no unprocessed voice data within a predetermined time before the customer utterance start time of the registered customer utterance section (S53: No), the processing is ended assuming that there is no previous talk section by the customer. To do.

  As described above, registration into the voice data management table 85, the clerk utterance section management table 86, and the customer utterance section management table 87 is performed by the processes shown in FIGS. And the customer talk section B can be specified. Here, the detection of the overlap section between the store clerk talk section and the customer talk section will be briefly described. After specifying each talking section, the management server 15 refers to the store clerk utterance section management table 86 and the customer utterance section management table 87, detects an overlapping section, and records it in each table 85, 86, 87. Specifically, first, referring to the clerk utterance section management table 86, the same talk No. The earliest clerk utterance start time and the latest clerk utterance end time are set as the start time and end time of the clerk talk section. Similarly, referring to the customer utterance section management table 87, the same talk No. The earliest customer utterance start time and the latest customer utterance end time are set as the start time and end time of the customer talk section. Then, it is determined whether or not there is an overlapping section between the clerk talk section and the customer talk section. If there is a duplicate section, the clerk utterance section management table 86 and the customer utterance section management table 87 indicate the overlapping sections (duplication start time and duplication end time). Time) and a duplication flag is set in the audio data management table 85 (corresponding to “customer / store” in the duplication flag column of FIG. 9A).

  Next, a method for measuring the number of times of duplication will be described with reference to FIG. In the example shown in FIG. 15A, there are four overlapping sections in which the store clerk talk section and the customer talk section overlap. However, since the second overlapping section is a short utterance section (talking section of a predetermined time or less), it is not subject to overlap measurement. Therefore, in the example of the figure, three times of overlap measurement sections (1) to (3) are measured as the number of overlaps.

  Note that since the overlap measurement sections (2) and (3) are sections included in one talk section, this may be measured as one overlap measurement section. In this case, in the example shown in FIG. 15A, the number of times of duplication is measured as two.

  In addition, all overlapping sections of the talking section having a predetermined time or less may be subjected to overlap measurement. In this case, in the example shown in FIG. 15A, the number of times of duplication is measured as four.

  Further, as in the example shown in FIG. 15B, only the sections that are overlapped by talking from the store clerk may be subject to overlap measurement. In the example of the figure, there are four overlapping sections in which the store clerk talk section and the customer talk section overlap. However, since the second and fourth overlapping sections are overlapping sections due to a talk from the customer, they are not subject to overlapping measurement. That is, when a customer's talk starts in a state where the store clerk talk section is started in advance, even if an overlap section occurs, it is not subject to overlap measurement. Therefore, in the example of the same figure, two times of the overlap measurement sections (1) and (2) are measured as the number of overlaps. In this way, by setting only the section that overlaps due to the conversation from the store clerk as the object of overlap measurement, the store clerk's customer service method can accurately determine whether or not it is good.

  Next, a viewer screen D for confirming customer service data will be described with reference to FIGS. 16 and 17. The viewer screen D is displayed on the display screen 16a of the display terminal 16. FIG. 16 is a diagram showing a viewer screen D01 of the conversation ratio list. The viewer screen D01 includes an inquiry condition selection area E11 for selecting an inquiry condition, a list display area E12 for displaying a list of conversation ratios, and a reproduction operation area E13 for performing an operation for reproducing recorded data included in the customer service data. And are included.

  In the inquiry condition selection area E11, “Store”, “Date”, and “Clerk (person identification information)” can be selected (input). The customer service data that meets the conditions selected (input) here is displayed in the list display area E12. Note that it is possible to select (input) “customer (person identification information)” instead of “store clerk”, and display the customer service data associated with the customer in the list display area E12. Further, both “clerk” and “customer” can be selected (input), and customer service data satisfying these and conditions or conditions may be displayed in the list display area E12.

  The list display area E12 includes, for each customer service data, “store”, “date”, “store clerk”, “customer service section No.”, “customer service start”, “customer service end”, “relative conversation ratio”, “duplication number”. , “Customer”, “sales amount”, and “sales number” are related and displayed in a table format. “Service section No.” is an identification number automatically assigned to each service section, and “service start” and “service end” indicate the start time and end time of the service section. “Relative conversation ratio” and “number of duplications” refer to the relative conversation ratio and the number of duplications in the customer service section. “Customer” displays the name of the customer who is the object of the customer service, and “Sales Amount” and “Sales No.” are the total amount of the receipt R and the receipt No. obtained from the conversion data. Point to. In the list display area E12, any one of the customer service data (rows) can be selected (see the frame 211), and the recorded data included in the selected customer service data can be reproduced in the reproduction operation area E13. .

  The reproduction operation area E13 displays a button group 212 for performing a reproduction operation of recording data and imaging data in the service section, a progress bar 213 indicating a reproduction position, and a volume adjustment slider 214. Although not particularly illustrated, the management server 15 includes a reproduction unit (including an audio output unit such as a speaker) for reproducing the recording data and the imaging data in accordance with the operation of the reproduction operation area E13. The progress bar 213 has a time scale in minutes in the horizontal axis direction. In addition, the progress bar 213 distinguishes and displays the clerk talk section, the customer talk section, the overlap section, and the non-conversation section that does not correspond to any of these sections. In the figure, these can be identified by different hatches and blanks, but any display form can be used as long as the user can identify each section such as color-coded display, mark-added display, and text-added display. The scale of the progress bar 213 may be a time unit instead of a minute unit. Further, the unit of the scale and the scale interval may be varied according to the length of the recorded data so that the reproduction status of the entire recorded data in the customer service data can be grasped.

  FIG. 17 is a diagram showing a viewer screen D02 of a conversation ratio-sales performance correlation diagram. The viewer screen D02 includes an inquiry condition selection area E21 for selecting an inquiry condition, and a correlation diagram display area E22 for displaying a correlation diagram between the conversation ratio and the sales performance (sales information).

  The inquiry condition selection area E21 can select (input) “store”, “date section”, and “store clerk (person identification information)”. In addition, “clerk” can select “ALL (all clerk)”. The correlation diagram display area E22 is displayed based on the customer service data that matches the condition selected (input) here.

  The correlation diagram display area E22 is a scatter diagram in which the horizontal axis represents the customer conversation ratio (unit:%) and the vertical axis represents the customer unit price (unit: yen). Here, on the basis of all customer service data received on March 5 by all the store clerk, “points” that are the intersections of the average conversation ratio and sales information (customer unit price) are plotted. Thereby, the user can grasp at a glance how much the sales ratio is improved for the store. For example, in the example shown in the figure, it can be estimated that by increasing the customer conversation ratio to about 70%, the sales can be improved.

  As described above, according to the customer service support system SY1 according to the first embodiment, by calculating the conversation ratio between the store clerk and the customer, this can be collected and used for management strategy, or used for customer service skill training. can do. In addition, since the clerk identification information and the customer identification information are associated with the customer service data, it is possible to specify which store clerk is serving the customer for which customer. Thereby, appropriate customer service training can be performed for each store clerk. Furthermore, since sales information is associated with customer service data, the correlation between the conversation ratio and sales can be collected as marketing data.

  In addition, since the conversation ratio is calculated for each customer service section and displayed on the viewer screen D, whether or not each customer service is an overall favorable customer service (for example, the length of the clerk voice: the length of the customer voice is Whether or not the value is close to 2: 8 can be grasped. Further, since the recorded data is associated with the customer service data, for example, the recorded data associated with a preferable conversation ratio can be extracted and reproduced to be used as a service training material. That is, since the customer service situation of a salesclerk with high conversation skills can be efficiently shown to other salesclerks, it is useful for horizontal development of conversation skills.

  In addition, since the number of times of duplication is associated with the customer service data, it can be estimated whether or not the customer service is preferable from the number of times of duplication as well as the conversation ratio. For example, when the number of times of duplication is large, even if the conversation ratio is a preferable value, it can be determined that the customer service is not preferable.

  As a modification of the first embodiment, the number of customer service points based on the conversation ratio and the number of overlaps may be calculated and displayed on the viewer screen D. Hereinafter, the modification will be described with reference to FIG. As shown in FIG. 18A, the number of customer service points is calculated using “conversation ratio level” and “duplication count level” as parameters. Further, the “conversation ratio level” and the “duplication number level” are multiplied by weights P1 and P2, respectively, and the total value thereof becomes the number of customer service points. In general, the weighting is 0 ≦ P2 ≦ P1 ≦ 1, and P1 is larger than P2. In other words, the number of customer service points is calculated with a higher weight for the “conversation ratio level” than for the “duplication number level”. However, it is preferable that the user can arbitrarily set depending on the situation of the store used.

  Here, the “conversation ratio level” is higher as the conversation ratio between the clerk and the customer is closer to 2: 8. That is, as shown in FIG. 18B, the “conversation ratio level” is a numerical value of “0 to 3” depending on the customer conversation ratio. Further, the “duplication number level” is higher as the number of duplication is smaller. That is, as shown in FIG. 18C, the “duplication number level” is a numerical value of “0 to 3” depending on the number of duplications.

  Thus, by calculating the number of customer service points, it is possible to objectively determine the customer service. Further, by recording the determination result and the number of customer service points as part of the customer service data and displaying them on the viewer screen D, managers such as store managers and managers can check the customer service results at a glance.

  It should be noted that at each end of the conversation section or each end of the customer service section, a determination result as to whether or not the customer service is favorable and / or the number of customer service points may be notified to the target store clerk. In this case, the management server 15 determines the customer service and calculates the customer service score, and receives the information from the voice instruction earphone (not shown) attached to the store clerk via the receipt printer 13 and the store clerk terminal 5. It is preferable to notify. According to this configuration, since it is possible to grasp whether or not the store clerk is performing a favorable customer service during the customer service, an improvement in the customer service method can be expected.

  In the above embodiment, a group of customer talk sections before and after the store staff talk section (that is, a collection of at least one and up to three talk sections) is defined as “one conversation section”. The restriction on the number of talking sections included in “one conversation section” may be eliminated. That is, a group of talking sections in which the talking sections of the store clerk and the customer are alternately repeated without interruption for a predetermined time (interval Y) or more may be set as “one conversation section”.

  Moreover, although said embodiment illustrated the case where the conversation ratio of a salesclerk and a customer was calculated, the relationship between persons is not limited to this. That is, it may be a company boss and subordinate, a couple, a couple, or friends.

  In the above embodiment, the conversation ratio is calculated for each conversation section or customer service section. However, the conversation ratio within a predetermined time may be calculated. For example, the conversation ratio may be calculated based on the clerk and customer voices spoken during a predetermined time of 10 minutes. Moreover, you may calculate the conversation ratio in the daily work of a store clerk.

  In the above embodiment, the conversation ratio is calculated for each conversation section or for each customer service section. However, simply based on the conversation section of the adjacent store clerk and the customer (based on the ratio of the two conversation sections), The conversation ratio may be calculated.

  In the above embodiment, the person identifying unit 152 identifies the customer using the face recognition technique, but other methods may be employed. For example, each customer possesses a membership card with a built-in wireless IC (RFID: Radio Frequency Identification, etc.), and this is read with an IC reader installed at the entrance of the store to identify the customer, thereby identifying the customer You may do it. Further, the store clerk may be obliged to have a store clerk card in which a wireless IC is incorporated, and the store clerk may be identified by reading the store clerk card. By this method, when the read clerk card and the membership card are read at the same time, it is possible to associate the two as if the clerk is serving the customer. Further, the customer may be identified by reading the membership card (magnetic card) on which the magnetic information is recorded with a magnetic card reader attached to the POS terminal 12. In this case, the store clerk in charge of customer service can associate the customer with the store clerk by having his / her store clerk card read at the same time. The magnetic card reader may be directly connected to the management server 15. Furthermore, voice recognition technology may be used instead of face recognition technology. In this case, the customer information storage unit 81 and the store clerk information storage unit 82 need to store a voiceprint instead of the face feature amount.

  In the above embodiment, the imaging result of the in-store camera 11 is directly transmitted to the management server 15 via the wired LAN, but may be transmitted to the management server 15 via the receipt printer 13. Conversely, the clerk terminal 5 is configured to transmit audio data to the management server 15 via the receipt printer 13, but may be directly transmitted from the clerk terminal 5 to the management server 15. Each function of the management server 15 may be realized by a POS system or a WWW server.

[Second Embodiment]
Next, with reference to FIG. 19 thru | or FIG. 23, the customer service data recording device of this invention, the customer service data recording method, and a program are demonstrated as 2nd Embodiment. The customer service support system SY2 according to the present embodiment records customer service data in which conversation ratio data and satisfaction data are associated, and uses this as marketing data. Therefore, only differences from the first embodiment will be described. In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to 1st Embodiment is applied similarly about this embodiment.

  FIG. 19 is a functional block diagram of the customer service support system SY2 according to the second embodiment. In the management server 15 according to the present embodiment, compared to the first embodiment, a talk section extraction unit 161, a customer emotion recognition unit 162, and a customer satisfaction calculation unit 163 are added.

  The talk section extraction unit 161 corresponds to the voice extraction unit 154 of the first embodiment, and extracts a store clerk talk section and a customer talk section from the acquired conversation (voice data).

  The customer emotion recognition unit 162 applies the voice strength, the voice generation speed (number of mora per unit time, etc.), the strength for each word to the customer talking section extracted from the recorded data (voice data corresponding to the customer service section), Emotion recognition is performed based on the amount of change such as volume and voice spectrum. Specifically, emotion recognition is performed for each customer utterance section included in the recorded data. Thus, highly accurate emotion data can be obtained by performing emotion recognition for each phrase. Further, as shown in FIG. 15), the customer emotion recognition unit 162 identifies an overlapping section in which the customer talk section and the store clerk talk section overlap on the time axis, and identifies the overlap section as an “emotion recognition non-target section”. Then, emotion recognition is performed on the customer talk section excluding the overlap section. In this way, misrecognition can be prevented by performing emotion recognition excluding overlapping sections in which customer voice and store clerk voice are mixed and accurate emotion recognition cannot be performed.

  The customer satisfaction calculation unit 163 calculates customer satisfaction based on the recognition result of the customer emotion recognition unit 162. Further, as the customer emotion recognition unit 162 performs emotion recognition for each utterance section, the customer satisfaction calculation unit 163 also calculates the customer satisfaction for each utterance section. A specific method for calculating customer satisfaction will be described later.

  The customer service data recording unit 158 of the present embodiment associates the conversation ratio data based on the conversation ratio calculated by the conversation ratio calculation unit 155 and the satisfaction data based on the customer satisfaction calculated by the customer satisfaction calculation unit 163, Recorded in the management server database DB2 as part of the customer service data.

  FIG. 20 is an explanatory diagram of the management server database DB2 according to the second embodiment. The management server database DB2 is different from the first embodiment in the storage contents of the customer service data storage unit 91. The customer service data storage unit 91 of the present embodiment includes customer identification information, store clerk identification information, recording data, imaging data, sales information, customer service date, conversation ratio data based on the calculation result of the conversation ratio calculation unit 155, and customer satisfaction. The satisfaction degree data based on the calculation result of the degree calculation unit 163 is stored in association with each other. Here, the conversation ratio data refers to the conversation ratio in the customer service section and the conversation ratio in each conversation section. The satisfaction data refers to customer satisfaction in the customer service section and customer satisfaction in each conversation section.

  Next, a customer satisfaction calculation algorithm (calculation formula) will be described with reference to FIG. As shown in the figure, the customer satisfaction is calculated in the order of the utterance section, the conversation section, and the customer service section. First, as shown in FIG. 21A, the satisfaction level for each utterance section is calculated by the formula ““ satisfaction level for each utterance section ”=“ joy data ”+“ laughing data ”× A”. Here, “joy data” is emotion data of “joy” (for example, numerical values from 0 to 50, the same applies hereinafter), “laugh data” is emotion data of “laughter”, and A is a constant satisfying 0 ≦ A ≦ 1. Means. The calculation algorithm is derived from the idea that the degree of satisfaction of a person is based on the psychological state of “pleasant” and the multiplication result of the psychological strength.

  Further, as shown in FIG. 21 (b), the actual satisfaction level for each utterance section is calculated using the formula "" Real satisfaction level for each utterance section "=" Satisfaction level for each utterance section "-" Satisfaction level for each utterance section "× C". That is, “(“ joy data ”+“ laugh data ”× A) − (“ anger data ”+“ sad data ”× B) × C” is calculated. Here, “anger data” means “anger” emotion data, “sad data” means “sadness” emotion data, B means a constant satisfying 0 ≦ B ≦ 1, and C means a constant satisfying 0 ≦ C ≦ 1. To do. As described above, by using emotion data of “anger” and “sadness” in addition to “joy” and “laughter”, it is possible to calculate a more reliable satisfaction considering complex emotions. . The calculation algorithm is based on the idea that the person's dissatisfaction level is based on the psychological state of “uncomfortable” and the multiplication result of the psychological strength, and the psychological state where the actual satisfaction level is “pleasant” and “unpleasant”. It is derived from the idea of being based on.

  Further, as shown in FIG. 21C, the satisfaction level for each conversation section is ““ satisfaction level for each conversation section ”= an average value of“ actual satisfaction level for each utterance section ”of each customer utterance section included in the conversation section”. Obtained from. In addition, as shown in FIG. 21 (d), the satisfaction level by customer service section is obtained from ““ Satisfaction level by customer service section ”= average value of“ satisfaction level by conversation section ”of each conversation section included in the service section. It is done.

  Next, the viewer screen D according to the second embodiment will be described with reference to FIGS. FIG. 22 is a diagram showing a viewer screen D03 of the satisfaction level-conversation ratio list. The viewer screen D03 is displayed by, for example, a button operation (not shown) for displaying the “satisfaction-conversation ratio list” from the viewer screen D01 shown in FIG. Section No. indicating A display area E31 and a list display area E32 for displaying a list in which the customer satisfaction and the customer conversation ratio are associated are included. The list display area E32 includes “time” indicating the start time and end time of the service section, “conversation section No.” of each conversation section included in the service section, “customer satisfaction” of each conversation section, and each conversation. The “customer conversation ratio” of the section is related and displayed in a table format. Here, “customer satisfaction” of each conversation section refers to the satisfaction degree by conversation section shown in FIG.

  FIG. 23 is a diagram showing a viewer screen D04 of the satisfaction-conversation ratio overlay graph. The viewer screen D04 is displayed by, for example, a button operation (not shown) for displaying the “satisfaction-conversation ratio superimposed graph” from the viewer screens D01 and D03 shown in FIG. 16 or FIG. A customer service data display area E41 that displays part of the information included in the customer service data, and a graph display area E42 that graphs the relationship between customer satisfaction and the conversation ratio are included.

  The customer service data display area E41 includes “date”, “store clerk”, “customer”, “customer service time”, “sales number”, “sales amount”, “average customer conversation ratio”, “average customer satisfaction” and “Service section No.” is displayed. Here, “customer service time” refers to a customer service start time and a customer service end time. Further, the “average customer satisfaction” refers to the satisfaction by customer service section shown in FIG.

  The graph display area E42 includes a first line graph (shown by a solid line and a black circle) whose horizontal axis is “conversation section No.” and whose vertical axis is “customer satisfaction”, and whose horizontal axis is “conversation section No.”. Then, a superposition graph in which a second line graph (shown by broken lines and white circles) with the vertical axis as “conversation ratio” is overlaid is displayed. Here, the “conversation section No.” on the horizontal axis is arranged in time series. Instead of “conversation section No.”, the horizontal axis may be time (time). In addition, it is assumed that the “customer satisfaction” includes emotion data and values of constants A, B, and C as represented by numerical values of 0 to 100. The “conversation ratio” indicates the customer conversation ratio in unit%, and is represented by 0 to 100%. In this way, by displaying the transition of customer satisfaction in each conversation section and the transition of the conversation ratio in each conversation section within the customer service section in a graph on the common time axis, the user can receive one customer service. It is possible to grasp at a glance changes in conversation status, customer emotions, and their correlations.

  As described above, according to the customer service support system SY2 according to the second embodiment, the conversation ratio data and the satisfaction level data are associated with each other and recorded as the customer service data, so that it can be used as marketing data. In addition, the customer service data can estimate how much the conversation ratio affects customer satisfaction, and can prove the results of conversation training. In addition, the conversation ratio in each conversation section is recorded as conversation ratio data, and the customer satisfaction in each conversation section is recorded as satisfaction data, and these are displayed on the viewer screens D03 and D04, thereby providing conversation within one customer service. You can see changes in situations and customer sentiment. Furthermore, since the average conversation ratio and average customer satisfaction in each customer service section are recorded and displayed as customer service data (see E41 in FIG. 23), it is possible to easily grasp the overall evaluation of each customer service.

  In the above embodiment, the conversation ratio between the store clerk and the customer and the customer satisfaction are calculated for the purpose of customer service management, but they may be used by individuals. As a result, the collected conversation ratio data and satisfaction data can be used to improve conversation skills (conversation techniques) between individuals.

[Third Embodiment]
Next, with reference to FIG. 24 to FIG. 31, a customer service support apparatus, a customer service support method, and a program according to the present invention will be described as a third embodiment. The customer service support system SY3 according to this embodiment is characterized in that a customer service section is specified for each customer serving as a customer service based on a monitoring result of a monitoring unit that monitors a store clerk or a customer. Therefore, only differences from the above embodiments will be described. In the present embodiment, the same components as those in the above-described embodiments are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to said each embodiment is applied similarly about this embodiment.

  FIG. 24 is a functional block diagram of the customer service support system SY3 according to the third embodiment. In the present embodiment, the voice acquisition microphone 2 functions as a monitoring unit. That is, the monitoring unit includes a conversation acquisition unit 102 (other examples of the monitoring unit will be described later), and the conversation acquisition unit 102 acquires a conversation between a salesclerk and a customer.

  In addition, the management server 15 according to the present embodiment is different from the second embodiment in that the customer service switching detection unit 171, the switching detection data recording unit 172, the customer service switching segment identification unit 173, the customer service conversation segment identification unit 174, and the customer service segment identification. A part 175 is added.

  The customer service switching detection unit 171 detects the switching of the target customer who is the customer service target of the store clerk based on the monitoring result of the monitoring unit, that is, the voice of the customer included in the conversation acquired by the conversation acquisition unit 102. In the present embodiment, voice print authentication is periodically performed on the customer's voice, and the change of the target customer is detected based on a change in the voice print authentication result. Note that it is also possible to discriminate voice feature amounts (voice pitch, speed, etc.) other than the voiceprint from the customer's voice, and detect the change of the target customer based on the change in the feature amount.

  The switching detection data recording unit 172 associates the clerk identification information for identifying the clerk and the detection time (time stamp) of the customer service switching detection unit 171, and records them in the management server database DB 3 as switching detection data.

  Based on the recorded switching detection data, the service switching section identification unit 173 sets the Nth detection time (where N is an integer satisfying N ≧ 1) as the section start time and the N + 1th detection time based on the recorded switching detection data. The customer service switching section used as the section end time is specified. That is, the “customer service switching section” is a section specified based on the switching detection data.

  The service conversation section identification unit 174 identifies a service conversation section based on the conversation section extracted by the conversation section extraction unit 161. The “service section” according to the first embodiment and the second embodiment corresponds to the “service conversation section” according to the present embodiment. As described above, since the conversation section is a collection of conversation sections that are alternately repeated without interruption for a predetermined time or more by the store clerk and the customer, a conversation section that is repeated without interruption for a predetermined time or more is defined as 1 Specify as a customer service section. That is, the “customer service conversation section” is a section specified based on the recording data included in the customer service data.

  Further, the customer service section specifying unit 175 specifies the customer service section based on the customer service switching section specified by the customer service switching section specifying unit 173 and the customer service conversation section specified by the customer service conversation section specifying unit 174. Specifically, the customer service section is specified by performing an AND operation or an OR operation on the customer service conversation section and the customer service switching section. The customer service section identification unit 175 associates arbitrary switching detection data and recorded data with the store clerk identification information, and performs a comparison operation. A method for identifying a service section by the service section identifying unit 175 will be described in detail later.

  With the above configuration, the conversation ratio calculation unit 155 of the present embodiment calculates the conversation ratio in the service section specified by the service section specification unit 175. In addition, the customer service data recording unit 158 of the present embodiment corresponds to recorded data that is voice data corresponding to the customer service section specified by the customer service section specifying unit 175, or a customer service section specified by the customer service section specifying unit 175. Imaging data that is video data is recorded as customer service data. Further, the screen display unit 159 of the present embodiment, based on the user's operation, the customer service switching section specified by the customer service switching section specifying unit 173, the customer service conversation section specified by the customer service conversation section specifying unit 174, and the customer service section The customer service section specified by the specifying unit 175 is displayed on the viewer screen D (see, for example, FIGS. 26 to 30). Further, the start time and end time of the customer service section can be corrected on the viewer screen D based on the user's operation.

  FIG. 25 is an explanatory diagram of the management server database DB3 according to the third embodiment. The management server database DB3 further functions as a switching detection data storage unit 93 in addition to the units shown in the first and second embodiments. The switch detection data storage unit 93 stores the switch detection data recorded by the switch detection data recording unit 172. Further, the customer service data storage unit 94 of the present embodiment includes customer identification information, store clerk identification information, recording data corresponding to voice data for the customer service section, imaging data corresponding to video data for the customer service section, and conversation ratio in the customer service section. In addition, customer service section data is stored. The customer service section data refers to data including a section start time and a section end time of the service section.

  Next, the switching detection data and the switching detection period will be described with reference to FIG. As shown in FIG. 26A, the switching detection data is information in which “salesperson identification information”, “date”, and “switching detection time” are associated with each other. The “switching detection time” refers to the time when the customer service switching detection unit 171 detects the switching of the target customer. In the present embodiment, voice print authentication is periodically performed on the customer's voice, and when the voice print authentication result is changed (when a voice print of a different customer is recognized), it is determined that the target customer has been switched.

  FIG. 26B is an image diagram showing the customer service switching section on the time axis. The customer service switching section is defined as a new section generated every time switching is detected, and therefore, as shown in the figure, there is no gap between adjacent sections, and the section is a continuous section.

  Next, the customer service conversation section will be described with reference to FIG. FIG. 27A is a diagram illustrating an example of the service conversation section identification result. The service conversation section is specified by the service section specifying method described in the first embodiment. Here, “clerk identification information”, “date”, “customer identification information”, and “customer conversation section” are shown as specific results. Further, it is assumed that the reference length of the interval of the service conversation section (interval Z in FIG. 5B) is 1 minute 30 seconds.

  FIG. 27B is an image diagram showing the customer service section on the time axis. Since the customer service section is defined as a collection of conversation sections that are repeated without interruption for a predetermined time or longer, a gap occurs between adjacent sections as shown in FIG.

  Next, with reference to FIG. 28 to FIG. 30, a method for specifying the customer service section will be described. Here, as a method for specifying the service section, three patterns (service section specification patterns A to C) are illustrated here. FIG. 28 is a diagram illustrating a service section identification pattern A. The customer service section specifying pattern A specifies a customer service section based on the customer service conversation section. However, when a plurality of consecutive customer service conversation sections are included in one customer service switching section (a relationship between the customer service conversation sections (1) and (2) and the customer service switching section (1)), the start of the plurality of customer service conversation sections From the time to the end time is defined as one customer service section (a customer service section (1)). In addition, when the customer service switching section is interrupted in the middle of the customer service conversation section (the relationship between the customer service conversation section (3) and the customer service switching sections (2) and (3)), the customer service section is divided according to the time when the customer service switching section is interrupted. To do. That is, in the example of the figure, the service period (2) is defined as the service period (2) from the start time of the service conversation section (3) to the end time of the service switching period (2) (service conversation section). The service section (3) is defined from the start time of (3) to the end time of the customer service section (3). In addition, when the section length of the customer service section specified by these specifying methods is less than a predetermined time, it is preferable not to specify (ignore) that section as the customer service section.

  As described above, the customer service section identification pattern A can accurately identify the customer service section for each customer in the configuration in which the change of the target customer is detected based on the change of the customer's voiceprint as in the present embodiment. For example, if a customer service section is specified only by the customer service switch section (customer service switch section = customer service section), the target customer has already been switched, but it is erroneously determined that the previous customer service continues because there is no customer utterance. (For example, in the case where there is no service conversation section (2) in the service switching section (1), a time corresponding to the service conversation section (2) is added to the service conversation section (1)). . Therefore, by specifying the customer service section on the basis of the customer service conversation section, the error of the customer service section end time can be eliminated.

  Also, if the customer service section is specified only by the customer service conversation section (customer service conversation section = customer service section), it is erroneously determined that the customer is a different customer service because the conversation is interrupted for a predetermined time or more, even though the customer is the same target customer. (For example, the relationship between the service conversation sections (1) and (2) and the service switching section (1)). On the other hand, since the conversation has not been interrupted for a predetermined time or more even though the target customer is switched, it may be erroneously determined that the customer is the same customer service (for example, the customer service conversation section (3) and the customer service switching section). (Relationship between (2) and (3)). Therefore, it is possible to specify an accurate customer service section by comparing and calculating not only one of the customer service conversation section and the customer service switching section, but also the service section.

  Note that the customer service switching detection unit 171 of this embodiment periodically performs voiceprint authentication, and determines that the target customer has been switched when the voiceprint authentication result changes. However, the appearance of the same voiceprint within a predetermined time When the rate is equal to or lower than the predetermined value, the switching of the target customer may be detected. According to this configuration, since the target customer is not always alone, such as when the customer service target is a family member, by detecting the switching of the target customer according to the appearance rate of the same voice print within a predetermined time, It is possible to accurately identify the customer service section for each target customer. For example, it is conceivable that the change of the target customer is not detected while the voice print of the same person can be recognized once or more per minute. In addition, instead of detecting the change of the target customer according to the appearance rate of the same voice print within a predetermined time, if the same voice print is not detected continuously for a predetermined time or more, it may be determined that the target customer has been switched.

  Subsequently, the customer service section specifying pattern B will be described with reference to FIG. The customer service section identification pattern B extracts a section where the customer service conversation section and the customer service switching section overlap (performs a comparison operation under the “and” condition) to identify the customer service section. For example, the customer service sections (1) and (2) are sections included in the customer service switching section (1), and thus are the same sections as the customer service conversation sections (1) and (2). Moreover, since the customer service switching section (2) is included in the customer service conversation section (3), the customer service switching section (2) becomes the customer service section (3). Furthermore, the customer service conversation section (3) and the customer service switching section (3) are compared, and the overlapping portion is extracted as the service section (4). In this example as well, it is preferable not to identify a customer service section whose section length is less than a predetermined time.

  Subsequently, the customer service section specifying pattern C will be described with reference to FIG. The customer service section specifying pattern C specifies a customer service section based on the customer service switching section. For example, the customer service section (1) is the same section as the customer service switching section (1). However, when the customer service conversation section is longer than the customer service switching section (a relationship between the customer service conversation section (3) and the customer service switching section (2)), the end time of the customer service switching section is not set as the service section switching time. That is, in the example of the figure, the total of the customer service switching sections (2) and (3) is defined as the service section (2) (the service section (2) start time is the service switching section (2) start time). And the end time of the service section (2) is the end time of the service switching section (3)). In this example as well, it is preferable not to identify a customer service section whose section length is less than a predetermined time.

  Next, with reference to FIG. 31, the customer service section setting (modified example of this embodiment) will be described. In the present embodiment, the monitoring means is the voice acquisition microphone 2 and it is determined that the target customer has been switched when a change in the voiceprint of the customer is detected. That is, the same figure (a-1) was employ | adopted as a monitoring means (monitoring content). In this case, it is described that the service section specifying pattern A is preferably used as the service section specifying method ((b-1) in the figure), but other specific patterns may be used. That is, the customer service section specifying pattern B (FIG. (B-2)) and the service section specifying pattern C (FIG. (B-3)) may be employed. Further, as other service area identification methods, a service switching section = a service section (FIG. (B-4)) and a service conversation section = a service section (FIG. (B-5)) are adopted. You may do it.

  Further, as shown in (a-2) in the figure, as a monitoring content, a clerk keyword may be monitored. In this case, the customer service switching detection unit 171 performs voice recognition on the clerk's voice, and detects a target keyword by detecting a predetermined keyword by the voice recognition. The management server 15 is assumed to include a speech recognition unit including an acoustic analysis unit, an acoustic model, a language model, a word dictionary, and a text conversion unit. Note that the voice recognition unit preferably recognizes the clerk's voice included in the recording data for each utterance section. According to this configuration, it is possible to easily detect the change of the target customer by detecting the predetermined keyword. For example, the time at which “welcome” indicating the start of the customer service section is detected may be set as the switching detection time. In addition, the time at which “waiting for another visit”, “thank you for your purchase”, and “please wait for a while” indicating the end of the customer service section may be used as the switching detection time. In addition, a keyword to be uttered at the end of customer service is determined for each store, and the time when the keyword is detected may be set as the switching detection time. As a keyword in this case, it is preferable to adopt a keyword that is not used during normal customer service, such as “End” and “Bye Bye”. Furthermore, the change of the target customer may be detected more accurately by detecting both the start keyword indicating the start of the customer service section and the end keyword indicating the end of the service section. That is, it may be regarded as the start of the customer service switching section by detecting the keyword “I welcome you”, and may be regarded as the end of the customer service switching section by detecting the keyword “Waiting for another visit”. In this case, unlike the example of the above embodiment, a gap is generated between adjacent customer service switching sections.

  In addition, as shown in FIG. 3A-3, the in-store camera 11 may be used as monitoring means to monitor the behavior of the store clerk. In this case, the customer service imaging unit 111 that captures the customer service situation of the clerk and the customer functions as a monitoring unit, and the customer service switching detection unit 171 detects the switching of the target customer based on the video imaged by the customer service imaging unit 111. . Specifically, a store clerk is identified by image recognition of a video, and a change in the target customer is detected by detecting a predetermined behavior of the store clerk. The in-store camera 11 may be attached to a ceiling or a desk, or a small camera may be attached to a store clerk's clothes or body instead of the in-store camera 11. Further, as the predetermined action, a naturally occurring action such as a bow performed at the end of customer service is conceivable, and the detection time of the bow in this case may be set as the switching detection time. In addition, for each store, an action (signal) to be performed at the end of customer service may be determined, and the change of the target customer may be detected by detecting the action. As an action in this case, it is preferable to adopt an action that is not normally performed at the time of customer service, such as “perform a V sign toward the camera” or “move to a predetermined place”. Further, by detecting both a start action indicating the start of the customer service section and an end action indicating the end of the customer service section, the switching of the target customer may be detected more accurately. That is, it may be regarded as the start of the customer service switching section by detecting the action of “V sign toward the camera” and the end of the customer service switching section by detecting the action of “bow”. Also in this case, a gap is generated between adjacent customer service switching sections.

  Further, as shown in FIG. 4A-4, an angle sensor (not shown) may be used as monitoring means to monitor the operation of the store clerk. In this case, an operation detection unit (not shown) that is attached to the store clerk and detects the operation of the store clerk functions as a monitoring unit, and the customer service switching detection unit 171 switches target customers based on the detection result of the operation detection unit. Is detected. The motion detection unit is preferably attached to the upper half of the store clerk. Further, a gravity sensor or a gyro sensor may be employed instead of the angle sensor. Moreover, it is preferable that an operation | movement detection part outputs the detection result to the terminal 5 for salesclerks, and the output result is transmitted to the management server 15 from the terminal 5 for salesclerks. In this case, the customer service switching detection unit 171 detects the switching of the target customer by detecting the motion of the store clerk being tilted by the motion detection unit. According to this configuration, since it is possible to determine that the bow performed at the end of customer service has been performed by the operation of tilting the upper body of the store clerk, switching of the target customer can be accurately detected by detecting the operation.

  In addition, as the operation of the store clerk, in addition to the naturally occurring operation as described above, an operation to be performed at the end of the customer service is determined in advance for each store, and the change of the target customer is detected by detecting the operation. good. As an operation in this case, for example, an operation that is not normally performed at the time of customer service such as “touching a clerk card” or “tapping a pocket” is conceivable. Further, as the operation detection unit, another sensor such as a contact sensor or an infrared sensor may be employed. In addition to the sensor, various operation means such as a touch button operated by a store clerk may be used as the motion detection unit. Furthermore, the switching of the target customer may be detected more accurately by detecting both the start operation indicating the start of the service section and the end operation indicating the end of the service section. In other words, it may be regarded as the start of the customer service switching section by detecting the operation of “touching the clerk card”, and may be regarded as the end of the customer service switching section by detecting the operation of “bowing”. Also in this case, a gap is generated between adjacent customer service switching sections.

  As described above, the user can select an arbitrary candidate from a plurality of options as the monitoring means and the customer service section specifying method by using the input device 55 provided in the management server 15 or the like. Further, the combination of the monitoring means and the customer service section specifying method is arbitrary, and can be changed according to the use situation and the needs of the user.

  As described above, according to the customer service support system SY3 according to the third embodiment, a change in the target customer serving as a customer is detected from the monitoring result of at least one of the customer and the customer, and the detection time is identified as the customer clerk. Since it is associated with information and recorded as switching detection data, the customer service switching section can be specified from the switching detection data. In addition, since the switching detection data is recorded, it can be aggregated as marketing data, which can be used to improve customer service skills of each store clerk.

  In addition, since the customer service switching section specified based on the switching detection data and the customer service conversation section specified based on the recording data are compared and specified to determine the customer service section, the customer service section for each customer to be serviced is accurately determined. Can be specified. Further, by specifying an accurate customer service section, a highly reliable conversation ratio can be calculated. Further, since the calculated conversation ratio is recorded as a part of the customer service data, it can be used as a teaching material for the customer service method, and it can be determined whether or not each customer (customer service data) is a good customer service. Thereby, for example, the customer service situation considered to be a good customer service based on the conversation ratio can be used for horizontal development of customer service skills, such as showing each store clerk.

  In the above embodiment, the conversation ratio in the customer service section specified by the customer service section specifying unit 175 is calculated. However, the customer satisfaction in the customer service section is calculated, and the conversation ratio data and the satisfaction data based on these are calculated. You may record as correlation and customer service data. That is, it is good also as a structure which combined 2nd Embodiment and 3rd Embodiment.

  While the first to third embodiments have been described above, it is possible to provide each process of the customer service support systems SY1, SY2, and SY3 shown in each of these embodiments as a program. Further, the program can be provided by being stored in various recording media (CD-ROM, flash memory, etc.). That is, a program for causing a computer to function as each component of the customer service support system SY1, SY2, SY3 and a recording medium on which the program is recorded are also included in the scope of rights of the present invention. In addition, the system configuration and processing steps of the customer service support systems SY1, SY2, and SY3 can be appropriately changed without departing from the gist of the present invention, regardless of the above-described embodiments.

  DESCRIPTION OF SYMBOLS 1 ... Body-conduction sound sensor 2 ... Sound acquisition microphone 5 ... Clerk terminal 6 ... Viewer screen 11 ... In-store camera 12 ... POS terminal 13 ... Receipt printer 14 ... Cash register counter 15 ... Management server 16 ... Display terminal 16a ... Display screen 18 ... Product master 19 ... Network 212 ... Button group 213 ... Progress bar 214 ... Volume control slider D ... Viewer screen DB ... Management server database E11 ... Inquiry condition selection area E12 ... List display area E13 ... Reproduction operation area E21 ... Inquiry condition selection Area E22 ... Correlation diagram display area E32 ... List display area E41 ... Customer service data display area E42 ... Graph display area R ... Accounting receipt SY ... Customer service support system

Claims (18)

A conversation acquisition unit for acquiring a conversation between the person A and the person B;
A voice extraction unit for extracting A voice, which is the voice of the person A, and B voice, which is the voice of the person B, from the acquired conversation;
Conversation ratio calculation for calculating a conversation ratio that is a ratio of the length of the A voice and the B voice or a ratio of the length of the A voice or the B voice to the total length of the A voice and the B voice A conversation ratio calculation device. An utterance detection unit that is attached to the person A and detects that the person A uttered;
The voice extraction unit determines whether the voice included in the conversation is the voice of the person A or the voice of the person B based on the detection result of the utterance detection unit, and based on the determination result 2. The conversation ratio calculation apparatus according to claim 1, wherein each voice is extracted. The person A and the person B refer to a store clerk and a customer during customer service,
The conversation ratio calculation apparatus according to claim 1 or 2, wherein the A voice and the B voice indicate a clerk voice and a customer voice. When a continuous speech section without breathing is a single utterance section, and a group of the utterance sections of the store clerk or the customer repeated without interruption for a predetermined time or more is a single talk section,
The length of the clerk voice refers to the length of the talk section by the clerk,
The conversation ratio calculation apparatus according to claim 3, wherein the length of the customer voice indicates a length of a talk section by the customer. When a continuous speech section without breathing is a single utterance section, and a group of the utterance sections of the store clerk or the customer repeated without interruption for a predetermined time or more is a single talk section,
The length of the clerk voice refers to the total length of each utterance section included in the talk section by the clerk,
4. The conversation ratio calculation apparatus according to claim 3, wherein the length of the customer voice indicates a total length of each utterance section included in the talk section by the customer. When the conversation section of the store clerk and the customer is alternately repeated without interruption for a predetermined time or more as a conversation section,
The conversation ratio calculation apparatus according to claim 4, wherein the conversation ratio calculation unit calculates the conversation ratio for each conversation section based on one or more conversation sections included in the conversation section. When a set of conversation sections that are repeated without interruption for a predetermined time or longer is defined as one customer service section,
The conversation ratio calculation apparatus according to claim 6, wherein the conversation ratio calculation unit calculates an average value of conversation ratios of the conversation sections included in the service section as a conversation ratio in the service section. The conversation section of the store clerk and the customer is alternately repeated without interruption for a predetermined time or more, and the conversation section is repeated for one conversation section, and the conversation section is repeated for a predetermined time or more without interruption. When
The conversation ratio calculation unit includes ΣLa (where ΣLa is a variable satisfying ΣLa ≧ 0) that is the sum of the length of each talking section or the total length of each utterance section included in the customer service section. ΣLb (where ΣLb is a variable satisfying ΣLb ≧ 0), and the ratio of ΣLa and ΣLb, or ΣLa or 6. The conversation ratio calculation apparatus according to claim 4, wherein a ratio of ΣLb to (ΣLa + ΣLb) is calculated as a conversation ratio in the customer service section. Each part in the conversation ratio calculation device according to claim 7 or 8,
A person identification unit for identifying the store clerk and the customer who are parties to the conversation acquired by the conversation acquisition unit;
A sales information acquisition unit that acquires sales information that is a sales record for the customer of the store clerk identified by the person identification unit;
A customer service data recording unit that associates the sales clerk and the customer identified by the person identification unit with the conversation ratio in the customer service section calculated by the conversation ratio calculation unit and the sales information, and records it as customer service data; A customer service data recording device comprising: A voice data recording unit that records voice data corresponding to the customer service section of the conversation acquired by the conversation acquisition unit;
The said customer service data recording part associates the recording data of the said audio | voice data recording part with the said salesclerk and the said customer, the said conversation ratio, and the said sales information, and records it as said customer service data. Hospitality data recording device. In the customer service section, further comprising a duplication number measuring unit that measures the number of times of duplication, which is the number of times that the voice of the clerk and the voice of the customer are duplicated,
The customer service data recording unit according to claim 9 or 10, wherein the customer service data recording unit associates the number of times of duplication with the store clerk and the customer, the conversation ratio, and the sales information, and records them as the customer service data. apparatus. A screen display unit for displaying a viewer screen for confirming the customer service data;
The screen display unit extracts the customer service data including the person identification information by selecting or inputting the person identification information for identifying the store clerk and / or the customer, and displays the data on the viewer screen. The service data recording apparatus according to claim 10 or 11.   13. The customer service data recording apparatus according to claim 12, wherein the screen display unit displays a playback operation area for performing a playback operation of the recorded data included in the extracted customer service data on the viewer screen. . The playback operation area includes a progress bar indicating a playback position,
14. The service data recording apparatus according to claim 13, wherein the progress bar distinguishes and displays a talk section by the store clerk, a talk section by the customer, and an overlapping section in which both sections overlap.   The said screen display part displays the correlation diagram which illustrated the correlation with the said conversation ratio contained in the said customer service data, and the said sales information on the said viewer screen, The any one of Claim 12 thru | or 14 characterized by the above-mentioned. Service data recording device according to item. Computer
A conversation acquisition step of acquiring a conversation between the person A and the person B;
A voice extraction step of extracting A voice, which is the voice of the person A, and B voice, which is the voice of the person B, from the acquired conversation;
Conversation ratio calculation for calculating a conversation ratio that is a ratio of the length of the A voice and the B voice or a ratio of the length of the A voice or the B voice to the total length of the A voice and the B voice And a step of executing a conversation ratio calculation method. The computer is attached to the person A and connected to an utterance detection unit that detects that the person A uttered,
In the voice extraction step, it is determined whether the voice included in the conversation is the voice of the person A or the voice of the person B based on the detection result of the utterance detection unit, and based on the determination result The method according to claim 16, wherein each voice is extracted.   The program for making a computer perform each step in the conversation ratio calculation method of Claim 16 or 17.

Images