JP5333175B2 - Operator support device, operator support program, and operator support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract the supplemental phrases matching the procedure manual explanation phrases. <P>SOLUTION: The operator support 1 has a correspondence relation memory 2, an operation time obtaining section 3, a section estimating section 4, and a supplemental phrase extracting section 5. The correspondence relation memory 2 stores the procedure manual explanation phrases and their operating positions by correlating them. The operation time obtaining section 3 obtains the operation time showing the time point that the predetermined operating section has been changed from the PC operation log storage storing the operated time points and positions by correlating them for every operated time point. The section estimating section 4 extracts the procedure explanation phrases relating to the operation after switching from the correspondence relating memory 2. From the recorded voice, the section estimating section 4 presumes the section to the start of the speaking time, longer than predetermined and appearing next to the speaking time starting after the operation time before the speaking section showing the procedure explanation phrase, as the voice section of the procedure manual explanation phrases. The supplemental phrase extracting section 5 extracts the supplemental phrases for the procedure manual explanation phrase from the voice section already estimated. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The technology disclosed in the present application relates to an operator support apparatus, an operator support program, and an operator support method.

  Many companies have a call center to respond to customer phone inquiries. In this call center, an agent specialized in customer service (hereinafter referred to as an operator) responds to inquiries from customers. When the operator receives an inquiry from the customer, the operator searches the electronic manual corresponding to the inquiry content using the terminal device, and explains the answer to the customer while referring to the search result.

  As this electronic manual, for example, QA (Question and Answer) cases are used. Such a QA example is a query item (Q) associated with an answer item (A) indicating a procedure until the query item is resolved. This answer procedure includes words (hereinafter referred to as procedure explanation words) that are key (important) when one procedure is described. The operator performs the response work by explaining each procedure centering on the procedure explanation words.

  By the way, the level of knowledge of customers who make inquiries to the call center varies. For this reason, even if an operator reads out the answer matter of the QA example displayed on the terminal device, the customer cannot always understand the content. If the customer's understanding does not proceed smoothly, the response time becomes long, and the customer satisfaction may be reduced. For this reason, the operator is required to respond flexibly such as changing the method of explaining the answer items according to the knowledge level of the customer. However, it is difficult for a beginner operator to perform such a response.

  For this reason, there has been proposed an operator support apparatus that presents words that supplement the procedure explanation words (hereinafter referred to as supplement words) as support keywords. This operator support device extracts supplemental phrases from voice data of cases that are the same case and are handled by different operators. That is, the operator support apparatus matches a keyword uttered by a skilled operator with a keyword uttered by a novice operator. At this time, the operator support device extracts keywords that are spoken in the vicinity of the time when the procedural explanation words are uttered and that are spoken only by skilled operators among skilled operators and novice operators as supplementary phrases. . As described above, when displaying the QA case, the operator support device displays the supplemental words and phrases along with the procedure explanation words and thereby supports the operator's reception work.

JP 2008-33835 A JP 2001-51876 A JP 10-69381 A JP-A-2005-293180

  However, the above-described conventional technique has a problem that it is not always possible to extract a supplemental phrase corresponding to the procedure explanation phrase.

  This point will be described with reference to FIGS. FIG. 21 and FIG. 22 are explanatory diagrams for explaining a supplement word extraction procedure in the prior art. In the examples of FIGS. 21 and 22, it is assumed that an inquiry about an operation of a personal computer (hereinafter referred to as a PC) is received at a call center. FIG. 21 and FIG. 22 both show an example of the dialog explaining the procedure X “Open the connection tab of the Internet option” in response to the inquiry item “Tell me how to set the browser” from the customer. Among these, the example of FIG. 21 shows a dialogue when the Internet option branches to a dialogue. Further, in the example of FIG. 22, the dialogue is shown when branching to a dialogue centered on the connection tab. Here, the procedure explanation words of the procedure X are “Internet option” and “Connection tab”. Note that the horizontal axis in FIGS. 21 and 22 indicates the time axis, and indicates that time advances as it advances in the right direction. Also, “u” in the figure is an abbreviation for “utterance”, and the blocks from “u1” to “u11” indicate the speech of the operator or the customer, and the horizontal size of these blocks is the length of the speech section. It shows. Furthermore, the block illustrated above the horizontal axis indicates the customer's speech, and the block illustrated below the horizontal axis indicates the operator's speech.

  In the example shown in FIG. 21, it is assumed that the following conversation is performed. The operator speaks, "Please open the Internet Options connection tab" (u1). The customer speaks “What?” (U2). The operator speaks “second from the right of the menu bar” (u3). The customer speaks “Menu bar?” (U4). The operator utters "It is lined up with files, edits, etc. at the top ..." (u5). The customer speaks “Ah yes yes” (u6). The operator speaks, “Click on the second tool from the right, click here” (u7). The customer speaks “second ... yes” (u8). The operator speaks, "There is an internet option at the bottom" (u9). The customer speaks “Oh, this is it” (u10). The operator speaks “Please select the connection tab there” (u11).

  In the example shown in FIG. 21, the operator support apparatus performs voice recognition that extracts “procedure explanation words of QA examples registered in advance” from voice data including dialogues from u1 to u11. In the example of FIG. 21, the operator support apparatus extracts “Internet option” and “connection tab” from the utterance of u1 as procedure explanation words. Further, the operator support apparatus extracts “Internet option” from the utterance of u9 as a procedure explanation word and extracts “connection tab” from the utterance of u11 as a procedure explanation word. Here, it is assumed that “menu bar” in the utterance of u3 and “second” in the utterance of u7 are extracted as supplemental phrases as utterances performed only by a skilled operator. According to the conventional technique, “menu bar” and “second” appear in the vicinity of “Internet option” and “connection tab” which are procedure explanation words of procedure X, and therefore are extracted as supplementary words of procedure X. . Actually, in the utterances from u2 to u9, a dialog regarding the procedure explanation phrase “Internet option” is performed, and it is not a supplementary phrase of the procedure explanation phrase “connection tab”. However, in the prior art, when there are a plurality of procedure explanation words for one procedure, it is not possible to specify which procedure explanation words are supplementary words.

  In the example shown in FIG. 22, it is assumed that the following conversation has been performed. The operator speaks, "Please open the Internet Options connection tab" (u1). The customer speaks “What is the connection tab?” (U2). The operator speaks “Is the Internet option open?” (U3). The customer speaks "I'm open" (u4). The operator utters, “It is lined up with General, Security, etc. at the top ...” (u5). The customer speaks “Oh, is this” (u6). The operator speaks “Yes, please open the connection tab there” (u7). The customer speaks “Yes” (u8).

  In the example shown in FIG. 22, the operator support apparatus extracts “Internet option” and “connection tab” from the utterance of u1 as procedure explanation words. Further, the operator support apparatus extracts “Internet option” from the u3 utterance as a procedure explanation word / phrase. Further, the operator support apparatus extracts “connection tab” from the u7 utterance as a procedure explanation word. Here, it is assumed that “first” in the u5 utterance is extracted as a supplemental phrase as an utterance performed only by a skilled operator. According to the prior art, “top” appears in the vicinity of “Internet option” and “connection tab”, which are procedure explanation words of procedure X, and is therefore extracted as a supplementary word of procedure X. Actually, it is a phrase to explain the display position of the connection tab on the screen, so it is not a supplemental phrase for "Internet Options", but in the conventional technology, it is identified as a supplemental phrase for only "Connection tab" I can't.

  As described above, in the above-described conventional technique, when a plurality of procedure explanation words / phrases exist in one procedure, they can be associated only as supplement words / phrases of the plurality of procedure explanation words / phrases. It is not possible to specify which procedure explanation word is associated with.

  The disclosed technology has been made in view of the above, and an object thereof is to provide an operator support device, an operator support program, and an operator support method that can extract supplemental words corresponding to procedure explanation words.

  In one aspect, the technology disclosed in the present application is a procedure explanation word that is a word for explaining a predetermined procedure, and an operation part that is operated in relation to the procedure explanation word among the operation parts displayed on the terminal. Are stored in association with each other. Furthermore, the technology disclosed in the present application includes an operation time acquisition unit that acquires an operation time indicating a time point when an operation of switching a predetermined operation part among operation parts displayed on the terminal is performed. The operation time acquisition unit acquires an operation time from a PC operation log storage unit that stores an operation part that is an operation target from that time in association with each switching operation time when the operation part is switched to another operation part. To do. Furthermore, the technology disclosed in the present application includes a section estimation unit. The section estimation unit extracts a procedure explanation word related to an operation part switched from the predetermined operation part from the correspondence storage unit. The section estimation unit extracts a time point when the utterance is started after the operation time acquired by the operation time acquisition unit from the voice recording. The section estimation unit displays a speech section that appears next to an utterance section at the time of extraction from an utterance section in which a procedure explanation word appears, up to a start time of an utterance section longer than a predetermined period, and the correspondence relationship It is estimated as a speech section related to the procedure explanation phrase extracted from the storage unit. Furthermore, the technology disclosed in the present application includes a supplemental phrase extracting unit that extracts a supplemental phrase that is a phrase for supplementing the procedure explanation phrase from the speech segment estimated by the segment estimation unit in the speech recording.

  According to one aspect of the technology disclosed in the present application, it is possible to extract a supplemental phrase corresponding to the procedure explanation phrase.

FIG. 1 is a block diagram illustrating the configuration of the operator support apparatus according to the first embodiment. FIG. 2 is a block diagram illustrating the configuration of the operator support apparatus according to the second embodiment. FIG. 3 is a diagram illustrating a configuration example of information stored in the correspondence relationship storage unit. FIG. 4 is a diagram illustrating an example of a response history list. FIG. 5 is a diagram showing an example of the prosodic data list related to the operator. FIG. 6 is a diagram illustrating an example of a prosodic data list related to a customer. FIG. 7 is a diagram illustrating an example of utterance section information related to an operator. FIG. 8 is a diagram illustrating an example of utterance section information related to a customer. FIG. 9 is a diagram illustrating an example of the recognized keyword list. FIG. 10 is a diagram illustrating an example of the operation time list. FIG. 11 is a diagram illustrating an example of a procedure-corresponding section estimation result. FIG. 12 is a diagram illustrating an example of the estimation result of the corresponding section of the procedure explanation word / phrase. FIG. 13 is a diagram illustrating an example of a supplemental phrase list. FIG. 14 is a flowchart illustrating a flow of processing in the operator support apparatus according to the second embodiment. FIG. 15 is a flowchart illustrating a procedure of a procedure corresponding section estimation process according to the second embodiment. FIG. 16 is a flowchart illustrating a procedure of a process for estimating a corresponding section of a procedure explanatory word according to the second embodiment. FIG. 17 is a flowchart illustrating a procedure of supplemental phrase extraction processing according to the second embodiment. FIG. 18 is an explanatory diagram for explaining the supplemental word extraction method according to the second embodiment. FIG. 19 is an explanatory diagram for explaining the supplemental phrase extraction procedure according to the second embodiment. FIG. 20 is a diagram illustrating a computer that executes an operator support program. FIG. 21 is an explanatory diagram for explaining a procedure for extracting supplemental phrases in the prior art. FIG. 22 is an explanatory diagram for explaining a supplementary word extraction procedure in the prior art.

  Hereinafter, an embodiment of an operator support device, an operator support program, and an operator support method disclosed in the present application will be described in detail with reference to the drawings. Note that the technology disclosed in the present application is not limited by the examples described below as an embodiment of the operator support device, the operator support program, and the operator support method disclosed in the present application.

  FIG. 1 is a block diagram illustrating the configuration of the operator support apparatus according to the first embodiment. As illustrated in FIG. 1, the operator support apparatus 1 according to the first embodiment includes a storage unit 2, an operation time acquisition unit 3, a section estimation unit 4, and a supplemental phrase extraction unit 5.

  The correspondence storage unit 2 associates and stores a procedure explanation phrase that is a phrase for explaining a predetermined procedure and an operation part that is operated in relation to the procedure explanation phrase among the operation parts displayed on the terminal. To do. The operation time acquisition unit 3 acquires an operation time indicating a point in time when an operation for switching a predetermined operation part among the operation parts displayed on the terminal is performed. The operation time acquisition unit 3 stores the operation time from the PC operation log storage unit that stores the operation part that is the operation target from that time in association with each other when the operation part is switched to another operation part. To get.

  The section estimation unit 4 extracts the procedure explanation words related to the operation part switched from the predetermined operation part from the correspondence storage part 2, and from the operation time acquired by the operation time acquisition part 3 from the voice recording. Thereafter, the time point when the utterance is started is extracted. Then, the section estimation unit 4 corresponds to the speech section that appears next to the utterance section at the time of extraction from the utterance section in which the procedure explanation word appears, and reaches the start time of the utterance section longer than the predetermined period. Estimated as a speech section related to the procedure explanation phrase extracted from the storage unit 2. The supplemental phrase extraction unit 5 extracts a supplemental phrase that is a phrase for supplementing the procedure explanation phrase from the speech segment estimated by the segment estimation unit 4 in the speech recording.

  As described above, the estimation of the voice section from the operation time to the point when the utterance is started as the voice section related to the procedure explanation phrase is performed between the terminal operation performed by the operator and the dialogue between the operator and the customer. This is because they are related.

  That is, when explaining the procedure of the answer items, the operator performs an explanation while operating the operation part up to the operation part to be reached in one procedure, that is, the operation part corresponding to the procedure explanation word / phrase. Then, at the stage where the operation part to be reached is reached, the operator performs a confirmation utterance regarding the operation part (procedure explanation phrase) in order to confirm whether or not the customer has been guided to the operation part to be the arrival target. Is the answer. When such a confirmation utterance is made, it is clear that the operator had previously explained the operation part that appeared in the confirmation utterance, and the procedure explanation words that appeared in the confirmation utterance before that It can be regarded as a dialogue.

  Thus, in the operator assistance apparatus 1 which concerns on a present Example, since the area estimation based on this knowledge is performed, the audio | voice area corresponding to a procedure explanatory note can be estimated. For this reason, according to the operator assistance apparatus 1 which concerns on a present Example, it becomes possible to extract the supplemental phrase corresponding to a procedure description word / phrase.

[Configuration of operator support device]
FIG. 2 is a block diagram illustrating the configuration of the operator support apparatus according to the second embodiment. In the present embodiment, an operator support apparatus that is assumed to receive an inquiry regarding the operation of a personal computer at a call center will be described. Hereinafter, the personal computer is abbreviated as PC.

  As shown in FIG. 2, the operator support apparatus 100 includes a PC operation log storage unit 110, a case data storage unit 120, a correspondence relationship storage unit 125, a voice data storage unit 130, a correspondence relationship setting unit 140, a prosody. An information extraction unit 150 and an utterance section extraction unit 155 are included. Further, the operator support apparatus 100 includes a voice recognition unit 160, an operation time acquisition unit 170, a section estimation unit 180, and a supplemental phrase extraction unit 190.

  The PC operation log storage unit 110 stores a PC operation log in the PC used by the operator. The PC operation log storage unit 110 stores, in association with each switching operation time when the operation site is switched to another operation site, the operation site that has been an operation target from that time. The “operation part” mentioned here includes not only a screen generated by an OS (Operating System) or an application, but also a tab or a pull-down menu displayed in response to an operation such as a mouse operation or a keyboard operation on the screen. Including. The PC operation log can be acquired by using existing operation log acquisition software or an OS function.

  The case data storage unit 120 stores case data that the operator refers to when responding. The case data storage unit 120 stores an inquiry item and an answer item of a QA (Question and Answer) case in association with each case ID (Identification). For example, the case ID “QA123” is associated with the inquiry item “Tell me how to set the browser” and each procedure included in the answer item. This answer includes procedure 1 “open connection tab of Internet option”, procedure 2 “open LAN (Local Area Network) setting”, and procedure 3 “check proxy server setting”. Note that each procedure of answer items is given a case procedure ID having a branch number as a number corresponding to each procedure number.

  The correspondence relationship storage unit 125 stores the correspondence relationship between a procedure explanation word which is a key (important) when explaining one procedure included in the answer items and an operation part operated in relation to the procedure explanation word. To do. The correspondence relationship storage unit 125 stores a procedure explanation word, a case procedure ID, and an operation part in association with each other. Note that the correspondence between the procedure explanation word and the operation part is set by the correspondence setting unit 140 described later.

  FIG. 3 is a diagram illustrating a configuration example of information stored in the correspondence relationship storage unit. As illustrated in FIG. 3, the correspondence relationship storage unit 125 stores the procedure instruction word “Internet option” in association with the case procedure ID “QA 123-1” and the operation part “Internet option window—general tab screen”. In addition, the correspondence relationship storage unit 125 stores the procedure instruction word “connection tab” in association with the case procedure ID “QA 123-1” and the operation part “Internet option window—connection tab screen”. Further, the correspondence relationship storage unit 125 stores the procedure instruction word “LAN” in association with the case procedure ID “QA123-2” and the operation part “local area network (LAN) setting window”. The example shown in FIG. 3 shows a situation where “Internet option” and “connection tab” are the same case and are procedure explanation words included in the same procedure.

  The voice data storage unit 130 stores voice data recorded in the past reception work. For example, the voice data storage unit 130 records one call as voice data from when a customer's telephone call is connected to one customer's reception until the connection is disconnected. The voice data storage unit 130 stores voice data in association with a call ID for uniquely identifying a call so that arbitrary voice data can be called from a function unit described later. Further, the voice data storage unit 130 stores the time at which recording is started in association with the call ID so that the time can be converted by a function unit described later. The audio data is, for example, stereo data in which operator audio data and customer audio data are input from different systems, but may be monaural data input from a single system. The voice data recording period is not limited from call connection to disconnection, the start time may be before or after the call connection, and the end time may be before or after the call connection. Absent.

  The voice data storage unit 130 stores a response history list. The response history list is a list in which case data referred to by the operator in the service related to the call is associated with each call ID. FIG. 4 is a diagram illustrating an example of a response history list. In the example illustrated in FIG. 4, the response with the call ID “call 001” indicates that an inquiry regarding the case ID “QA123” has been received. The reception of the call ID “call 002” indicates that an inquiry regarding the case ID “QA456” has been received. Further, the reception of the call ID “call 003” indicates that an inquiry regarding the case ID “QA135” has been received. Note that these call ID and case ID are input in association with each other at the terminal used by the operator after the service is completed. In addition, here, the case where there is one case in which an inquiry is received by one call is illustrated, but an inquiry regarding a plurality of cases can also be received by one call.

  Correspondence setting section 140 sets a correspondence relation between procedure explanation words included in each procedure of the answer items in the QA cases stored in case data storage section 120 and operation parts related to the overall operation of the PC. As an example of such setting, the correspondence setting unit 140 reads out case data included in the procedure in ascending or descending order of the case procedure IDs stored in the case data storage unit 120. At this time, the correspondence setting unit 140 analyzes the document of each procedure included in the answer items in the case data by using a morphological analysis technique or the like, thereby applying a predetermined condition, for example, shading in the document. Keywords that satisfy the selected word are extracted as procedure explanation words. Then, the correspondence setting unit 140 causes the call center administrator or a skilled operator to select an operation part to be associated with the previously extracted procedure explanation word / phrase via an input device (not shown). After that, the correspondence setting unit 140 associates the case procedure ID corresponding to the procedure including the procedure explanation word together with the operation part and the procedure explanation word selected as the correspondence destination in this manner, and the correspondence relation storage unit 125. Register settings to. Here, a case has been described in which the correspondence relationship is set by an input operation by a call center person, but the correspondence relationship may be acquired via a network. In addition, here, the procedure explanation words of each procedure included in the answer items are extracted using the morphological analysis technology. However, any specification such as a specification by a call center person or an utterance frequency exceeding the threshold in the same case is used. Can be extracted by criteria.

  The prosody information extraction unit 150 generates a prosody data list from the speech data stored in the speech data storage unit 130. For example, the prosody information extraction unit 150 reads voice data from the voice data storage unit 130 when there is a call ID from which prosody information has not been extracted. Then, the prosodic information extraction unit 140 extracts the operator's prosodic data by sampling the power value at predetermined time intervals from the operator's audio data among the audio data that is stereo data. Then, the prosodic information extraction unit 150 generates an operator prosodic data list from the extracted operator prosodic data. The prosody information extraction unit 150 also performs processing for sampling the power value on the customer's voice data as in the case of the operator to generate a customer's prosody data list. The power value is calculated from the amplitude of the signal waveform acquired from the audio data.

  FIG. 5 is a diagram showing an example of the prosodic data list related to the operator. As shown in FIG. 5, the operator's prosodic data list is information in which power values are associated with each fixed time. In FIG. 5, according to the prosodic data list of the operator illustrated in FIG. 5 exemplifying the case where sampling is performed every 12.8 [msec], it is indicated that the power value at time “0” is −60.008152. This indicates that the power value at time “12.8” is −51.801006. It indicates that the power value at time “25.6” is −49.1711921. It indicates that the power value at time “38.4” is −50.005344. It shows that the power value at time “51.2” is −52.422977. It shows that the power value at time “64.0” is −57.180473. This indicates that the power value at time “76.8” is −62.3303638. It indicates that the power value at time “89.6” is −57.468826. This indicates that the power value at time “102.4” is −53.068714. In addition, the power value at time “115.2” is −51.655396. The time mentioned here refers to the elapsed time when the start time of recording of audio data is zero, but other times such as a time stamp of standard time can also be used.

  FIG. 6 is a diagram illustrating an example of a prosodic data list related to a customer. As shown in FIG. 6, the customer's prosodic data list is information in which power values are associated with each other at regular intervals. FIG. 6 shows that the power value at time “0” is −57.180473 according to the customer prosody data list shown in FIG. 6 exemplifying the case where sampling is performed every 12.8 [msec]. This indicates that the power value at time “12.8” is −62.3303638. The customer's prosodic data list indicates that the power value at time “25.6” is −57.468826. This indicates that the power value at time “38.4” is −53.068714. It shows that the power value at time “51.2” is −51.655396. It indicates that the power value at time “64.0” is −60.080152. This indicates that the power value at time “76.8” is −51.801006. The power value at time “89.6” is −49.1711921. This indicates that the power value at time “102.4” is −50.005344. In addition, the power value at the time “115.2” is −52.422977.

  The utterance section extraction unit 155 extracts utterance section information from the prosodic data extracted by the prosody information extraction unit 150. For example, the utterance section extraction unit 155 detects a section in which a power value equal to or greater than a predetermined threshold value continues from the operator's prosodic data list, and determines the start time and end time of the section in which the period of the detected section continues for a predetermined period or more. Extracted as the operator's utterance section. Then, the utterance section extraction unit 155 assigns an utterance section ID for identifying the utterance section to the utterance section of the operator extracted in this way, and the operator who associates the utterance section with the numbered utterance section ID. The utterance section information is generated.

  FIG. 7 is a diagram illustrating an example of utterance section information related to an operator. As shown in FIG. 7, the utterance section information of the operator is information in which the start time and the end time are associated with each utterance section ID. As illustrated in FIG. 7, the example of the utterance section ID “call001-1” indicates that the operator uttered from the start time “10 msec” to the end time “14320 msec”. The example of the utterance section ID “call001-3” indicates that the operator uttered from the start time “... Msec” to the end time “. Note that these start time and end time indicate elapsed time when the start time of recording of audio data is zero, but other times such as a time stamp of standard time can also be used.

  In addition, the utterance section extraction unit 155 extracts customer utterance section information for the customer's prosodic data list in the same manner as the operator's prosodic data list. FIG. 8 is a diagram illustrating an example of utterance section information related to a customer. As shown in FIG. 8, the example of the utterance section ID “call001-2” indicates that the customer uttered from the start time “14350 msec” to the end time “17800 msec”. The example of the utterance section ID “call001-4” indicates that the customer uttered from the start time “... Msec” to the end time “.

  The voice recognition unit 160 performs voice recognition on the voice data stored in the voice data storage unit 130 to create a recognition keyword list. For example, the voice recognition unit 160 reads the voice data corresponding to the call ID to be processed among the voice data stored in the voice data storage unit 130 and reads the case ID corresponding to the call ID in the answering history list. Further, the voice recognition unit 160 reads out the procedure explanation words / phrases corresponding to the case ID read out from the correspondence relationship storage unit 125. In this manner, the voice recognition unit 160 reads out the procedure explanation words and the voice data including the utterances included in the answer items of the QA cases actually handled by the operator. After that, the speech recognition unit 160 uses the procedure explanation word / phrase as a recognition keyword, and the time when speech of the procedure explanation word / phrase is started by performing speech recognition based on the recognition keyword on the voice data (hereinafter referred to as an appearance time). Is identified. When the appearance time of the procedure explanation word / phrase is specified in this way, the voice recognition unit 160 generates a recognition keyword list by adding a record including the call ID, the recognition keyword (procedure explanation word / phrase), the case procedure ID and the appearance time. To do. Note that the voice recognition unit 160 has a call ID in which voice data for which no supplemental phrase has been extracted exists in the voice data storage unit 130 and a response history list corresponding to the voice data is registered in the voice data storage unit 130. Can be the target of speech recognition processing.

  FIG. 9 is a diagram illustrating an example of the recognized keyword list. As shown in FIG. 9, when “10550 msec” has elapsed from the recording start time of the voice data “call 001”, the utterance of the procedure explanation phrase “Internet option” included in the procedure 1 of the QA case 123 is started. Indicates. This indicates that the utterance of the “connection tab” included in the procedure 1 of the QA example 123 is started when “12870 msec” has elapsed from the recording start time of the voice data “call 001”. This indicates that the utterance of “Internet” included in the procedure 1 of the QA case 123 is started when “99180 msec” has elapsed from the recording start time of the voice data “call 001”. In addition, it indicates that the utterance of “LAN” included in the procedure 2 of the QA case 123 is started when “110300 msec” has elapsed from the recording start time of the voice data “call 001”.

  The operation time acquisition unit 170 is an operation indicating a point in time when an operation for switching a predetermined operation part among operation parts displayed on the PC used by the operator is performed from the PC operation log stored in the PC operation log storage unit 110. Get time.

  Here, the switching operation time stored as the PC operation log is the time when the operation part switching operation is performed. This switching operation time is compared with the recording time of the audio data, but since it is a log collected using operation log acquisition software in a separate system from the audio data, it should be regarded as the elapsed time of the audio data. I can't. In this state, the operation time switching operation time and the elapsed time of the voice data cannot be compared by the section estimation unit 180 described later, so the operation time acquisition unit 170 converts the time into the elapsed time so that the two can be compared. Or convert elapsed time to time.

  In order to establish such conversion, it is preferable that either the PC operation log storage unit 110 or the audio data storage unit 130 has a correspondence relationship between the elapsed time of the audio data and the switching operation time of the operation part. As an example, recording software for recording audio data registers the time at the start of recording audio data in the audio data storage unit 130 in association with the call ID of the audio data when recording audio data. Here, the time at which the recording is started is registered in the voice data storage unit 130 together with the voice data, but the time difference from the recording start time of the call to the switching operation time is associated with the call ID by the PC operation. You may make it register with this log acquisition software.

  Using the time of the recording start time registered in advance as described above, the operation time acquisition unit 170 uses the recording period of the audio data stored in the audio data storage unit 130 as a cutout range, and from the PC operation log storage unit 110. Cut out the PC operation log. That is, the operation time acquisition unit 170 cuts out a PC operation log in which the switching operation time is included in the section from the recording start time to the recording end time of the audio data. And the operation time acquisition part 170 calculates | requires the difference between each switching operation time and the time of a recording start time, The elapsed time (operation time) from the recording start time of the switching operation for every switching operation time Is calculated. In addition, the operation time acquisition unit 170 associates the call ID of the audio data used for cutting out the PC operation log, the operation part after switching, and the operation time indicating the time when the switching operation is performed, to thereby match the operation time. Generate a list. Note that the operation time acquisition unit 170 can use any call ID as a target for acquiring the operation time as long as it is a call ID of voice data from which a supplemental phrase has not been extracted.

  FIG. 10 is a diagram illustrating an example of the operation time list. The operation time list is information in which an operation part and an operation time are associated with each call ID. The example illustrated in FIG. 10 indicates that the operation part has been switched to “browser window” when the elapsed time from the recording start time of the voice data of the call ID “call 001” reaches “17560 msec”. In the example shown in FIG. 10, when the elapsed time from the recording start time of the voice data of the call ID “call 001” reaches “100230 msec”, the operation part is switched to the “Internet option window-general tab screen”. Indicates.

  The section estimation unit 180 estimates a voice section related to the procedure explanation word / phrase. The section estimation unit 180 targets the call ID of the voice data from which the supplemental phrase has not been extracted, and follows the utterance of the operator included in the voice data in time series, thereby obtaining the voice related to the procedure explanation phrase included in the voice data. Estimate the start time and end time of the section.

  The section estimation unit 180 estimates a procedure-corresponding section in which one procedure included in an answer item is described using a recognition keyword list by the speech recognition unit 160 when estimating a speech section related to a procedure explanation word / phrase. In addition, the section estimation unit 180 estimates the corresponding section of the procedure explanation phrase using the operation time acquired by the operation time acquisition unit 170. Here, (1) estimation of the procedure-corresponding section is described, and (2) estimation of the corresponding section of the procedure explanation phrase is described.

(1) Estimation of Procedure Corresponding Section Here, the estimation of the procedure corresponding section by the section estimation unit 180 will be described. The section estimation unit 180 uses the recognition keyword list output by the speech recognition unit 160 and the utterance section information of the operator output by the utterance section extraction unit 155 to relate each procedure included in the answer items of the QA case. Estimate the start time of the description. That is, the section estimation unit 180 estimates the time point when the procedure explanation word appears as the time point when the explanation of the procedure including the procedure explanation word is started.

  To explain this step by step, the section estimation unit 180 obtains the case procedure ID and the appearance time (see FIG. 9) from the recognition keyword list, and the operator's utterance section information including the appearance time (see FIG. 7). To get.

  At this time, when the section estimation unit 180 first acquires the case procedure ID and the appearance time from the recognition keyword list, the section estimation unit 180 sets the start time of the utterance section including the appearance time to the start of the voice section corresponding to the case procedure ID. Estimated as time. For example, in the example illustrated in FIGS. 7 and 9, the section estimation unit 180 first acquires the case procedure ID “QA 123-1” and the appearance time “10550 msec” from the recognition keyword list illustrated in FIG. 9. Therefore, the section estimation unit 180 uses the speech start time “10 msec” (see FIG. 7) including the appearance time “10550 msec” in the speech section as the voice section start time corresponding to the case procedure ID “QA123-1.” Estimate as is.

  Further, if the case procedure ID and the appearance time acquired from the recognition keyword list are not the first, the section estimation unit 180 determines whether or not the case procedure ID acquired this time is the same as the case procedure ID acquired last time. judge. At this time, if the case procedure ID acquired this time is the same as the case procedure ID acquired last time, the section estimation unit 180 considers that the description of the same case procedure ID is continuing, and the recognition keyword list To obtain the next case procedure ID and appearance time. For example, in the example shown in FIG. 7 and FIG. 9, the section estimation unit 180 does not acquire the case examples until the first to third case procedure IDs and appearance times from the top in the recognition keyword list shown in FIG. It is assumed that the description of the procedure ID “QA 123-1” is continuing.

  On the other hand, when the case procedure ID acquired this time is different from the case procedure ID acquired last time, the section estimation unit 180 obtains the start time of the utterance section including the appearance time acquired this time as the case procedure acquired this time. Estimated as the start time of the speech segment corresponding to the ID. For example, when the case procedure ID “QA123-2” and the appearance time “110300 msec” are acquired from the recognition keyword list illustrated in FIG. 9, the section estimation unit 180 considers that the utterance has moved to the next procedure. That is, the section estimation unit 180 considers that the case procedure ID “QA123-2” has been started and the case procedure ID “QA123-2” has been started. In this case, since it is regarded as the start time of the voice section corresponding to the case procedure ID “QA123-2”, this start time is temporarily estimated as the end time of the voice section corresponding to the case procedure ID “QA123-1”. Will be.

  As described above, the section estimation unit 180 obtains the estimation result of the procedure-corresponding section by performing the above-described processing on all the recognition keyword list records obtained as the voice recognition result of the voice data of one call ID. That is, the section estimation unit 180 performs a process of partitioning each procedure in which one procedure included in the answer item is described by separating the voice data of one call ID by the appearance time of the procedure explanation word / phrase. It should be noted that here, the procedure explanation word / phrase is only separated by the appearance time, and the actual end time of the speech section is not necessarily the start time of the next procedure explanation.

  FIG. 11 is a diagram illustrating an example of a procedure-corresponding section estimation result. For example, as illustrated in FIG. 11, the estimation result of the procedure corresponding section is information in which the case procedure ID, the start time [msec] and the end time [msec] of the procedure corresponding section are associated with each call ID. In the example shown in FIG. 11, it is shown that the conversation related to the case procedure ID “QA 123-1” was performed from the start time “10” to the end time “131050”. In addition, it shows that the conversation related to the case procedure ID “QA123-2” was performed from the start time “131050” to the end time “...”. Here, only the procedure-corresponding section related to the call “call 001” is illustrated, but the same processing is performed for all call IDs of voice data from which supplemental words / phrases have not been extracted.

(2) Estimation of Corresponding Section of Procedure Explanation Words Next, estimation of a corresponding section of procedure explanation words by the section estimation unit 180 will be described. The section estimation unit 180 uses the operation time list output by the operation time acquisition unit 170 to estimate the corresponding section of the procedure explanation phrase. The section estimation unit 180 uses one procedure included in the answer item of the QA example as a processing unit, and repeats the process until the corresponding sections of the procedure explanation words are estimated for all the procedure corresponding sections estimated previously. .

  If this point is explained in order, the section estimation unit 180 acquires the start time and end time of the procedure-corresponding section estimated previously. Thereafter, the section estimation unit 180 selects the operation time and the operation part after switching in order from the operation time with the shortest operation time (elapsed time from the recording start time) in the operation time list output by the operation time acquisition unit 170. get. Then, the section estimation unit 180 extracts a procedure explanation word / phrase having a correspondence relationship with the operation part after switching from the correspondence relationship storage unit 125.

  Further, the section estimation unit 180 refers to the utterance section information of the operator, acquires the utterance (N) of the operator corresponding to the operation time when the switching operation is performed, and utters the operator next to the acquired utterance of the operator. (N + 1) is acquired. Here, when there is an operator utterance including the operation time, the section estimation unit 180 acquires the utterance (N) of the operator, and the utterance (N + 1) of the operator next to the acquired operator utterance. get. On the other hand, when there is no operator utterance including the operation time, the section estimation unit 180 acquires the operator utterance (N) immediately after that and the operator utterance (N + 1) next to the acquired operator utterance. To get.

  At this time, when the next operator's utterance section (N + 1 utterance section) is longer than the predetermined period, the section estimation unit 180 uses the start time of the next operator's utterance (N + 1) as the voice section related to the procedure explanation phrase. Is estimated to be the end time of If the next operator's utterance section (N + 1 utterance section) is shorter than the predetermined period, the next operator's utterance (N + 2) is acquired, and the next operator's utterance (N + 2) is acquired from the predetermined period. It is further determined whether or not it is too short. In this manner, the section estimation unit 180 searches for an operator's utterance until an operator's utterance longer than a predetermined period is found. Note that the predetermined period mentioned here can be arbitrarily set by the call center personnel as a period during which the operator is expected to perform a response utterance such as “Yes” or “Yes” in response to the customer's utterance.

  As described above, the section estimation unit 180 performs the above-described processing on all the operation time list records output by the operation time acquisition unit 170, thereby obtaining the estimation result of the corresponding section of the procedure explanation phrase. That is, the section estimation unit 180 uses the operation time when the operation part is switched to divide the previously estimated procedure corresponding section into the corresponding sections of the procedure explanatory words.

  FIG. 12 is a diagram illustrating an example of the estimation result of the corresponding section of the procedure explanation word / phrase. For example, as shown in FIG. 12, the estimation result of the corresponding section of the procedure explanation phrase is information in which the case procedure ID, the procedure explanation phrase, and the end time are associated with each call ID. The example shown in FIG. 12 indicates that the conversation of the procedure explanation phrase “Internet option” included in the case procedure ID “QA 123-1” has ended at the end time “110280 (msec)”. Further, it indicates that the dialogue of the procedure explanation word / phrase included in the case procedure ID “QA 123-1” is ended at the end time “... (Msec)”.

  Returning to the explanation of FIG. 2, the supplemental phrase extraction unit 190 uses the estimation result of the speech section of the procedure explanation phrase estimated by the section estimation unit 180 and the utterance section information of the operator extracted by the utterance section extraction unit 155. Thus, a supplemental phrase for supplementing the procedure explanation phrase is extracted. The supplemental phrase extraction unit 190 extracts the supplemental phrase from the call ID of the speech data in which the supplementary phrase has not been extracted and the speech data of the procedure explanation phrase estimated by the section estimation unit 180 exists. Can be targeted.

  Specifically, the supplemental phrase extraction unit 190 includes the operator's utterance included in the speech section of the procedure explanation phrase estimated by the section estimation unit 180 out of the utterance section information of the operator extracted by the utterance section extraction unit 155. Get the utterance interval. For example, in the example shown in FIGS. 11 and 12, the supplemental phrase extracting unit 190 performs each of the operators included in the speech sections “10 (msec)” to “110280 (msec)” related to the procedure explanation phrase “Internet option”. Get the utterance section of the utterance.

  Then, the supplemental phrase extraction unit 190 sets the speech data of the utterance section of the operator's utterance acquired previously as the supplemental phrase extraction range, performs speech recognition on the speech data in the supplemental phrase extraction range, and extracts the supplemental phrase. Extract. For example, as a phoneme recognition technique for extracting a phrase that can be a supplemental phrase from speech data, a technique for searching and extracting an unknown word from prosodic data and other known techniques can be arbitrarily selected and used. In addition, as a technique for extracting a phrase to be a supplemental phrase from a phrase that can be a supplemental phrase, a technique disclosed in Japanese Patent Application Laid-Open No. 2008-33835 and other known techniques can be arbitrarily selected and used.

  Thereafter, the supplemental phrase extraction unit 190 associates the supplemental phrase extracted in this way with the procedure explanation phrase and the example procedure ID of the speech section used to determine the extraction range of the supplemental phrase, thereby obtaining the supplemental phrase list. Generate.

  FIG. 13 is a diagram illustrating an example of a supplemental phrase list. For example, as illustrated in FIG. 13, the supplemental phrase list is information in which a procedure explanation phrase and a supplemental phrase are associated with each case procedure ID. In the example illustrated in FIG. 13, the supplemental phrase “menu bar” is associated with the procedure explanation phrase “Internet option” of the case procedure ID “QA 123-1”. Further, it indicates that the supplementary phrase “second” is associated as the procedure explanation phrase “Internet option” of the procedure ID “QA 123-1”. Further, it is shown that the supplemental phrase “first” is associated with the procedure explanation phrase “connection tab” of the procedure ID “QA 123-1”. Further, it indicates that the supplementary phrase “bottom 1” is associated with the procedure explanation phrase “LAN” of the procedure ID “QA123-2”.

  The operator support apparatus 100 shown in FIG. 2 includes each of the prosodic information extraction unit 150, the speech segment extraction unit 155, the speech recognition unit 160, the operation time acquisition unit 170, the segment estimation unit 180, and the supplemental phrase extraction unit 190. It can also be realized by installing the function in the information processing apparatus. Examples of the information processing apparatus include a known personal computer, workstation, ASP (Application Service Provider) server apparatus, mobile phone, PHS terminal, mobile communication terminal, or PDA.

  The PC operation log storage unit 110, the case data storage unit 120, the correspondence relationship storage unit 125, and the voice data storage unit 130 are storage devices as described below. For example, the storage device is a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), or a flash memory, a hard disk, or an optical disk. The correspondence setting unit 140, prosodic information extraction unit 150, speech segment extraction unit 155, speech recognition unit 160, operation time acquisition unit 170, segment estimation unit 180, and supplemental phrase extraction unit 190 are realized by an integrated circuit or an electronic circuit. Is done. As the integrated circuit, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) can be applied. Further, as an electronic circuit, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like can be applied.

[Processing by operator support device]
Next, a processing flow of the operator support apparatus according to the present embodiment will be described. Here, (1) the process corresponding to the procedure explanation word / phrase is explained by (2) the procedure corresponding section estimation process performed by the section estimation unit 180 while explaining the flow of processing in the operator support apparatus 100. The estimation process will be described. Thereafter, (1) the supplemental phrase extraction process performed by the supplemental phrase extraction unit 190 will be described while explaining the flow of processing in the operator support apparatus 100.

(1) Process Flow in Operator Support Device The process flow in the operator support device according to the present embodiment will be described with reference to FIG. FIG. 14 is a flowchart illustrating a flow of processing in the operator support apparatus according to the second embodiment. Note that the operator support apparatus 100 activates the process when there is a call ID and case ID from which no supplemental phrase has been extracted in the response history list stored in the voice data storage unit 130.

  As shown in FIG. 14, the operator support apparatus 100 targets the voice data of the call ID from which the supplemental phrase has not been extracted, the processing of the voice data system of steps S101A to S104A, and the operation history system of steps S101B to S102B. Processing can be performed in parallel.

  Explaining the processing of the speech data system, the prosody information extraction unit 150 generates an operator's prosody data list (see FIG. 5) from the speech data stored in the speech data storage unit 130 (step S101A). Then, the utterance section extraction unit 155 generates utterance section information (see FIG. 7) from the operator's prosody data list extracted by the prosody information extraction unit 150 (step S102A). Thereafter, the voice recognition unit 160 performs voice recognition on the voice data stored in the voice data storage unit 130 to generate a recognition keyword list (see FIG. 9) including the appearance times of the procedure explanation words (step S103A). Here, the case where speech recognition is performed after extracting prosodic information and utterance intervals has been described, but speech recognition can also be performed in parallel with the extraction of prosodic information and the extraction of utterance intervals. .

  After the processing of the speech data system in steps S101A to S103A is thus completed, the section estimation unit 180 performs processing for estimating the start time of the speech section using the speech section information and the recognition keyword list (step S104A). ).

(2) Procedure Corresponding Section Estimation Process Here, the procedure corresponding section estimation process shown in step S104A of FIG. 14 will be described with reference to FIG. FIG. 15 is a flowchart illustrating a procedure of a procedure corresponding section estimation process according to the second embodiment.

  As illustrated in FIG. 15, the section estimation unit 180 sequentially acquires the case procedure ID and the appearance time from the recognition keyword list (Step S201). Then, the section estimation unit 180 acquires the utterance section information of the operator's utterance including the appearance time acquired from the recognition keyword list (step S202).

  At this time, the section estimation unit 180 determines whether or not the case procedure ID acquired this time is the same as the case procedure ID acquired last time (step S203). If the current and previous case procedure IDs are the same (No in step S203), the section estimation unit 180 sets the start time of the utterance section including the appearance time acquired this time as the case procedure ID acquired this time. Estimated as the start time of the corresponding procedure-corresponding section (step S204). Even when the case procedure ID and the appearance time are first acquired from the recognition keyword list, the section estimation unit 180 uses the start time of the utterance section including the appearance time as the start of the procedure corresponding section corresponding to the case procedure ID. Estimated as time. Further, only when the current and previous case procedure IDs are the same, the process of step S202 for acquiring the utterance section information of the utterance of the operator including the appearance time may be performed.

  On the other hand, when the current case procedure ID and the previous case procedure ID are the same (Yes in step S203), the section estimation unit 180 considers that the description of the same case procedure ID is continuing, and sets the start time of the section corresponding to the procedure. The process proceeds to step S205 without estimation.

  Then, the section estimation unit 180 performs the processes from step S201 to step S204 until the processes from step S201 to step S204 are performed on all records registered as the recognition keyword list (No in step S205). Repeat. Thereafter, when the processing for all the records registered as the recognition keyword list is completed (step S205 Yes), the section estimation unit 180 ends the estimation of the procedure corresponding section.

  Returning to the description of FIG. 14, while processing the voice data system, the operation time acquisition unit 170 sets the recording period of the voice data of the call ID for which the supplemental word / phrase has not been extracted as the extraction range of the PC operation log, and performs the PC An operation time list is generated from the log storage unit 110 (step S101B).

  After steps S101A to S104A and step S101B are completed in this way, the section estimation unit 180 performs processing for estimating the corresponding section of the procedure explanation phrase using the operation time list (step S105).

(3) Process for Estimating Corresponding Section of Procedure Explanation Words Here, the process of estimating the corresponding section of the procedure explanation words shown in step S105 of FIG. 14 will be described using FIG. FIG. 16 is a flowchart illustrating a procedure of a process for estimating a corresponding section of a procedure explanatory word according to the second embodiment. The section estimation unit 180 executes the following process with one procedure included in the answer item of the QA example as a processing unit, and repeats the process until all the procedure corresponding sections estimated in step S104A are processed. To do.

  As illustrated in FIG. 16, the section estimation unit 180 acquires the start time and end time of the procedure corresponding section obtained by the previous procedure corresponding section estimation (step S301). Thereafter, the section estimation unit 180 selects the operation time and the operation part after switching in order from the operation time with the shortest operation time (elapsed time from the recording start time) in the operation time list output by the operation time acquisition unit 170. Obtain (step S302).

  Then, the section estimation unit 180 extracts a procedure explanation word / phrase having a correspondence relationship with the operation part after switching from the correspondence relationship storage unit 125 (step S303). At this time, when the procedure explanation word is extracted (step S304 Yes), the section estimation unit 180 refers to the utterance section information of the operator and utters the operator's utterance corresponding to the operation time when the switching operation is performed (N ) Is acquired (step S305). Furthermore, the section estimation unit 180 acquires the utterance (N + 1) of the operator next to the acquired utterance of the operator (step S306). In step S305, if the operator's utterance including the operation time exists, the utterance (N) of the operator is acquired. If the operator's utterance including the operation time does not exist, the operator's utterance (N) is acquired. The operator's utterance (N) is acquired.

  At this time, if the next operator's utterance section (N + 1 utterance section) is longer than the predetermined period (step S307 Yes), the section estimation unit 180 describes the start time of the next operator's utterance (N + 1) as a procedure. The end time of the speech section of the phrase is estimated (step S308).

  On the other hand, when the next operator's utterance section (N + 1 utterance section) is shorter than the predetermined period (No in step S307), the section estimation unit 180 acquires the next operator's utterance (N + 2) (step S306). . Then, the section estimation unit 180 further determines whether or not the next utterance (N + 2) of the operator is shorter than a predetermined period. In this manner, the section estimation unit 180 searches for an operator utterance until an operator utterance longer than a predetermined period is found (step S307 Yes).

  If the procedure explanation word is not extracted (No in step S304), the section estimation unit 180 estimates the corresponding section of the procedure explanation word because the transition of the operation part after switching is an operation part not related to the procedure explanation word. Without moving to step S309.

  Then, the processing from step S302 to step S308 is repeated until the processing from step S302 to step S308 is performed on all the records registered as the operation time list (No in step S309).

  Thereafter, when the processing for all the records registered as the operation time list is completed (step S309 Yes), the section estimation unit 180 outputs the estimation result of the corresponding section related to the procedure explanation word to the supplemental word extraction unit 190 (step S310). . Note that the processes in steps S301 to S310 are repeated until all the procedure corresponding sections are processed.

  Returning to the description of FIG. 14, the supplemental phrase extraction unit 190 uses the estimation result of the speech section of the procedure explanation phrase estimated by the section estimation unit 180 and the utterance section information of the operator extracted by the utterance section extraction unit 155. Then, supplemental phrases are extracted (step S106).

(4) Supplemental Phrase Extraction Processing Here, the processing for estimating the corresponding section of the procedure explanatory words shown in step S106 of FIG. 14 will be described using FIG. FIG. 17 is a flowchart illustrating a procedure of supplemental phrase extraction processing according to the second embodiment. Note that the supplemental phrase extraction unit 190 executes the following processing using the estimation result of one speech segment as a processing unit, and repeatedly performs the processing until all the estimation results of the speech segment estimated in step S105 are processed. .

  As shown in FIG. 17, the supplemental phrase extraction unit 190 includes the operator's utterance section information extracted by the utterance section extraction unit 155, and includes the operator's speech section of the procedure explanation phrase estimated by the section estimation unit 180. The utterance section of the utterance is acquired (step S401).

  Then, the supplemental phrase extracting unit 190 sets the speech data of the utterance section of the operator's utterance acquired previously as the supplemental phrase extraction range, and performs speech recognition on the speech data in the supplemental phrase extraction range to extract the supplemental phrase. (Step S402).

  Thereafter, the supplemental phrase extraction unit 190 associates the supplemental phrase extracted in this way with the procedure explanation phrase and the example procedure ID of the speech section used to determine the extraction range of the supplemental phrase, thereby obtaining the supplemental phrase list. Generate (step S403). Note that the processing in steps S401 to S403 is repeated until processing is performed on the estimation results of all speech sections.

[Example of supplemental word extraction]
An example in which the operator support apparatus 100 described above extracts supplemental phrases from voice data will be described with reference to FIGS. 18 and 19. FIG. 18 and FIG. 19 are explanatory diagrams for explaining an extraction example of supplemental phrases according to the second embodiment. In the example of FIGS. 18 and 19, it is assumed that an inquiry regarding the operation of the PC is received at the call center. FIG. 18 and FIG. 19 both show an example of the dialog explaining the procedure “open the connection tab of the Internet option” for the inquiry item “Tell me how to set the browser” from the customer. Among these, the example in FIG. 18 shows a dialog when the Internet option branches to a dialog centered on. Further, in the example of FIG. 19, the dialog when the connection tab is branched to the center is shown.

  Also, the horizontal axis in FIGS. 18 and 19 indicates the time axis, and indicates that time elapses as it proceeds in the right direction. Also, “u” in the figure is an abbreviation for “utterance”, and blocks from “u1” to “u12” indicate an utterance of an operator or a customer, and the horizontal size of these blocks is the length of the utterance section. It shows. Of the upper and lower blocks on the time axis, the block shown above the horizontal axis indicates the customer's utterance, and the block shown below the horizontal axis indicates the operator's utterance. 18 and FIG. 19, the “operation part” below the time axis indicates the operation part after switching acquired from the PC operation log, and the horizontal size of the range indicated by the arrow indicates that the operation part is operated by the operation part. Indicates the length of time covered by. 18 and FIG. 19, the “corresponding section” further below the time axis indicates a speech section related to the procedure explanation word estimated by the section estimation unit 180, and the horizontal size of the range indicated by the arrow is estimated. The length of the voice section related to the procedure explanation word is shown.

  In the example shown in FIG. 18, it is assumed that the following conversation has been performed. The operator speaks, "Please open the Internet Options connection tab" (u1). The customer speaks “What?” (U2). The operator speaks “second from the right of the menu bar” (u3). The customer speaks “Menu bar?” (U4). The operator utters "It is lined up with files, edits, etc. at the top ..." (u5). The customer speaks “Ah yes yes” (u6). The operator speaks, “Click on the second tool from the right, click here” (u7). The customer speaks “second ... yes” (u8). The operator speaks, "There is an internet option at the bottom" (u9). The customer speaks “Oh, this is it” (u10). The operator speaks “yes” (u ′). The operator speaks “Please select the connection tab there” (u11). The operator speaks “Next, open the LAN settings” (u12).

  In the example shown in FIG. 18, the following procedure explanation words are extracted from the audio data including the dialogues from u1 to u12. From the utterance of u1, “Internet option” and “connection tab” are extracted as procedure explanation words. “Internet option” is extracted from the utterance of u9 as a procedure explanation phrase. The “connection tab” is extracted from the utterance of u11 as a procedure explanation word. “LAN” is extracted from the utterance of u12 as a procedure explanation word.

  Further, the example shown in FIG. 18 indicates that the operation site is switched in the order of “connection tab”, “browser window”, “Internet option”, and “connection tab” with the passage of time. In the example shown in FIG. 18, time t1 indicates the utterance start time of u3, time t2 indicates the time when the screen transitions to the Internet option screen, time t3 indicates the utterance start time of u11, and time t4 is u12. The utterance start time of

  With reference to the example shown in FIG. 18, an estimation procedure for estimating a speech section related to the procedure explanation word “Internet option” will be described. First, the operator support apparatus 100 acquires the time “t2” when the operation part changes from the “browser window” to the “Internet option”. Then, the operator support apparatus 100 acquires the utterance of the operator including “t2”, that is, “u9”. Then, the operator support apparatus 100 further acquires the next operator utterance, that is, “u ′”. Here, the operator support apparatus 100 further acquires the next utterance when the acquired utterance period is shorter than the predetermined period. This is to prevent the start time of the next speech section from starting from a meaningless utterance such as a conflict. Here, assuming that “u ′” is shorter than the predetermined period, the operator support apparatus 100 acquires the next utterance “u11”. Here, “u11” is longer than a predetermined period. Then, the operator support apparatus 100 acquires the start time of “u11”, that is, “t3”, and estimates it as the end time of the voice section related to the explanation of the procedure explanation phrase “Internet option”. In this manner, the operator support apparatus 100 estimates the voice section related to the description of “Internet option” as “from t1 to t3”.

  In the example shown in FIG. 19, it is assumed that the following conversation is performed. The operator speaks, "Please open the Internet Options connection tab" (u1). The customer speaks “What is the connection tab?” (U2). The operator speaks “Is the Internet option open?” (U3). The customer speaks "I'm open" (u4). The operator utters, “It is lined up with General, Security, etc. at the top ...” (u5). The customer speaks “Oh, is this” (u6). The operator speaks “Yes, please open the connection tab there” (u7). The customer speaks “Yes” (u8). The operator speaks “Next, open the LAN settings” (u9).

  In the example shown in FIG. 19, it is assumed that the following procedure explanation words / phrases are extracted from audio data including dialogues from u1 to u9. From the utterance of u1, “Internet option” and “connection tab” are extracted as procedure explanation words. “Internet option” is extracted from the utterance of u3 as a procedure explanation phrase. The “connection tab” is extracted from the utterance of u7 as a procedure explanation word. “LAN” is extracted from the utterance of u9 as a procedure explanation word.

  Further, the example shown in FIG. 19 indicates that the operation site is switched in the order of “connection tab”, “Internet option”, and “connection tab” with the passage of time. In the example shown in FIG. 19, the time t1 indicates the utterance start time of u5, the time t2 indicates the time of transition to the connection tab screen, and the time t3 indicates the utterance start time of u9.

  With reference to the example illustrated in FIG. 19, an estimation procedure for estimating a speech section related to the procedure explanation word “connection tab” will be described. First, the operator support apparatus 100 acquires the time “t2” when the operation part changes from “Internet option” to “connection tab”. Then, the operator support apparatus 100 tries to acquire the utterance of the operator including the time “t2”, but if there is no corresponding utterance, the operator support apparatus 100 further acquires the next utterance “u7”. Then, the operator support apparatus 100 further acquires the next operator utterance “u9”. Here, it is assumed that “u9” is longer than a predetermined period. Then, the operator support apparatus 100 acquires the start time “t3” of “u9” and estimates it as the end time of the voice section related to the description of the procedure explanation phrase “connection tab”. As described above, the start time of the next utterance “u9” rather than the utterance end time of “u7” is terminated to prevent the start time of the next speech section from being started from the silent section. It is. In this way, the operator support apparatus 100 estimates the voice section related to the description of the “connection tab” as “from t1 to t3”.

[Effects of Example 2]
As described above, the operator support apparatus 100 according to the second embodiment extracts the procedure explanation words related to the screen that has been changed by the operation of the predetermined operation part among the operation parts displayed on the screen. In addition, the time when the utterance is started after the time when the predetermined operation part is operated is extracted, and the speech section up to the extracted time is estimated as the speech section related to the explanation of the extracted procedure explanation phrase. For this reason, the operator assistance apparatus 100 can appropriately extract the supplemental words / phrases corresponding to the procedure explanation words / phrases.

  Further, according to the operator support apparatus 100 according to the second embodiment, a meaningless speech such as a conflict can be included at the end of the speech section of the procedure explanation phrase, and the start time of the speech section of the procedure explanation phrase to be estimated next Can be started from the explanation of the substantial procedure explanation words.

  In addition, in the operator assistance apparatus 100 according to the present embodiment, the case where the supplemental phrase is extracted using the speech section estimated by the section estimation unit 180 has been described. However, the configuration of the disclosed apparatus is not limited thereto. Absent. For example, the length of the speech section estimated by the section estimation unit 180 is equivalent to the time required for one procedure. For this reason, when the time required for the service is statistically calculated, the operator support apparatus 100 according to this embodiment does not perform the statistics for each service as in the conventional case, but the response service is a procedural unit of answer items. It is possible to divide the time into statistics and to calculate the time required for each procedure. Further, in the operator support apparatus 100 according to the present embodiment, the length of the voice section estimated by the section estimation unit 180 is used for the estimation and evaluation of the operator skill and the review of the QA case as a manual. You can also. As a result, according to the operator support apparatus 100 according to the present embodiment, it becomes possible to analyze the improvement policy of the response work by the operator.

  Although the embodiments relating to the disclosed apparatus have been described above, the disclosed apparatus may be implemented in various different forms other than the above-described embodiments. Therefore, other embodiments included in the disclosed apparatus will be described below.

(1) Application to other than PC operation description In the above-described second embodiment, the description has been made assuming that the call center accepts an inquiry related to the PC operation. However, the technology disclosed in the present application is not limited to this. For example, when an inquiry about a predetermined device or software is received at a call center and the device or software can be operated and answered while leaving an operation log, the technology disclosed in the present application can be similarly applied.

(2) Device Configuration, etc. Each component of each device shown in FIG. 2 does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, for example, the functions of the prosodic information extraction unit 150 and the speech segment extraction unit 155 can be integrated and integrated. In addition, the operator support apparatus may include a part or all of the correspondence setting unit 140, the prosody information extraction unit 150, the speech segment extraction unit 155, the speech recognition unit 160, the operation time acquisition unit 170, the segment estimation unit 180, or the supplemental phrase extraction unit 190. You may connect via a network as an external device. In addition, a part or all of the correspondence setting unit 140, the prosody information extraction unit 150, the speech segment extraction unit 155, the speech recognition unit 160, the operation time acquisition unit 170, the segment estimation unit 180, or the supplemental phrase extraction unit 190 may be replaced with another device. The structure which has is adopted. In addition, the functions of the operator support apparatus 100 described above may be realized by another apparatus connected to the network and cooperating. Further, a configuration is adopted in which another device has a part or all of the PC operation log storage unit 110, the case data storage unit 120, the correspondence relationship storage unit 125, and the voice data storage unit 130, respectively. In addition, the functions of the operator support apparatus 100 may be realized by another apparatus connected to the network and cooperating.

(3) Operator support program The various processes (for example, see FIG. 14) of the operator support apparatus 100 described in the above embodiment are executed by a computer system such as a personal computer or a workstation. Can also be realized. Therefore, in the following, an example of a computer that executes an operator support program having the same function as the operator support apparatus described in the above embodiment will be described with reference to FIG. FIG. 20 is a diagram illustrating a computer that executes an operator support program.

  As shown in FIG. 20, the computer 200 as an operator support apparatus is configured by connecting a communication control unit 210, an HDD 220, a RAM 230, and a CPU 240 via a bus 300.

  Here, the communication control unit 210 controls communication related to the exchange of various types of information. The HDD 220 stores information necessary for the CPU 240 to execute various processes. The RAM 230 temporarily stores various information. The CPU 240 executes various arithmetic processes.

  As shown in FIG. 20, the HDD 220 stores in advance an operator support program 221 that exhibits the same functions as the processing units of the operator support apparatus 100 described in the above embodiment, and operator support data 222. Has been. Note that the operator support program 221 may be appropriately distributed and stored in a storage unit of another computer that is communicably connected via a network.

  Then, the CPU 240 reads out the operator support program 221 from the HDD 220 and expands it in the RAM 230, so that the operator support program 221 functions as an operator support process 231 as shown in FIG.

  That is, the operator support process 231 reads the operator support data 222 and the like from the HDD 220, expands the data in the area allocated to itself in the RAM 230, and executes various processes based on the expanded data and the like.

  Note that the operator support process 231 is performed by, for example, the correspondence setting unit 140, the prosody information extraction unit 150, the speech segment extraction unit information 155, the voice recognition unit 160, and the operation time acquisition unit 170 of the operator support apparatus 100 illustrated in FIG. Corresponds to the process to be executed. The operator support process 231 corresponds to, for example, processing executed in the section estimation unit 180 and the supplemental phrase extraction unit 190 of the operator support apparatus 100 illustrated in FIG.

  Note that the above-described operator support program 221 is not necessarily stored in the HDD 220 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 200. Then, the computer 200 may read and execute each program from these.

  Further, each program is stored in “another computer (or server)” connected to the computer 200 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 200 may read and execute each program from these.

DESCRIPTION OF SYMBOLS 1 Operator assistance apparatus 2 Memory | storage part 3 Operation time acquisition part 4 Section estimation part 5 Supplement word extraction part 100 Operator assistance apparatus 110 PC operation log memory | storage part 120 Case data memory | storage part 125 Correspondence relation memory | storage part 130 Voice data memory | storage part 140 Correspondence relation setting Unit 150 prosodic information extraction unit 155 speech segment extraction unit 160 speech recognition unit 170 operation time acquisition unit 180 segment estimation unit 190 supplemental phrase extraction unit

Claims (3)

A correspondence storage unit that stores a procedure explanation word that is a word for explaining a predetermined procedure and an operation part that is operated in relation to the procedure explanation word among the operation parts displayed on the terminal in association with each other; ,
Of the operation parts displayed on the terminal, from the PC operation log storage unit that stores the operation part that has been the operation target from that time for each switching operation time when the operation part is switched to another operation part. An operation time acquisition unit for acquiring an operation time indicating a time point when an operation for switching a predetermined operation part is performed;
When the procedure explanation words related to the operation parts switched from the predetermined operation parts are extracted from the correspondence storage unit, and when the procedure explanation words are first extracted from the voice recording , the appearance time of the procedure explanation words is recorded as voice. When extracting a procedure explanation phrase different from the procedure explanation phrase extracted last time from the voice recording, extracted as the start time of the section, extract the utterance start time after the operation time acquired by the operation time acquisition unit, and the procedure When the explanatory words are extracted for the first time, the voice interval from the start time of the voice interval to the utterance start time point, and when the procedure explanatory words different from the previously extracted procedure explanatory words are extracted, the different Opening an utterance section that appears next to the utterance section at the time of extraction from the utterance section in which the procedure explanation phrase appears and is longer than a predetermined period Voice section up to the time, the interval estimation unit that estimates a speech section on the procedures described phrase extracted from the correspondence relation storage unit,
An operator support apparatus, comprising: a supplemental phrase extracting unit that extracts a supplemental phrase that is a phrase for supplementing the procedure explanation phrase from the speech section estimated by the section estimation unit in the voice record. On the computer,
For each switching operation time when the operation part is switched to another operation part, from the PC operation log storage unit that stores the operation part that has been an operation target from that time in association with the predetermined operation part displayed on the terminal An operation time acquisition procedure for acquiring an operation time indicating the time point when the operation of switching the operation part of
Correspondence storage unit that associates and stores a procedure explanation word that is a word for explaining a predetermined procedure and an operation part that is operated in relation to the procedure explanation word among the operation parts displayed on the terminal When the procedure explanation word / phrase related to the operation part switched from the predetermined operation part is extracted, and the procedure explanation word / phrase is first extracted from the voice recording , the appearance time of the procedure explanation word / phrase is set as the start point of the voice section. When a procedure explanation phrase that is different from the procedure explanation phrase previously extracted from the voice recording is extracted, the utterance start time after the operation time acquired by the operation time acquisition means is extracted, and the procedure explanation phrase is first In the case where the speech segment is extracted from the start point of the speech segment to the start point of the utterance, the procedure explanation different from the previously extracted procedure explanation phrase In case of extracting the clause, the different procedures described phrase appears from the appearance and speech segment to the next speech period of time of the extracted speech segment up to the start time of a long speech period than the predetermined period, and interval estimation procedure for estimating the speech section on the procedures described phrase extracted from the correspondence relation storage unit,
An operator assistance program for executing a supplemental phrase extraction procedure for extracting a supplemental phrase that is a phrase for supplementing the procedure explanation phrase from a speech section estimated by the section estimation procedure in the speech recording . Computer
For each switching operation time when the operation part is switched to another operation part, from the PC operation log storage unit that stores the operation part that has been an operation target from that time in association with the predetermined operation part displayed on the terminal An operation time acquisition step of acquiring an operation time indicating the time point when the operation of switching the operation part of
Correspondence storage unit that associates and stores a procedure explanation word that is a word for explaining a predetermined procedure and an operation part that is operated in relation to the procedure explanation word among the operation parts displayed on the terminal When the procedure explanation word / phrase related to the operation part switched from the predetermined operation part is extracted, and the procedure explanation word / phrase is first extracted from the voice recording , the appearance time of the procedure explanation word / phrase is set as the start point of the voice section. When a procedure explanation phrase that is different from the procedure explanation phrase previously extracted from the voice recording is extracted, an utterance start time after the operation time acquired by the operation time acquisition step is extracted, and the procedure explanation phrase is first In the case where the speech section is extracted from the start time of the speech section to the utterance start time, a procedure different from the previously extracted procedure explanation word / phrase In case of extracting the light phrase, the different procedures described phrase appears from the appearance and speech segment to the next speech period of time of the extracted speech segment up to the start time of a long speech period than the predetermined time period , A section estimation step for estimating a voice section related to a procedure explanation word extracted from the correspondence storage unit;
An operator assistance method comprising: performing a supplemental phrase extracting step of extracting a supplemental phrase that is a phrase for supplementing the procedure explanation phrase from the speech section estimated by the section estimation step in the speech recording .

Images