JP4272658B2 - Program for functioning a computer as an operator support system - Google Patents

Description

  The present invention relates to a program for causing a computer to function as an operator business support system.

  In call center operations, how to shorten the processing time per case is a major proposition for reducing operational costs. In a standardized business of ordering and receiving (for example, mail order), various support tools have been developed and applied, such as a system that pops up a response manual and customer information on the screen as soon as an incoming call is received. On the other hand, technical support call centers such as home appliances and PCs, which are inquiries for operation methods and troubleshooting, can be long, such as a wide range of response contents and several tens of calls. For this reason, it is difficult to standardize the response scenario, and there are many parts that rely on operator skills.

  In these tasks, the operator usually creates the contents of inquiries and answers from the user as a work log after each call. However, when explaining the operation to the user, there are many cases where the operator also makes a call while confirming the operation and display contents at the same time, and it is difficult to record the contents of the conversation during the call. For this reason, the work log creation work has to be carried out while remembering the conversation with the user, which takes a lot of load and time. For this reason, a system for converting voice data during a call into a character string and recording it has been developed. An example of such an operator work support system is disclosed in Patent Document 1.

JP 11-338494 A

  However, in the conventional operator work support system, the text obtained as a result of the speech recognition is only displayed as it is, so that there is a problem that the working efficiency of the operator is not improved. For example, a display function for helping the operator to grasp the text in a short time and a function for obtaining information that the operator wants to refer to in relation to the text are not provided.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a program for causing a computer to function as an operator work support system that improves the work efficiency of an operator.

In order to solve the above-mentioned problems, a program according to the present invention is a program for recording a call via a telephone and for supporting the operator work associated with the call and for causing the computer to function as an operator work support system. The computer includes a voice recognition unit that converts voice data of a call into text data, a first dictionary, the voice recognition dictionary, and a keyword specified in advance for each word used for voice recognition, As a dictionary, a second dictionary that includes keywords specified in advance, a response log processing unit that processes text data, an output unit that displays processed text data, and an input unit that accepts operator operations The keyword in the second dictionary can be changed, and the voice recognition unit When the speech recognition unit converts the speech data into text data, the speech recognition unit performs the marking on the keywords included in the first dictionary, and the response log processing unit performs the first processing on the marked text data. The keywords included in the two dictionaries are further marked, and the output unit highlights the marked keywords, the first dictionary further includes explanatory text relating to the keywords, and the second dictionary further includes When the input unit accepts an operation for designating a keyword, the output unit obtains and displays a description related to the keyword from either the first dictionary or the second dictionary. Yes, in a different area from the text data display, keywords are listed in the order they appear in the call, When accepting an operation for designating a radical 19 is listed keywords, it causes a computer to the text data including the specified keyword as an operator service support system is for color display.

The output unit may highlight the marked keyword by surrounding it with black brackets.
The output unit may further display a list of keywords in an order different from that of text data in the order in which the keywords appear in the call.
Output unit further when the input unit accepts an operation for designating a keyword may be one for reproducing audio data corresponding to the text data including the keyword.
The computer, in addition, a computer to function as a recognition result storage unit for storing text data in a storage unit having an output unit further displays the text data in the text editing area, the recognition result storage unit, an input unit When an operation for correcting text data is received, the corrected text data may be stored in a storage unit included in the computer.
When the input unit further receives an operation for designating a keyword, the output unit may display a list of stored calls including the designated keyword.

  According to the program for causing the computer to function as the operator work support system according to the present invention, the keyword specified in advance is highlighted, so that the work efficiency can be improved.

Embodiments of the present invention will be described below with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration including an operator work support system 100 including a voice recognition engine and an operator work support tool according to Embodiment 1 of the present invention.
A private branch exchange (PBX) 4 for a call center is connected to a public switched telephone network (PSTN) 2. This private branch exchange 4 is, for example, a definition of AVAYA.

The private branch exchange 4 is connected to a telephone 12 that is an operator telephone that receives a call addressed to the operator 20. Further, the telephone set 12 is connected to a PC 16 and a headset 18 which are operator PCs used by the operator 20 for business via a plug adapter 19 and a switch box 14 which is a switching unit.
The telephone 12 receives a signal from the private branch exchange 4 and transmits it to the switch box 14 via the plug adapter 19 and receives a signal from the switch box 14 and transmits it to the private branch exchange 4. A microphone output terminal 12b is used. For example, Callmaster IV of AVAYA is used. Depending on the type of telephone, the function of the plug adapter may be included, or it may be a normal modular jack.

  The headset 18 includes a speaker 18a that converts a signal from the switch box 14 into voice and transmits the voice to the operator 20, and a microphone 18b that converts voice generated by the operator 20 into a signal and sends the signal to the switch box 14, for example. VXI's TuffSet series and Passport series are used.

The PC 16 is a computer having a known configuration including an arithmetic device and a storage device, and includes a display and speaker output terminals as output devices, and a keyboard, mouse, and line input terminals as input devices. For example, a Pentium (registered trademark) 4 processor operating at a clock frequency of 3 GHz is provided as an arithmetic unit, and a memory having a capacity of 1 GB and a hard disk drive having a capacity of 160 GB are provided as a storage device. The PC 16 may include other well-known input / output devices such as a network interface.
Note that the PC 16 can simultaneously receive two audio inputs, for example, signals that are independent audio data such as stereo audio L channel and R channel, through a line input terminal, and store them in a storage device as a single file. Is possible. However, in this file, the L channel and the R channel are recorded independently and can be reproduced independently through a speaker output terminal later.

FIG. 3 shows an outline of the logical configuration of the PC 16. The PC 16 includes a speech recognition engine 30 that receives input speech and performs speech recognition.
The storage device of the PC 16 includes a business-use dictionary storage unit 40 that stores and provides a speech recognition dictionary necessary for performing speech recognition processing according to business.
The business dictionary storage unit 40 has, for example, a plurality of speech recognition dictionaries for each business handled by the operator, language models suitable for the contents of the call are registered, and company names that are necessary in accordance with the situation in each business Further, a word such as a product name may be additionally registered, a highlighting keyword and an unnecessary word keyword may be additionally registered, and further an explanatory text relating to the highlighting keyword may be registered. These words, highlighting keywords, unnecessary word keywords, and explanations can be easily registered, changed, and deleted using a system or program that is not included in the operator work support tool 39.

  Further, the storage device of the PC 16 has a keyword dictionary storage unit 45. The keyword dictionary storage unit 45 includes highlighting keywords, unnecessary word keywords, and immediate reference keywords that are extracted after the speech recognition process. Furthermore, an explanatory note relating to the highlighting keyword may be registered. These highlighting keywords, unnecessary word keywords, immediate reference keywords, and explanations can be easily registered, changed, and deleted using a system or program that is not included in the operator work support tool 39.

In the aforementioned speech recognition dictionary, keywords are registered in units of words used for speech recognition, and keywords that match during speech recognition are extracted. However, in the keyword dictionary, it is possible to register document keywords as well as words. Yes, it is used to further extract keywords from the speech recognition result.
The highlighting keyword is used to highlight the corresponding keyword of the speech recognition result.
The unnecessary word keyword is used for changing the display format of the keyword or deleting the display of the unnecessary word.
Immediately after, the reference keyword is used to reproduce the sound that follows the keyword.
The explanatory text is used to explain the highlighting keyword.
Further, the storage device of the PC 16 includes an acoustic dictionary (not shown). The acoustic dictionary has a plurality of acoustic dictionaries such as men, women, and men and women, and is used for recognition of phoneme levels.

The voice recognition engine 30 includes an echo canceling unit 31 that reduces the voice of the operator 20 leaking into the output from the telephone 12 to clarify the user voice, a recording unit 32 that records the output of the echo canceling unit 31, Of the outputs from the echo canceling unit 31, the operator speech recognition unit 33 that performs speech recognition processing using the operator voice, the business dictionary storage unit 40, and an acoustic dictionary (not shown) as inputs, and the output from the echo canceling unit 31 It includes a user voice, a business dictionary storage unit 40, and a user voice recognition unit 34 that performs voice recognition processing using an acoustic dictionary (not shown) as an input.
Note that the voice recognition engine 30 may not include the echo canceling unit 31. For example, in the case where only the operator voice recognition unit 33 is included and the user voice recognition unit 34 is not included, or in the voice output from the telephone set 12 or the switch box 14, the user voice and the operator voice can be separated to such a degree that voice recognition is possible. In this case, the echo cancellation unit 31 is not necessary.

The storage device of the PC 16 stores a voice storage unit 41, an operator voice recognition result storage unit 42, and a user voice recognition result storage for storing outputs of the recording unit 32, the operator voice recognition unit 33, and the user voice recognition unit 34, respectively. Part 43 is included.
The operator work support tool 39 includes a reception log processing unit 35, a call management unit 38, an output unit 36, and an input unit 37.
The output unit 36 performs output to the operator 20, for example, display on a display, and sound reproduction to the speaker 18 a via the switch box 14.
The input unit 37 receives an input from the operator 20 using, for example, a keyboard or a mouse.

The response log processing unit 35 displays a response log screen, which will be described later with reference to FIGS. 8 to 10, by the output unit 36, accepts a user instruction through the input unit 37, and stores a voice storage unit 41 for one response log during a call or in the past. , The operator voice recognition result storage unit 42, the user voice recognition result storage unit 43, and the call log management information storage unit 44 are referred to and updated, and the keyword dictionary storage unit 45 is referred to.
The call management unit 38 displays a setting screen, which will be described later with reference to FIG. 11, on the output unit 36, receives a user instruction through the input unit 37, and manages information related to the operator stored in the call log management information storage unit 44. Further, the call management unit 38 displays a dictionary setting screen, which will be described later with reference to FIG. 12, by the output unit 36, accepts a user instruction through the input unit 37, and receives a response log and voice recognition stored in the call log management information storage unit 44. Manage relationships with dictionaries. Further, the call management unit 38 displays a response log list screen, which will be described later with reference to FIG. 13, on the output unit 36, accepts a user instruction through the input unit 37, and stores each response log stored in the call log management information storage unit 44. Manage attribute information.

Further, the storage device of the PC 16 includes a call log management information storage unit 44 that stores information used when the call management unit 38 manages a call log.
The voices of the operator and the user input in real time via the echo canceling unit 31 are stored in the voice storage unit 41 by the recording unit 32 and transmitted to the operator voice recognition unit 33 and the user voice recognition unit 34 in parallel. Is done. The operator voice recognition unit 33 and the user voice recognition unit 34 immediately recognize the transmitted voice and transmit the result to the reception log processing unit 35 in real time, and at the same time, the operator voice recognition result storage unit 42 and It transmits to the user speech recognition result storage unit 43. Thereby, the reception log processing unit 35 can refer to the voice recognition result in real time during the call. In FIG. 3, the operator speech recognition unit 33 and the user speech recognition unit 34 directly transmit the recognition result to the operator speech recognition result storage unit 42 and the user speech recognition result storage unit 43. 33 and the user voice recognition unit 34 may be transmitted to the operator voice recognition result storage unit 42 and the user voice recognition result storage unit 43 via the response log processing unit 35.

Further, the voice input to the echo cancellation unit 31 is input by assigning the operator voice to the R channel and the user voice to the L channel. Both the operator voice and the user voice are transmitted from the echo cancel unit 31 to the recording unit 32 and recorded as one file in the voice storage unit 41. In parallel, the operator voice is transmitted to the operator voice recognition unit 33 for voice recognition, and a start time and an end time are added in sentence units. In parallel, the user voice is transmitted to the user voice recognition unit 34 for voice recognition, and start time and end information are added for each sentence. With such a configuration, the operator voice and the user voice stored in the voice storage unit 41, the start time and the end time included in the recognition result stored in the operator voice recognition result storage unit 42, and the user voice recognition result storage Time consistency is maintained between the start time and the end time included in the recognition result stored in the unit 43.
Thus, the recording unit 32, the operator voice recognition unit 33, and the user voice recognition unit 34 receive a call in parallel.
Note that only one of the operator voice recognition unit 33 and the user voice recognition unit 34 may be configured. In general, the operator voice is clear because it is a microphone voice, and it is a utterance trained so that it can be easily recognized. Therefore, the recognition rate is high, the user voice is unclear because it is a telephone voice, and the utterance is not individual. Many recognition rates are low. In such a case, the user voice recognition unit 34 may not be provided, and only the operator voice recognition unit 33 may be configured.

  Note that the configuration of the PC 16 described above may be distributed in a plurality of computers or the like instead of a single PC. Further, the configuration of the PC 16 described above may be a system in which the voices of all operators are collectively processed by a server, instead of being mounted on individual PCs arranged at the operator's seat.

FIG. 4 shows data formats stored in the voice storage unit 41, the operator voice recognition result storage unit 42, and the user voice recognition result storage unit 43.
The voice storage unit 41 is, for example,. A WAVE format audio file including an extension of wav at the end of the file name is stored. This voice file includes voice data of a call between the operator 20 and the user, and one voice file is created for one call. Further, in one file, user voice is stored as L channel data and operator voice is stored as R channel data in a format that can be independently extracted.

The operator speech recognition result storage unit 42 stores a text format file (text file). This text file includes text data corresponding to the operator voice in a call, for example, text data of a voice recognition result, and one text file is created for one call. Each text file has a one-to-one correspondence with the voice file stored in the voice storage unit 41. For example, the operator voice data included in the file “call 1.wav” in the voice storage unit 41 includes This corresponds to the text included in the file “call log 1 operator.dct” in the speech recognition result storage unit 42.
Similarly to the operator voice recognition result storage unit 42, the user voice recognition result storage unit 43 stores text data corresponding to the user voice in a call. Similarly, for example, user voice data included in the file “call 1.wav” in the voice storage unit 41 corresponds to text included in the file “call log 1 user.dct” in the user voice recognition result storage unit 43. .

FIG. 5 shows a data format of the call log management information stored in the call log management information storage unit 44.
The call log management information is stored in a tabular format with data related to one call as one line. Each row stores the start and end times of the corresponding call, the operator name of the corresponding operator 20, the call category specified by the operator 20, the dictionary name used for voice recognition, and voice data. Voice file names and file names in which voice recognition results of operator voices and user voices are respectively stored.

FIG. 6 shows a data format of operator information stored in the call log management information storage unit 44. Each line includes an operator name, which is an operator identification name, and the gender of the operator, and may include other information such as a registration date and remarks. These pieces of information may be stored in a CSV format text, or may be stored in a database format.
FIG. 7 shows a data format of call log data stored in the operator voice recognition result storage unit 42. Each line includes start time, end time, recognition result text, and may include other information. The start time and the end time are relative times from the start of the audio data stored in the audio storage unit 41. Note that the start time and the end time are not necessarily time, and may be values that can calculate the start time and the end time to be displayed on the screen and can serve as pointers to the audio data. These pieces of information may be stored in a CSV format text, or may be stored in a database format.

  The connection between the switch box 14 and the PC 16 in FIG. 1 may be either an audio format or a USB format. In the case of the USB format, it is not necessary to perform digitization in the switch box and provide the PC 16 with a sound card.

The operator work support system 100 includes a switch box 14, a PC 16, and a headset 18. In addition, the operator work support system 100 determines whether the operator 20 uses the headset 18, the telephone set 12, and the PC 16 to make a call via the telephone set 12 or to listen to a call stored in the PC 16. It can be switched. This switching is performed by operating the switch 14a of the switch box 14. When the switch 14 a is set to the “call” side, the connection of the headset 18, the speaker output terminal 12 a of the telephone set 12, the microphone input terminal 12 b of the telephone set 12, and the PC 16 performs a call via the telephone set 12. Composed. At this time, the operator 20 can talk with an external call partner, that is, a user through the telephone set 12 in a call, that is, a call allocated by the private branch exchange 4. Also, when the switch 14a is set to the “listen” side, these connections are configured to wake up the call stored in the PC 16.
Hereinafter, in this specification, the state in which the switch 14a is set to the call side is referred to as “during call”, and the state in which the switch 14a is set to the listening side is referred to as “listen”.

FIG. 2 shows the configuration of the switch box 14.
The switch box 14 includes an interlocking relay 14b for switching the connection of the headset 18, the telephone set 12, and the PC 16. This interlocking relay 14b switches the speaker 18a and microphone 18b of the headset to either the call side or the listening side according to the state of the switch 14a. The switching of the speaker 18a and the microphone 18b is interlocked. When the call is made, that is, when the speaker 18a is connected to the call side, the microphone 18b is also connected to the call side, and when the call is made, that is, the speaker 18a is connected to the call side. When it is set, the microphone 18b is also connected to the listening side.
The switch box 14 further includes a plurality of amplifiers 14c for amplifying the audio signal at various points in the circuit.

A signal input to the switch box 14 of FIG. 1 from the speaker of the telephone 12 is amplified by the amplifier 14 c and then output to the L channel of the voice input of the PC 16. In addition, this signal is output to the speaker 18a of the headset 18 during a call, but is not output to the speaker 18a during listening.
The signals input to the switch box 14 from the L channel and the R channel of the audio output of the PC 16 are first amplified by the amplifier 14c, synthesized, and then further amplified by another amplifier 14c. The finally amplified signal is output to the speaker 18a of the headset 18 when aroused, but not output to the speaker 18a during a call.
The signal input from the microphone 18b of the headset 18 to the switch box 14 is amplified by the amplifier 14c and then output to the R channel of the audio input of the PC 16. Also, this signal is output to the microphone of the telephone 12 during a call, but is not output to the microphone of the telephone 12 during a conversation.

Next, a response log screen, which is an example of a screen displayed when the PC 16 executes the response log processing unit 35 of the operator work support tool 39, will be described with reference to FIGS.
FIG. 8 shows a reception log screen 50 when the call is not being made. The response log screen 50 is a button for transmitting an instruction to the PC 16 when the operator 20 operates (for example, clicks), a call start button 52, a confirmation button 60, a play button 62, a pause button 64, a stop button 66, An enlarge button 68, a reduce button 70, a full button 72, a list display button 74, and a setting button 76 are included. On the upper left of the screen, a call start time, a call end time, and a response operator related to the response log to be awakened are displayed.

The response log screen 50 includes a category selection box 80 that indicates the type of response log when operated by the operator 20 (for example, click to display options and then click to select). Furthermore, the reception log screen 50 includes a reception log display unit 92 that displays call contents in units of lines by characters, and a waveform display unit 94 that displays voice waveforms of the call contents. The response log display unit 92 includes a time column 92a, a user column 92b, an operator column 92c, and a scroll bar 92d. The waveform display unit 94 includes an operator column 94a, a user column 94b, and a scroll bar 94d. Including.
As described above, in the response log screen 50, the response log display unit 92 which is a voice recognition result display region having a time on one axis and a user and an operator on the other axis, and an operator voice region for displaying a waveform related to the operator voice. A waveform display unit 94 including a user voice area for displaying a waveform related to the user voice is displayed on the same screen.
The response log screen 50 includes an operator input box 77 and a voice recognition dictionary selection box 78 for selecting an operator name and a voice recognition dictionary, respectively. Furthermore, the reception log screen 50 includes a save button 81 for storing the reception log and the call category in the operator voice recognition result storage unit 42, the user voice recognition result storage unit 43, and the call log management information storage unit 44. And an end button 98 for ending the execution of the operator work support tool 39.

FIG. 9 shows a reception log screen 50 when a call is in progress and voice recognition is being executed. Compared to FIG. 8, the call start button 52, the confirm button 60, and the end button 98 are not displayed, and the call end button 58 and the interruption button 54 are displayed.
FIG. 10 shows a response log screen 50 when a call is in progress and voice recognition is interrupted. Compared to FIG. 8, the call start button 52, the confirm button 60, and the end button 98 are not displayed, and the call end button 58 and the resume button 56 are displayed.

The operation of the operator work support system 100 configured as described above will be described below using the screen examples shown in FIGS. 8 to 13 and the flowchart shown in FIG. 14 together with the operation of the operator work support tool 39. .
The flowchart of FIG. 14 shows the flow of processing of the operator work support tool 39 when the operator 20 responds to a call from an external user.
The telephone 12 is not busy, and a response log screen 50 shown in FIG. 8 is displayed on the display of the PC 16 (step S11). The switch 14a is set to the call side.

When a call from an external user arrives at the telephone 12, the operator 20 confirms that the switch 14a is set to the call side. If it is set to the listening side, switch to the call side.
The operator 20 receives a call with the headset 18 and clicks the call start button 52 (step S13). In response to this input, the output unit 36 deletes any call contents and waveforms that have been displayed in the response log display unit 92 and the waveform display unit 94 so far (step S14).

  Subsequently, the recording unit 32 starts recording the content of the call (step S16). In recording the contents of the call, the recording unit 32 records, that is, records the user voice inputted from the switch box 14 to the L channel for voice input and the operator voice inputted to the R channel in the form of stereo voice. Furthermore, the text obtained by voice recognition described later and the related time are also recorded. 2 cancels the operator voice leaked into the output from the telephone 12 to clarify the user voice, and the recording section 32 stores the operator voice and the user voice in the voice storage section 41.

  The operator voice recognition unit 33 detects a part where the volume of the operator voice is less than a certain level over a period of the utterance of the operator 20, for example, 300 ms or longer (step S17). When this segment is detected, the operator voice recognition unit 33 receives the operator voice input between the segment detected immediately before or the recording start time and the newly detected segment as one sentence, and this is the unit. Is recognized and converted into text data, and matched with keywords important for the corresponding business registered in the business dictionary storage unit 40. If a matching keyword is included, before and after the keyword Marking is performed by placing a predetermined reserved word on (step S18). The results of performing speech recognition and keyword detection in this way are stored in the operator speech recognition result storage unit 42.

The response log processing unit 35 detects the keyword stored in the keyword dictionary storage unit 45 and performs the marking in the same manner as in step S18, and also displays the recognition result via the output unit 36 in the operator column 92c of the response log display unit 92. To display. At this time, the break of the previous utterance or the time at the start of recording is displayed in the time column on the same line as the text (step S19).
In this way, during the call, the PC 16 stores the utterance content of the operator 20 in real time as the call progresses and displays it on the display.
When the voice data input is more than a certain interval, the converted text data is also divided at that time and managed as a plurality of text data strings arranged in time series.

Similarly, in the user column 92b of the reception log display unit 92, text that is a voice recognition result of the user voice is displayed. The voice recognition of the user voice is performed by the user voice recognition unit 34, and the resulting text is stored in the user voice recognition result storage unit 43. Similarly, the user voice recognition unit 34 and the response log processing unit 35 perform keyword detection in the text of the user voice.
At this time, the text of the operator voice and the text of the user voice may be alternately displayed every other line, or the output unit 36 may control the display line according to the state of the call. Further, in steps S17 to S19 described above and step S20 described later, a process corresponding to the operator voice, a process corresponding to the user voice, and the two processes are performed independently and simultaneously.
The displayed lines advance from the top to the bottom of the response log display unit 92. When the bottom line is reached, the latest text is always displayed on the bottom line, and the previous display contents scroll up sequentially. Is displayed.

Next, the output unit 36 highlights the keywords detected in Step S18 and Step S19 (Step S20).
The highlighting may be simply highlighting the keywords so that they are conspicuous, but an explanatory text may be displayed when the highlighted character string is selected. Although the details will be described in the arousal support process, when the highlighted character string is selected, the immediately following voice may be reproduced.
For example, as shown in FIG. 8, highlighting is performed by enclosing a keyword in black brackets. This can be done by changing the keyword color, font type, size, etc., underlining, or bolding. Or italicized. Depending on the type of highlighting described above, it may be displayed in a different format.
Examples of registered keywords include the following.
・ Company name ・ General name of the product such as “DVD-RAM” ・ Product name ・ Product model number ・ Expression expressing the phenomenon such as “Cannot format” ・ “I want to receive a phone number” A character string indicating that important information such as the user's personal information or product name appears immediately after that, such as "I need your name" or "I will repeat your name"

The operator 20 operates the scroll bar 92d of the reception log display unit 92 to display the text of a desired time recorded in the past without affecting the execution of voice recognition and the recording of text and time. be able to.
In addition, the operator 20 can instruct each part in the PC 16 that the call is ended by clicking the call end button 58 in the process of step S17 to step S20 described above.
Further, the operator 20 designates a category recorded in the call log management information storage unit 44 in FIG. 3 in relation to the reception log by operating the category selection box 80 in the process of the above-described steps S17 to S20. can do. The designated category is stored in the call log management information storage unit 44 at the end of step S22 described later. Since the category is usually not known before the call is started, the operation is often input immediately before the end of the call and at the time of listening. The default category is stored unless otherwise specified.

  Further, in the process of steps S17 to S20, the operator 20 clicks the interruption button 54 in FIG. 9 to give a temporary interruption instruction to the recording unit 32, the operator voice recognition unit 33, and the user voice recognition unit 34. Giving, recording of call contents, voice recognition, and text display processing can be suspended. Further, in this temporarily interrupted state, by clicking the restart button 56 in FIG. 10, a restart instruction can be given and these processes can be restarted. The operator voice recognition unit 33 and the user voice recognition unit 34 recognize that a break of the operator voice is detected between the time before the temporary interruption and the time after the restart when the temporary interruption and resumption operations are performed.

  This suspension and resumption is used, for example, when a conversation that does not need to be recorded during a call with a user continues for a long time. In addition, the temporary interruption and resumption are performed when the operator voice recognition unit 33 or the user voice recognition unit 34 erroneously detects an utterance break and the error in the break detection is assumed to continue in a chain. It is also used to cause the speech recognition unit 33 and the user speech recognition unit 34 to forcibly detect the utterance break, thereby setting the utterance break at a desired timing.

  After step S20, the input unit 37 determines whether or not the call end button 58 is clicked by the operator 20 (step S21). If the call end button 58 has not been clicked since the call start button 52 was last clicked, that is, if the call is still in progress, the processing of the PC 16 returns to step S17.

  When the call end button 58 is clicked, that is, when the operator 20 gives an instruction to end the call, the recording unit 32, the operator voice recognition unit 33, and the user voice recognition unit 34 start the call started in step S16. The content recording and the voice recognition are finished (step S22). The text obtained as a result of the voice recognition, that is, the response log, and the recording data of the operator voice and the user voice are associated with each other and stored in the call log management information storage unit 44 by the call management unit 38 as a set. In parallel with the voice recognition and result display repetition processing from step S17 to step 21, the recording of the voice started in step S16 is the end of the recording, except for the period temporarily suspended, until step S22. Executed. When the recording unit 32, the operator voice recognition unit 33, and the user voice recognition unit 34 are all temporarily interrupted, the voices stored in the operator voice recognition result storage unit 42 and the user voice recognition result storage unit 43 are temporarily interrupted. Data will not advance relative time during suspension. Further, at the time of temporary interruption, the recording unit 32 may continue recording, and the operator voice recognition unit 33 and the user voice recognition unit 34 may be temporarily interrupted. In this case, the operator voice recognition result storage unit 42 and the user voice may be used. The audio data stored in the recognition unit 43 advances the relative time even during a temporary interruption.

  Next, the reception log processing unit 35 reads the contents of the call recorded in the audio file format including the stereo audio, and obtains the waveforms of the operator audio and the user audio, respectively. The obtained waveforms are respectively displayed in the operator column 94a and the user column 94b of the waveform display unit 94 via the output unit 36 (step S23). The display proceeds from the left to the right of the waveform display unit 94 with the time, but if the call is long and does not fit in the waveform display unit 94, only a part of it is displayed. The operator 20 can display a waveform in a desired time zone by operating the scroll bar 94d of the waveform display unit 94.

Thereafter, the response log processing unit 35 performs a wake-up support process according to the operation of the operator 20 (step S24).
The operator 20 inputs a work log related to a call using a system or program not included in the operator work support tool 39 while checking the response log display unit 92. This business log includes the personal information of the user who is the other party of the call, the inquiry content from the user, the response content of the operator 20 in response thereto. At this time, there is a case where it is necessary to confirm the voice during the call, that is, the voice at the time of the telephone call, but what is executed in accordance with this work is the voice assist process in step S24. This is performed in cooperation with the response log processing unit 35, the output unit 36, and the input unit 37.
In the response log screen 50, a call category can be selected at an arbitrary timing and saved by a save button.

In the following, the contents of the awakening support process will be described.
First, the operator 20 switches the switch 14a to the listening side.
When the operator 20 wants to confirm the utterance content of the operator 20 with real voice, the operator 20 first operates the scroll bar 92d to display a desired portion of the response log. Next, the corresponding cell (the operator column 92c in the line where the corresponding text is displayed) in the reception log display unit 92 is clicked. As shown in FIG. 8, the output unit 36 displays a color corresponding to the background of the clicked cell in the response log display unit 92 and displays text corresponding to the voice log editing window 93, and a waveform display unit 94. The background of the speech waveform corresponding to the text is also colored. Note that the colored portion is indicated by diagonal lines in FIG. When the reproduction button 62 is clicked in this state, the output unit 36 reproduces the sound of the corresponding part and provides it to the operator 20. Thus, the operator 20 can easily hear the operator voice.
Note that the operator 20 can also specify a plurality of continuous cells. When the operator 20 selects a plurality of continuous cells in the operator column 92c of the response log display unit 92, the output unit 36 displays the background of the corresponding voice waveform in the operator column 94b of the waveform display unit 94 in a colored manner. The text corresponding to the first cell is displayed in the editing window 93. When the operator 20 selects a plurality of continuous cells in the user column 92b of the response log display unit 92, the output unit 36 displays the background of the corresponding audio waveform in the user column 94a of the waveform display unit 94 in a colored manner. The text corresponding to the first cell is displayed in the editing window 93. When the operator 20 selects a plurality of continuous cells in the user column 92b and the operator column 92c of the response log display unit 92, the output unit 36 displays the corresponding voice waveform in the user column 94a and the operator column 94b of the waveform display unit 94. The background is displayed in color, and the text corresponding to the first cell is displayed in the voice log editing window 93. When the reproduction button 62 is clicked in this state, the output unit 36 reproduces the sound of the corresponding part and provides it to the operator 20.

  In this way, since each text data string has its start time and end time linked to the voice, one text data is designated and the recorded voice corresponding to that part is cued in one shot. It has a function to play.

At this time, the operator 20 designates the right end or the left end of the portion where the background of the audio waveform is colored with the mouse pointer and moves it by a drag and drop operation, thereby reproducing the portion where the background is displayed in color. The range of the part can be changed.
Similarly, the operator 20 indicates the right end or the left end of the user column 92b and the operator column 92c of the response log display unit 92 with a mouse pointer and moves them by drag and drop operation, thereby changing the respective display widths, One can be hidden.

  During playback of the operator voice, the operator 20 can pause playback by clicking on the pause button 64. The pause is canceled by clicking the pause button 64 again, and the sound reproduction is resumed. The operator 20 can stop the reproduction by clicking the stop button 66.

Further, the operator 20 can correct the content of the text displayed as a result of the reception log, that is, the voice recognition. When this correction is performed, first, the operator 20 displays the cell and the background of the corresponding waveform portion in a colored manner as described above, and displays the text corresponding to the voice log editing window 93. The voice log editing window 93 is a text editing area for editing text data.
In the voice log editing window 93, the text of the cell whose background is colored in the response log display section 92 can be edited. The operator 20 performs editing such as correction of text via an input device such as a keyboard, and clicks the confirmation button after the completion. When the confirmation button is clicked, the contents of the edited text are reflected in the corresponding cell of the response log display unit 92 and the operator voice recognition result storage unit 42.

Similarly, when the user's utterance content is to be confirmed with real voice, the corresponding cell (the user column 92b in the row displaying the corresponding time) is similarly clicked. The output unit 36 colors the background of the clicked cell and also displays the background of the corresponding waveform of the voice waveform. When the playback button 62 is clicked in this state, the output unit 36 plays back the corresponding part, that is, the part from the end time of the immediately preceding operator voice to the start time of the next operator voice. As a result, the operator 20 can easily awaken the user voice portion sandwiched between the front and rear operator voices.
In the configuration diagram of FIG. 3, even when the user speech recognition unit 34 is not included, the user's utterance content can be confirmed with real speech. In this case, the user column 92b of the response log display unit 92 is displayed blank, but the user voice waveform is displayed in the user column 94b of the waveform display unit 94 as described above.

  When the operator 20 wants to confirm the content of the user's utterance with real voice, the operator 20 clicks the cell to be reproduced in the adjacent user column 92b with the operator analysis result displayed in the operator column 92c as a mark. The output unit 36 displays a color (but blank) corresponding to the voice log editing window 93 in a colored display on the background of the clicked cell in the response log display unit 92, and the waveform display unit 94 displays the text. The background of the speech waveform corresponding to the text is also colored. When the reproduction button 62 is clicked in this state, the output unit 36 reproduces the user's voice of the corresponding part and provides it to the operator 20. Further, when the operator 20 inputs text in the voice log editing window 93 that is blank and clicks the confirm button, the contents of the input text are recognized in the corresponding cell of the response log display unit 92 and user voice recognition. This is reflected in the result storage unit 43.

  In addition, "I would like to receive your phone number", "Please give me the model number", "I need your name", "I will repeat your name", etc. By selecting a character string (immediate reference keyword) indicating that important information appears, the immediately following voice may be reproduced. In this case, the response log processing unit 35 acquires the end time of the sentence including the selected reference keyword immediately after, and the output unit 36 corresponds to the corresponding part of the waveform display unit 94 for the operator voice or user voice immediately after this. The background of the audio waveform is colored. When the reproduction button 62 is clicked in this state, the output unit 36 reproduces the sound of the corresponding part and provides it to the operator 20.

Also, in the arousal support process in step S24, the operator 20 can change the waveform display format by clicking the enlarge button 68, the reduce button 70, and the full button 72, respectively. When the enlarge button 68 is clicked, the output unit 36 enlarges the displayed waveform in the horizontal direction. That is, a shorter time waveform change is displayed in more detail. When the reduction button 70 is clicked, the output unit 36 conversely reduces the displayed waveform in the horizontal direction. That is, a longer waveform change is displayed. When the full button 72 is clicked, the output unit 36 displays all waveforms from the start to the end of the call being processed on the screen.
Further, in this audible support process of step S24, the operator 20 can change the category by the category selection box 80 when the content of the queried inquiry does not match the currently set category.

The above is the flow of processing when a call is made immediately after a call from an external user is terminated, but the operator work support system 100 displays the call contents recorded in the past according to the operation of the operator 20. Talking support can also be provided.
When a call recorded in the past is to be awakened, the operator 20 clicks the list display button 74 on the response log screen 50 after setting the switch 14a to the awakening side in the same manner as the above-mentioned awakening.

When the list display button 74 is clicked, the output unit 36 displays the response log list screen shown in FIG. 13 on the display. On the response log list screen, for each call recorded in the past, a recording start time, a recording end time, a call time, a corresponding operator name, which are a part of the contents of the call log management information storage unit shown in FIG. The category of the call designated by the category selection box 80 is displayed. The operator 20 selects a call to be a subject of the awakening work, displays the background in color, and clicks the OK button. When the OK button is clicked, the output unit 36 deletes the response log list screen, reads the record of the selected call content, displays it on the response log screen 50, and assists in the wake-up in the same manner as in step S24 described above. Process.
Alternatively, the operator 20 can cancel the selection and return to the response log screen 50 by clicking a cancel button.

Note that the operator 20 can make settings related to the operator 20 at any time regardless of the flow of the above processing except during the recording of the call content. This setting is started by clicking the setting button 76 on the reception log screen 50.
When the setting button 76 is clicked, the output unit 36 displays the setting screen shown in FIG. 11 on the display. On this setting screen, operator information stored in the call log management information storage unit 44, that is, an operator name that is an operator identification name for the operator work support tool 39, and the gender of the operator are displayed.

For this setting screen, select the line corresponding to the operator you want to edit via an input device such as a keyboard and mouse, and click the edit button. Then, edit the operator name field display or click the gender selection button. The operator 20 can change the setting by performing an input such as clicking. A new operator can be added by clicking the new button.
When the setting change is completed, the operator 20 clicks an OK button on the setting screen shown in FIG. As a result, the call management unit 38 reflects the input change content and performs the subsequent execution of the operator work support tool 39 using the changed content.
Alternatively, the operator 20 can cancel the selection and return to the response log screen 50 by clicking a cancel button.
When a dictionary editing button is clicked on this setting screen, a dictionary setting screen shown in FIG. 12 is newly displayed. On this dictionary setting screen, by selecting a desired dictionary and clicking the OK button, the speech recognition dictionary stored in the business dictionary storage unit 40 to be used for speech recognition can be selected. Also, by selecting a speech recognition dictionary and clicking an edit button, it is possible to edit the registered contents of the corresponding dictionary.
Alternatively, the operator 20 can cancel the selection and return to the setting screen of FIG. 11 by clicking the cancel button.

The operator and the voice recognition dictionary set in the setting screen in FIG. 11 and the dictionary setting screen in FIG. 12 by clicking the setting button 76 on the response log screen in FIG. That is, when voice recognition is started by the call start button 52, the operator voice recognition unit 33 acquires the set gender of the operator from the operator information in the call log management information storage unit 44, and an acoustic dictionary that matches the corresponding gender is obtained. Voice recognition is performed using the set voice recognition dictionary.
The user voice recognition unit 34 performs voice recognition using a male-female acoustic dictionary and using the set voice recognition dictionary. When the end button 98 is clicked to end, the set operator and voice recognition dictionary are stored in the call log management information of the call log management information storage unit 44.

As described above, according to the program for causing the computer to function as the operator work support system according to the present invention, when displaying the text of the call content, the keyword specified in advance is highlighted, so that the operator 20 can perform the call. Important parts can be recognized in a shorter time, and work efficiency can be improved.
After extracting the keywords to be highlighted at the same time by performing voice recognition in real time, it is possible to highlight another keyword with respect to the voice recognition result. Further, at the time of voice recognition, even if a keyword cannot be detected for a word in a recognition unit, it is possible to detect a keyword for a sentence unrelated to the recognition unit after voice recognition.
Since unnecessary words can be made inconspicuous or deleted, the speech recognition result can be confirmed efficiently in a limited screen.
By assigning operator voice and user voice to stereo recording, time consistency between voice data and recognition result can be obtained.
Since the time consistency between the voice data and the recognition result is obtained, the voice immediately after the predetermined keyword can be reproduced.
Since the time consistency between the speech data and the recognition result is obtained, the speech recognition result and the speech waveform can be synchronized and displayed on the screen. Even if the user's voice is not recognized, the operator's voice recognition result, and the operator voice and the user voice are time-consistent, so the user voice that exists between the operator voices can be reproduced, The speech recognition result can be corrected.
Since the display width of the operator recognition result or the user recognition result can be changed or deleted, easy-to-see display can be performed in a limited screen.

  Also, since operator voice and user voice are separately recorded as separate channel data, each voice can be evoked independently, and it is possible to avoid a situation in which utterances overlap and are difficult to wake up. . For this reason, the working efficiency of the operator 20 can be improved.

  Furthermore, since the content uttered by the operator 20 is recognized as voice and automatically converted into text, the operator 20 does not need to input text and the work efficiency can be improved. Even if the result of voice recognition is wrong, you can select the relevant part while looking at the text, and you can easily wake up and check only that part, so you can play the whole voice of the call There is no need to search for a corresponding voice part, and work efficiency can be improved.

In addition, since the operator 20 can suppress the recording of the unnecessary part of the call by the interruption button 54 and the resume button 56, it is possible to save the trouble of processing the unnecessary part in the listening work and improve the work efficiency. it can.
Furthermore, since the break button 54 and the resume button 56 can forcibly recognize and set the utterance break in real time during a call, it is possible to record an accurate voice recognition result and to improve the efficiency of the listening work. Can be improved.
In addition, since the call log management information storage unit 44 stores the correspondence between the operator name and the gender, if the operator name is specified at the time of voice recognition, the voice log management information storage unit 44 uses a sound dictionary that matches the corresponding gender. Recognition can be performed.

  Further, according to the program for causing the computer to function as the operator operation support system 100 according to the present invention, the operator 20 can wake up a single headset 18 for a call by switching with the switch 14a of the switch box 14. Can be used in common, so that it is not necessary to mount the headset again for each work and use it separately, and the work efficiency of the operator 20 can be improved.

In the first embodiment, the following functions may be further executed.
For example, the output unit 36 may have a function of recognizing an unimportant character string as an unnecessary word and automatically deleting the unnecessary word from the display. This unnecessary word is registered in the business-use dictionary storage unit 40 or the keyword dictionary storage unit 45, and is determined by collating with the speech-recognized text in the same manner as in step S18 or step S19. This collation is performed by the output unit 36 via any one of the response log processing unit 35, the operator voice recognition unit 33, and the user voice recognition unit 34. Examples of unnecessary words include “yes” and “good morning”. Further, the operator 20 may be able to change at any time whether or not to automatically delete unnecessary words. Further, unnecessary words may not be deleted, but may be displayed in a display format different from other text, for example, in a light color.
As a result, unimportant keywords are not displayed on the screen or can be easily discriminated instantaneously, so that it becomes easy for the operator 20 to recognize and understand important parts and improve work efficiency. Can do.

Furthermore, in the displayed text, when a highlighted keyword portion is clicked, information associated with the keyword, for example, a function for popping up a manual or the like, and a sound immediately after the keyword are reproduced. It may have a function. This information is specified in advance in association with each keyword and stored in the business dictionary storage unit 40 or the keyword dictionary storage unit 45. Thereby, the labor for the operator 20 to acquire the information regarding a keyword can be saved, and work efficiency can be improved.
The operator voice text display and keyword highlight display are performed in real time as described above. For this reason, for example, the operator 20 can utter a product name by a call and then click the corresponding product name highlighted as a keyword immediately thereafter to display the manual and refer to the manual while making a call. it can.

  In addition, in response to a click on the keyword portion, the past call records may have a function of listing those including the keyword. This list-up is performed, for example, in the same format as the response log list screen in FIG. 13. By selecting one call from the list, the response log display screen in FIG. 8 is newly created and displayed, and the contents are confirmed. It can be performed. By doing so, work over a plurality of related call contents is facilitated, and work efficiency can be improved.

Also, some of the keywords may be distinguished from other keywords as immediate reference keywords. Immediately after the reference keyword, for example, "I would like to receive your phone number", "Please give me the model number", "Please give me your name", etc. Is a character string indicating Immediately after this, when the reference keyword is clicked, the reproduction start point is determined based on the utterance break corresponding to the end of the text of the cell, and the user voice after that point is reproduced. This reproduction start point is, for example, an utterance break itself.
By doing so, the operator 20 can confirm the user voice including important information based on the fixed character string highlighted so as to be easy to find, so that it is not necessary to reread the response log display unit 92 over the entire sentence. , Work efficiency can be improved. Furthermore, even if the converted text data is ambiguous, it is possible to easily awaken before and after that, so that the contents of the response can be accurately grasped.

Further, the keyword may be displayed not only in the response log display unit 92 but also in a list on another screen in addition to the highlighted display. Further, the keyword displayed in a list may be associated with the reception log display unit 92. For example, a keyword list display screen is displayed on the display, and keywords appearing in the call are listed in that order, and the operator 20 clicks each keyword to display the corresponding cell and waveform in the response log display unit 92. The background of the corresponding portion in the portion 94 may be colored and displayed, and the sound of the corresponding waveform portion may be automatically reproduced. Keywords can be easily registered, changed, and deleted by the operator.
By doing so, the part including the information required by the operator can be confirmed instantaneously, so that the operator does not need to reread the response log display unit 92 over the entire sentence, and the work efficiency can be improved.

It is a figure which shows the structure containing the operator business assistance system 100 which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the switch box 14 concerning Embodiment 1 of this invention. 1 is a diagram illustrating a configuration of an operator business support system 100. FIG. It is a figure which shows the data format stored in the voice memory | storage part 41, the operator voice recognition result memory | storage part 42, and the user voice recognition result memory | storage part 43. FIG. It is a figure which shows the data format of the call log management information stored in the call log management information storage part 44. It is a figure which shows the data format of the operator information stored in the call log management information storage part. It is a figure which shows the data format of the operator information stored in the operator voice recognition result memory | storage part. In Embodiment 1 of this invention, it is a figure which shows the reception log screen 50 displayed when it is not in a telephone call. In Embodiment 1 of this invention, it is a figure which shows the reception log screen 50 displayed when a telephone call is in progress and voice recognition is being performed. In Embodiment 1 of this invention, it is a figure which shows the reception log screen 50 displayed when it is in a telephone call and speech recognition is interrupted. It is a figure which shows the setting screen displayed in Embodiment 1 of this invention. It is a figure which shows the dictionary setting screen displayed in Embodiment 1 of this invention. It is a figure which shows the reception log list screen displayed in Embodiment 1 of this invention. It is a flowchart which shows the flow of a process of the operator business support tool which concerns on Embodiment 1 of this invention.

Explanation of symbols

  12 telephone, 20 operator, 33 voice recognition unit (operator voice recognition unit), 34 voice recognition unit (user voice recognition unit), 35 reception log processing unit, 36 output unit, 37 input unit, 100 operator work support system.

Claims (5)

A program for functioning a computer as an operator work support system that records a call through a telephone and supports an operator work associated with the call,
The computer,
A voice recognition unit that converts voice data of the call into text data;
A first dictionary including a voice recognition dictionary and a keyword designated in advance for each word used for voice recognition;
A second dictionary containing pre-specified keywords;
A response log processing unit for processing the text data;
An output unit for displaying the processed text data ;
Function as an input unit for receiving the operation of the operator ,
The keyword included in the second dictionary is changeable,
The voice recognition unit converts the voice data into the text data,
When the voice recognition unit converts the voice data into the text data, the keyword included in the first dictionary is marked,
In the text data that has been subjected to the marking, the response log processing unit further performs marking for keywords included in the second dictionary,
The output unit state, and are not to highlight the marked keyword,
The first dictionary further includes an explanatory text relating to the keyword,
The second dictionary further includes an explanatory text relating to the keyword,
The output unit further includes:
When the input unit accepts an operation for designating a keyword, an explanation regarding the keyword is acquired and displayed from either the first dictionary or the second dictionary;
In a different area from the display of the text data, the keywords are listed in the order in which they appear in a call,
When the input unit accepts an operation for designating the keyword displayed in the list, the text data including the designated keyword is displayed in color, for causing the computer to function as an operator business support system. program.   The program for causing the computer to function as an operator business support system according to claim 1, wherein the output unit highlights the marked keyword by enclosing it in black brackets. Before Symbol output unit further wherein the input unit accepts an operation for designating a keyword, as an operator business support system according to claim 1 wherein in which to reproduce the audio data corresponding to the text data including the keyword A program that allows a computer to function. The computer, to further
The recognition result storage unit for storing the text data in a storage unit included in the previous SL computer
And then allowed to function,
The output unit further displays the text data in a text editing area,
3. The operator service according to claim 1, wherein the recognition result storage unit stores the corrected text data in a storage unit of the computer when the input unit receives an operation of correcting the text data. A program that causes a computer to function as a support system. When the input unit further receives an operation for designating the keyword, the output unit displays a list of stored calls including the designated keyword.
A program for causing a computer to function as the operator business support system according to claim 1.

Images