JP5685702B2 - Speech recognition result management apparatus and speech recognition result display method - Google Patents

Description

  The present invention relates to a speech recognition result management apparatus that manages a result of speech recognition processing on speech data of a conversation, and a speech recognition result display method that displays the result of the speech recognition processing.

  Conventionally, in a call center, a contact center, or the like, recording of a voice call between a customer and an operator is widely performed. By making the call voice reproducible, the operator itself, the supervisor, the center manager, etc. (hereinafter referred to as “manager”) can later confirm the contents of the call to improve the service.

  However, it is difficult for an operator or an administrator to find a necessary call voice by using only the recorded call voice, and it is not possible to easily listen back. Therefore, for example, as described in Non-Patent Document 1, speech recognition processing is performed on speech, and a character string that is a speech recognition result and a speech section that is a time range in which the speech recognition result is obtained are included. There has been proposed an apparatus that displays a speech waveform in a sequence of utterance timings while being distinguished for each speaker. This apparatus displays a speech segment and a speech waveform along a right-facing time axis, and displays a character string for each speech segment under the speech waveform display. The operator's utterance or the customer's utterance is distinguished by displaying it as speaker information on the left side of the character string. By using such a device, it becomes possible for an operator or administrator to easily find and analyze the required call voice by visually observing the voice waveform, speech segment, character string of the voice recognition result, full text search, etc. , Service can be improved.

Masaaki Nakamura, “Meeting Support System for Minutes of Meetings”, Automatic Recognition, Nippon Kogyo Publishing, October 2004, Vol. 17, No. 12, p. 38-43

  However, the device described in Non-Patent Document 1 displays the character strings of each utterance section of the operator and the customer in a mixed order in the utterance start time, and further displays each character string apart from the utterance section. To do. Therefore, in the apparatus described in Non-Patent Document 1, it is difficult for the administrator to associate the character string with the utterance section, and it is necessary to move the line of sight, which is inconvenient for confirming the content of the call later.

  In order to solve this problem, it is conceivable to arrange character strings above and below the waveform display of each utterance section. However, in this case, if you try to increase the time width of the range that is displayed on the screen at one time (hereinafter referred to as “display time width”), the number of character strings that should be displayed at one time increases, and the character strings are The problem of not being able to fit occurs.

  An object of the present invention is to provide a voice recognition result management apparatus and a voice recognition result display method capable of easily confirming the contents of a call.

A speech recognition result management apparatus according to an aspect of the present invention is a speech recognition result management apparatus that manages a result of speech recognition processing on speech data of a conversation, and a character string that is a result of the speech recognition processing and the speech recognition A screen generation unit that generates a conversation display screen that distinguishes each utterance section that is a time range in which results are obtained for each speaker and displays them in the order of utterance timings, and the conversation display screen generated by the screen generation unit The conversation display screen associates the utterance section and the character string with each speaker, and displays the utterance section in the time axis direction and the character string arrangement direction. Display without matching.

A speech recognition result display method according to an aspect of the present invention is a speech recognition result display method for displaying a result of speech recognition processing on speech data of a conversation, the character string being the result of the speech recognition processing and the speech recognition A step of generating a conversation display screen for distinguishing each utterance section, which is a time range in which results are obtained, from each speaker and displaying them in the order of utterance timing; and a step of outputting the generated conversation display screen. The conversation display screen displays the utterance interval and the character string for each speaker, and displays the utterance interval display without matching the time axis direction of the utterance interval and the arrangement direction of the character strings.

  According to the present invention, since the utterance section and the character string are displayed without matching their directions, the character string can be easily put on the screen as compared with the case where they are matched, and the contents of the call can be easily confirmed. Make it possible to do.

The system block diagram which shows an example of a structure of the call center system containing the speech recognition result management apparatus which concerns on embodiment of this invention The block diagram which shows the structure of the telephone call recording and management apparatus containing the speech recognition result management apparatus which concerns on this Embodiment The figure which shows an example of a structure of the conversation display screen in this Embodiment The figure which shows partially the structure of the conversation display screen in the modification 1 of this Embodiment. The figure which shows the structure of the conversation display screen in the modification 2 of this Embodiment. The figure which shows the structure of the conversation display screen in the modification 3 of this Embodiment. The figure which shows partially the structure of the conversation display screen in the modification 4 of this Embodiment. The figure which shows partially the structure of the conversation display screen in the modification 5 of this Embodiment. The figure which shows the structure of the conversation display screen in the modification 6 of this Embodiment.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram showing an example of a configuration of a call center system including a speech recognition result management apparatus according to an embodiment of the present invention.

  In FIG. 1, a call center system 100 includes a call recording / management apparatus 400 including customer terminals 200-1 to 200-M, operator terminals 300-1 to 300-N, and a speech recognition result management apparatus according to the present embodiment. And a voice recognition server 500.

  Customer terminals 200-1 to 200-M and operator terminals 300-1 to 300-N are connected to each other via a communication network 700 so as to be able to talk. The call recording / management apparatus 400 is disposed between the communication network 700 and the operator terminals 300-1 to 300-N. The call recording / management apparatus 400 and the voice recognition server 500 are connected to be communicable via, for example, a local area network (LAN).

  The customer terminals 200-1 to 200-M are customer-side devices having a call function, such as a fixed telephone, a mobile phone, and a personal computer having an IP telephone function.

  The operator terminals 300-1 to 300-N are devices on the call center side having a call function and an information processing function. For example, the operator terminals 300-1 to 300-N are units of a personal computer having an IP phone function and a call headset.

  The call recording / management apparatus 400 records a call voice between the customer terminal 200 and the operator terminal 300, uses the voice data of the recorded voice as call identification information (hereinafter referred to as "call information"), and speaker identification information. (Hereinafter referred to as “speaker information”) and information indicating the time at each time point (hereinafter referred to as “time information”) are added.

  Further, the call recording / management apparatus 400 uses a voice recognition server 500 to be described later to separately obtain the result of voice recognition for the voice data of the call voice for each speaker, and based on the obtained voice recognition result data. Generate and display a conversation display screen. The conversation display screen is a screen that displays the speech waveform and the speech recognition result for each speaker in order of speech timing. Further, the call recording / management apparatus 400 receives an operation on the conversation display screen from, for example, a call center administrator. The conversation display screen will be described later.

  The voice recognition server 500 is a device that performs voice recognition processing, and is, for example, a personal computer having a LAN function.

  The communication network 700 is the Internet, a public telephone line network, or the like.

  The call center system 100 having such a configuration includes not only the contents of calls between the customer terminals 200-1 to 200-M and the operator terminals 300-1 to 300-N, but also the state of vocabulary of customers and operators (hereinafter referred to as appropriate). It is possible to display a conversation display screen indicating “call status”. Therefore, the administrator can easily confirm the call status of each operator with the customer.

  Next, the configuration of the call recording / management apparatus 400 according to the present invention will be described.

  FIG. 2 is a block diagram showing the configuration of the call recording / management apparatus 400.

  As shown in FIG. 2, the call recording / management apparatus 400 includes a voice data input unit 410, a feature extraction unit 420, a screen generation unit 430, a screen output unit 440, an operation reception unit 450, a voice output unit 460, and a file output unit. 470.

  In the present embodiment, in the voice recognition process, a process for extracting feature amounts from voice data is performed in the call recording / management apparatus 400, and a process for performing voice recognition based on the extracted feature amounts is a voice recognition process. This is performed in the server 500.

  The voice data input unit 410 inputs and stores voice data from the customer terminal 200 via the communication network 700, and outputs the stored voice data to the feature extraction unit 420 and the screen generation unit 430. The audio data includes at least audio waveform data indicating a change in sound pressure. The voice data input unit 410 adds call information and speaker information to each voice data.

  The feature extraction unit 420 detects an utterance interval from the audio data input from the audio data input unit 410, extracts a feature amount from the audio data of the utterance interval, and includes call information and speaker information as time-series data of the feature amount. And feature amount data to which time information of the utterance section is added. Here, the utterance section is a section in which the uttered voice is continuously detected without leaving an interval of a predetermined time or more. Then, the feature extraction unit 420 outputs the generated feature amount data to the speech recognition server 500 and instructs the speech recognition server 500 to execute speech recognition processing.

  As a result, in the speech recognition server 500, text data indicating the uttered speech is generated from the feature amount data for each utterance section, using acoustic model data, dictionary data, and language model data prepared in advance. Then, the voice recognition server 500 outputs each text data as a voice recognition result to the call recording / management apparatus 400 with call information, speaker information, and time information of the utterance section added.

  The screen generation unit 430 of the call recording / management apparatus 400 configures a conversation display screen for each call based on the voice data input from the voice data input unit 410 and the text data input from the voice recognition server 500. To generate image data. Then, the screen generation unit 430 stores, in the data storage unit 431, screen data obtained by integrating image data and audio data on the time axis. As described above, the conversation display screen is a screen that displays the speech waveform and the speech recognition result for each speaker in the order of speech timing. More specifically, the conversation display screen is a screen that associates an utterance section and a character string for each speaker and displays the utterance section display with a time axis direction orthogonal to a character string arrangement direction. The screen generation unit 430 generates a conversation display screen based on the screen data in accordance with an operation performed via the operation reception unit 450 and displays the conversation display screen via the screen output unit 440. Further, the screen generation unit 430 outputs a sound via the sound output unit 460 in accordance with an operation performed via the operation reception unit 450 based on the screen data. Details of the conversation display screen will be described later.

  The screen output unit 440 is, for example, a liquid crystal display, and displays a conversation display screen under the control of the screen generation unit 430.

  The operation accepting unit 450 is, for example, a mouse having a keyboard and a vertical wheel, and accepts an operation on the conversation display screen displayed by the screen output unit 440. The contents of this operation will be described later.

  The audio output unit 460 outputs audio under the control of the screen generation unit 430.

  The file output unit 470 performs data file output under the control of the screen generation unit 430.

  Although not shown, the call recording / management apparatus 400 includes a central processing unit (CPU), a storage medium such as a ROM (read only memory) that stores a control program, a working memory such as a random access memory (RAM), and It can be realized by a communication circuit or the like. In this case, the function of each unit described above is realized by the CPU executing the control program.

  Since the call recording / management apparatus 400 having such a configuration displays the utterance section and the character string orthogonally, the character string can be displayed on the screen even if the display time width is widened, and the content of the call can be confirmed. Makes it easy to do. In addition, the call recording / management apparatus 400 also displays the voice waveform separately for each speaker, so that the administrator can analyze in detail the relationship between the conversation situation and the emotional change of the speaker. To do.

  Next, the configuration of the conversation display screen will be described.

  FIG. 3 is a diagram illustrating an example of the configuration of the conversation display screen.

  As shown in FIG. 3, the conversation display screen 800 includes a call information display unit 810, an overall image display unit 820, a call status display unit 830, a display state change unit 840, a toolbar display unit 850, and a search bar 860.

  The call information display unit 810 displays call information including the contents of call information, speaker information, and time information added to the original voice data. Call information includes, for example, speaker, call start date / time, call time, call direction, call attribute, editor, collection edit date / time, version number, comment, evaluation, voice recognition process start date / time, voice recognition process end date / time, voice recognition It includes processing time, voice recognition reliability, number of utterances, voice recognition parameters, public information, and access history.

  According to such a call information display unit 810, the administrator can know which call status of which operator's call is displayed on the conversation display screen 800.

  The overall image display unit 820 displays an overall view 821 of the audio waveform and a display range marker 822. The entire speech waveform FIG. 821 displays the speech waveform of the entire call separately for each speaker along the vertical time axis. In addition, the manager can easily find an important part such as a part where the customer or operator's vocabulary is rough or a silent section. The display range marker 822 indicates the range currently displayed on the call status display unit 830 (hereinafter referred to as “display range”), and the position is set to an arbitrary vertical position (during a call) via the operation reception unit 450. It can be moved to any position).

  The call status display unit 830 defines the voice waveform 831 around the time corresponding to the position of the display range marker 822 of the overall image display unit 820 for each speaker along the time axis 832 displayed in the vertical direction. Separately displayed. That is, the time of the audio waveform 831 is indicated by the position in the vertical direction, and the sound pressure of the audio waveform 831 is indicated by the position in the horizontal direction. Here, the call status display unit 830 displays the customer's voice waveform 831-1 on the left side and the operator's voice waveform 831-2 on the right side.

  In addition, the call status display unit 830 displays the utterance section bar 833 indicating the utterance section from which the voice recognition result is obtained, separately for each speaker along the time axis 832. Here, the call status display unit 830 displays the customer's speech section bar 833-1 on the left side of the center and the operator's speech section bar 833-3 on the right side of the center. Thus, the administrator can easily distinguish each piece of information for each speaker.

  In addition, the call status display unit 830 displays a text box 834 indicating the voice recognition result in horizontal writing characters in association with the corresponding utterance section bar 833 and superimposed on the voice waveform 831. That is, the call status display unit 830 displays the character string of the voice recognition result in the arrangement direction orthogonal to the direction of the time axis 832. Thereby, the administrator can browse the speech waveform and the character string at the same time. In addition, even when the display time width is wide (the scale is small or the scale denominator is large), the width of the screen can be kept constant, making it easier to display the character string on one screen and improving the screen usage efficiency. Can be made. Each text box 834 has a text display column and control icons for operations such as reproduction, copying, restoration before editing, restoration to the initial state, and deletion. The administrator can bring the text box 834 into an active state (to be operated) simply by bringing the mouse over the text box 834, and can immediately specify a character string or operate a control icon. In other words, it is possible to perform an operation on a specific character string or sound data corresponding thereto with one click.

  Each text box 834 may display a control icon only when the pointer is positioned on the text box 834. Each text box 834 normally displays a character string with a small font size, and displays the character string by increasing the font size or changing the color only when the pointer is positioned on the text box 834. You may make it do. As a result, the screen can be simplified and the administrator can easily recognize on which text box 834 the pointer is located.

  In addition, each text box 834 may be displayed by changing the font size of the character string according to the display time width of the call status display unit 830 or overlapping with other text boxes 834 as necessary. good. Thereby, even if the display time width is set to be large by a zoom bar 841, which will be described later, and the number of character strings to be displayed increases, each character string can be prevented from being omitted as much as possible.

  In addition, the call status display unit 830 displays a playback position marker 835 indicating a playback position so as to overlap the voice waveform 831 when voice playback is performed. Thereby, the administrator can easily confirm the voice waveform 831 and the character string corresponding to the reproduced voice.

  Note that the image displayed on the call status display unit 830 can be dragged in the vertical direction.

  The display state changing unit 840 displays a zoom bar 841 and a summary bar 842. The zoom bar 841 is a slide bar for changing the display time width of the call status display unit 830, that is, for changing the scale of the time axis 832 (zooming in and zooming out). The summary bar 842 is a slide bar for designating that the character string displayed in the text box 834 of the call status display unit 830 is replaced with a summarized character string (hereinafter referred to as “summary character string”) and the degree of the summary. It is. The zoom bar 841 and the summary bar 842 both slide in the vertical direction. As a result, the administrator can operate the display state changing unit 840 with an operation feeling that is unified with the overall image display unit 820 and the call status display unit 830.

  The toolbar display unit 850 displays a plurality of control icons for continuously reproducing the entire audio data. More specifically, the toolbar display unit 850 is used for operations such as automatic scroll stop, playback start, playback stop, playback range selection, copy, memo, save, return to pre-edit, and return to the initial state. Displays the control icon for.

  When the playback start control icon is clicked on the conversation display screen 800, the playback of the audio data is started, and each time the playback location reaches the lowermost end of the call status display unit 830, the display of the call status display unit 830 is displayed. Scroll image up one page. In the conversation display screen 800, the display image of the call status display unit 830 may be continuously scrolled upward so that the playback position is fixed at the center of the call status display unit 830.

  In addition, when the copy control icon is clicked, the conversation display screen 800 stores the selected character string as a copy target.

  When the memo control icon is clicked, the conversation display screen 800 accepts input of memos such as supplementary explanations and caution points, and stores the input contents.

Also, the conversation display screen 800 saves the edit result for the character string when the save control icon is clicked.

  In addition, when the control icon for returning to the previous editing state is clicked, the conversation display screen 800 returns the edited character string to the character string before the last editing.

  In addition, when the control icon for returning to the initial state is clicked, the conversation display screen 800 returns the edited character string to the initial state (that is, the voice recognition result).

  The search bar 860 is a text box for searching the input character string from the speech recognition result.

  By displaying such a conversation display screen 800, the administrator can know at what timing the customer or operator has put on hold and at what timing from the change in the characteristics of the voice waveform 831. it can. Further, the administrator can confirm at a glance whether there is any abnormality (sound cracking or noise mixing) in the voice level or the recorded voice from the characteristics of the voice waveform 831. In addition, the administrator can check many character strings at a glance with a wide display time width. Therefore, the administrator can easily check the call status.

  The screen generation unit 430 holds data for configuring such a conversation display screen 800 in advance. The screen generation unit 430 generates a summary character string for each character string displayed in the text box 834 according to a summary rule predetermined for each of a plurality of summary levels, and stores the generated summary character string. . The summary character string is a character string in which the number of characters is reduced from that of the original character string, and is used when display of the summarized character string is instructed in the summary bar 842.

  Hereinafter, the operation of the call recording / management apparatus 400 related to the conversation display screen 800 will be described.

  First, the screen output unit 440 receives a call selection by displaying a call list that lists voice data calls stored in the voice data input unit 410, for example. Then, when any call is selected, screen output unit 440 instructs voice data input unit 410 to output the voice data of the call to feature extraction unit 420. As a result, the voice data input unit 410 outputs the selected voice data to the feature extraction unit 420.

  The call list may be displayed by appropriately grouping, for example, “my call”, “my group call”, “favorite call”, “call commented by the administrator”, and the like. Further, the screen output unit 440 may accept the selection of a call not by a call list but by a condition search (date and time, operator name, telephone number, various classifications, a full text search of a recognition result or an edit result, etc.). .

  The feature extraction unit 420 detects the speech section and extracts the feature amount of the speech section from the input speech data, outputs the feature data to the speech recognition server 500, and returns the speech recognition result. Note that the feature extraction unit 420 and the speech recognition server 500 may perform processing related to speech recognition in advance at the timing of recording a call.

  When the voice recognition result is input, the screen generation unit 430 acquires corresponding voice data from the voice data input unit 410. Then, the screen generation unit 430 generates the above-described screen data from the voice recognition result and the voice data, and displays the conversation display screen 800 via the screen output unit 440. In the initial state, screen generation unit 430 places display range marker 822 of overall image display unit 820 at a position where voice waveform 831 at the start position of the call is displayed. Further, in the initial state, screen generation unit 430 sets the display time width of call status display unit 830 to a relatively small width.

  The screen generation unit 430 changes the display content of the conversation display screen 800 according to the operation from the operation reception unit 450. Specifically, it is as follows.

  When the position of the display range marker 822 is changed, the screen generation unit 430 changes the display range of the call status display unit 830 in response to the change. Thereby, the administrator can easily change the position to be confirmed to an arbitrary position. In addition, since the entire diagram 821 of the voice waveform is displayed, the administrator can easily find a place where the customer or the operator's vocabulary is rough and can cue the important part at the pinpoint. it can.

  When a portion other than the text box 834 in the call status display unit 830 is selected and a vertical drag operation is performed, the screen generation unit 430 scrolls the display range in the vertical direction according to the operation direction and operation amount. Let At this time, the screen generation unit 430 changes the position of the display range marker 822 in response to the scrolling of the display range. As a result, the administrator can continuously check changes in the call status at his / her own pace. Further, since the scroll direction is the vertical direction, it is easy to perform a scroll operation with a normal mouse wheel, and the operation burden on the administrator can be reduced. In this respect, the conventional technique in which the time axis is arranged in the horizontal direction cannot obtain such an effect.

  When the position of the slider of the zoom bar 841 is changed, the screen generation unit 430 changes the display time width of the call status display unit 830 in response to the change. This makes it easy for the administrator to browse according to the purpose, such as viewing the whole or viewing details. Further, the screen generation unit 430 changes the character string displayed in the text box 834 to the corresponding summary character string at the summary level in response to the change in the display time width. Thereby, the character string of the voice recognition result can be shortened in a state where the contents of the conversation can be understood, and the character string and the voice waveform 831 in each text box 834 can be displayed in an easy-to-see manner.

  When the position of the slider of summary bar 842 is changed, screen generation unit 430 changes the character string displayed in text box 834 to the summary character string of the corresponding summary level in response to the change.

  When the control icon in the text box 834 is clicked, the screen generation unit 430 performs a corresponding process. For example, when the reproduction icon in the text box 834 is clicked, the screen generation unit 430 reproduces audio data corresponding to the displayed character string using the audio output unit 460. In addition, when the character string portion of text box 834 is clicked, screen generation unit 430 accepts editing for the character string.

  When the audio reproduction is performed, the screen generation unit 430 displays the reproduction position marker 835 superimposed on the audio waveform 831 and moves the display position corresponding to the reproduction position.

  The screen generation unit 430 performs a corresponding process when the control icon on the toolbar display unit 850 is clicked. For example, when the playback icon of the toolbar display unit 850 is clicked, the screen generation unit 430 uses the audio output unit 460 to reproduce audio data corresponding to the audio waveform 831 being displayed. In addition, when the save icon of the toolbar display unit 850 is clicked, the screen generation unit 430 outputs a part or all of the display contents of the conversation display screen 800 to a file using the file output unit 470.

  When a character string is input to the search bar 860, the screen generation unit 430 searches for the input character string from the speech recognition result, and selects a corresponding portion of the character string displayed in the text box 834 as a color. Highlight by highlighting or highlighting. As a result, the administrator can easily confirm the appearance location and frequency of appearance of the specific character string, and further easily reproduce the location including the specific character string as a pinpoint.

  As described above, the audio output unit 460 reproduces the audio data under the control of the screen generation unit 430. At this time, the audio output unit 460 reproduces the audio data with stereo audio. More specifically, the audio output unit 460 reproduces the customer's audio data with the left audio and the operator's audio data with the right audio. That is, the call recording / management apparatus 400 associates the right and left direction of the call status display with the left and right direction of the voice output for each speaker. This makes it easy for the administrator to distinguish and grasp the voice for each speaker, and to associate it with the display content of the conversation display screen 800. In this regard, in the conventional technology in which information for each speaker is arranged vertically and displayed, normal stereo sound cannot be reproduced by distinguishing the vertical direction, so the reproduction direction and the display direction are associated sensuously. It is difficult.

  As described above, according to the call recording / management apparatus 400 of the present embodiment, since the speech section and the character string are displayed orthogonally, the character string can be stored on the screen even if the display time width is widened. This makes it possible to easily check the content of a call.

  In the present embodiment, the case has been described where the conversation display screen displays information related to customer voice data on the left side and information related to operator voice data on the right side. However, the present invention is not limited to this. For example, on the conversation display screen, customer information and operator information may be arranged in the left-right direction.

  Moreover, in this Embodiment, although the reference | standard of the position of each text box was made into the center side where the speech area bar is displayed, it is not limited to this. For example, the conversation display screen may align the left end of the text box in the vertical direction for each speaker. In this case, the administrator can more accurately grasp the time interval of the utterance timing of each character string from the positional relationship of the left end portion of the text box.

  The conversation display screen may be arranged without dividing customer information and operator information into left and right. In this case, for example, on the conversation display screen, as in FIG. 3, the customer and the operator display the utterance section bars arranged in two columns in the vertical direction at the end of the screen, and the text box is not particularly divided. Arranged in one vertical column, the customer's character string and the operator's character string are displayed at closer positions. In this case, the administrator can check the flow of conversation between the two in a shorter time. However, in this case, the conversation display screen not only displays the text box connected to the corresponding voice bar so that the speaker is distinguished, but also displays the display form such as the color and arrangement of the text box with the customer. It is desirable to vary between operators.

  The conversation display screen may display only customer information or operator information. Furthermore, the conversation display screen may switch between customer information and operator information.

  Further, the time axis direction and the character string arrangement direction on the conversation display screen do not necessarily have to be orthogonal to each other. If the direction of the time axis and the arrangement direction of the character strings do not match, it becomes easier to display the character strings on one screen than the matching conventional technology, and the use efficiency of the screen can be improved. Further, the time axis does not necessarily have to be the vertical direction.

  The conversation display screen may change the display state (for example, color and size) of the character string according to the score (reliability) of the voice recognition result. Thereby, the administrator can perform a call analysis in consideration of the reliability of the voice recognition result. In this case, the call recording / management apparatus 400 needs to obtain, for example, the scores of the various parts of the voice recognition result from the voice recognition server 500.

  The conversation display screen may highlight at least one of the character string of the complaint part, the text box, and the utterance section bar. As a result, the administrator can quickly find the location of the complaint. In this case, for example, the call recording / management apparatus 400 needs to extract a complaint part by searching for a phrase unique to the complaint.

  Further, the conversation display screen may highlight at least one of a character string, a text box, and an utterance section bar where a so-called talk fogging occurs, where the operator starts talking while blocking the customer's story. In this case, for example, the call recording / management apparatus 400 needs to extract a location where the operator's speech section starts in the middle of the customer's speech section.

  In addition, the call recording / management apparatus 400 may analyze the text information of the speech recognition result and classify the utterance contents into utterance contents types in time series. For example, the call recording / management apparatus 400 classifies each utterance section into a classification such as “greeting”, “question”, “answer”, “complaint”, “request”, and the like, and a tag indicating the classification is set as the utterance section. It is added and displayed on the conversation display screen in an easy-to-understand manner. Thereby, the administrator can quickly determine the type of each utterance. In this case, for example, the call recording / management apparatus 400 searches a word that appears in each utterance section from a list of phrases created for each type of utterance content, and acquires the type of list with the largest number of hits, etc. It is necessary to determine the type of each utterance section.

  Further, the call recording / management apparatus 400 may copy and save the display contents and screen image of the conversation display screen as separate files or print them. As a result, the range of utilization of the speech recognition result is widened, and the administrator can further deepen the analysis of the call. In this case, it is desirable that an object to be copied or printed can be arbitrarily designated such as only a character string, only a predetermined time range, and only a specific speaker.

  That is, the mode of the conversation display screen is not limited to the mode shown in FIG. Hereinafter, as a modification of the present embodiment, an example of another aspect of the conversation display screen will be described.

(Modification 1)
FIG. 4 is a diagram partially showing the configuration of the conversation display screen in Modification 1 of the present embodiment.

  As shown in FIG. 4, the conversation display screen 800a according to the first modification example has a plurality of scale lines 836a extending in the vertical direction arranged in the horizontal direction on the call status display unit 830a with reference to the sound pressure 0 of the voice waveform 831. indicate.

  This makes it easier for the administrator to grasp the change in the sound volume. In addition, since it becomes easy for the administrator to quantitatively grasp the sound volume, it is possible to easily perform quantitative extraction of “parts where the voice is too loud” and “parts where the voice is too loud”. For example, the administrator can easily pick up a portion where the volume exceeds a predetermined value by searching for a portion where the sound waveform 831 protrudes from the second scale line 836a on both sides from the position of the sound pressure 0. .

(Modification 2)
FIG. 5 is a diagram showing a configuration of a conversation display screen in the second modification of the present embodiment.

  As shown in FIG. 5, the conversation display screen 800b in the second modification displays customer information and operator information in the call status display unit 830b in different forms. Specifically, the conversation display screen 800b displays the customer's voice waveform 831b-1 and the operator's voice waveform 831b-2 in different colors, and the customer's text box 834b-1 and the operator's text box 834b-2. Are displayed in different colors. This also makes it easier for the administrator to distinguish and grasp the information for each speaker.

  In addition, the conversation display screen 800b displays on the toolbar display unit 850 the movement to the first utterance section, the movement to the previous utterance section, the movement to the next utterance section, the movement to the last utterance section, and repeated playback. A control icon for each operation is further displayed. When these control icons are clicked, the conversation display screen 800 moves the reproduction position to the start position of the corresponding utterance section and activates the text box 834b of the corresponding utterance section.

  The conversation display screen 800b may have different line types and fonts between the customer information and the operator information. Further, the conversation display screen 800b may change the display form in the call status display unit 830b depending on the volume level, whether or not the displayed character string includes a specific word or phrase, and the like. For example, the conversation display screen 800b highlights a text box 834b or a character string where the volume exceeds a certain level. As a result, the administrator can easily find a place where the customer or the operator's language is rough.

(Modification 3)
FIG. 6 is a diagram showing a configuration of a conversation display screen in the third modification of the present embodiment.

  As shown in FIG. 6, the conversation display screen 800c according to the third modification displays voice spectrograms 837c-1 and 837c-2 instead of the voice waveform of voice data in the call status display unit 830c.

  The audio spectrogram is a graph in which the power of each frequency region is displayed in a shaded manner so that it can be easily understood visually, and is also called a voiceprint. From the voice spectrogram, it is possible to read voice characteristics that are difficult to appear in the voice waveform. Depending on the training, the administrator reads the voice pitch, the gender of the speaker, phoneme symbols such as consonants and vowels, and the emotions of the speaker from the speech spectrogram, and performs a kind of call analysis that is difficult only with the speech waveform. It becomes possible. Note that the conversation display screen 800c may be switched between the speech waveform 831 and the speech spectrogram 837c. In this case, it is possible to perform call analysis in a multifaceted manner.

(Modification 4)
FIG. 7 is a diagram partially showing a configuration of a conversation display screen in Modification 4 of the present embodiment.

  As illustrated in FIG. 7, the conversation display screen 800d according to the fourth modification uses a text box 834d as the information indicating the utterance section in the call status display unit 830d without using the utterance section bar. Specifically, the conversation display screen 800d displays a text box 834d at a position corresponding to the utterance section on the speech waveform 831. Thereby, simplification of a screen can be achieved. In addition, the administrator can more easily grasp the correspondence between the speech waveform 831 and the character string.

(Modification 5)
FIG. 8 is a diagram partially showing a configuration of a conversation display screen in Modification 5 of the present embodiment.

  As shown in FIG. 8, in the conversation display screen 800 e in the fifth modification, in the call status display unit 830 e, the utterance section bar 833 e is arranged not on the center of the call status display unit 830 but on the voice waveform 831. Thereby, the area used for displaying the utterance section bar 833e can be allocated to other information display, and the screen can be used effectively. In addition, the administrator can more easily grasp the correspondence between the utterance section and the speech waveform 831.

  Furthermore, as shown in FIG. 8, the conversation display screen 800e may display a text box 834d at a corresponding position on the speech waveform 831 as in the fourth modification. The administrator can more easily grasp the correspondence relationship between the speech waveform 831, the utterance section, and the character string.

(Modification 6)
FIG. 9 is a diagram showing a configuration of a conversation display screen in the sixth modification of the present embodiment.

  As shown in FIG. 9, the conversation display screen 800f according to the modified example 6 displays the overall image display unit 820f at the center of the call status display unit 830f. Then, the conversation display screen 800f displays the entire waveform waveform 821 in the up-down direction and displays the entire image display unit 820f in association with the display range marker 822f of the entire image display unit 820f.

  Thereby, information and an operation target can be arranged collectively in the center of the screen, and the visibility and operability of the screen can be improved. In addition, the administrator can easily intuitively grasp the position of the display range of the overall image display unit 820f in the entire call.

  The voice recognition result management apparatus and the voice recognition result display method according to the present invention are useful as a voice recognition result management apparatus and a voice recognition result display method that can easily check the content of a call.

DESCRIPTION OF SYMBOLS 100 Call center system 200 Customer terminal 300 Operator terminal 400 Call recording and management apparatus 410 Voice data input part 420 Feature extraction part 430 Screen generation part 431 Data storage part 440 Screen output part 450 Operation reception part 460 Voice output part 470 File output part 500 Voice Recognition server 700 Communication network 800, 800a to 800f Conversation display screen 810 Call information display unit 820, 820f Overall image display unit 821, 821f Overall view of voice waveform 822, 822f Display range marker 830, 830a, 830b, 830c, 830d, 830e , 830f Call status display part 831, 831b Voice waveform 832 Time axis 833, 833e Speaking section bar 834, 834b, 834d Text box 835 Playback position marker 836a Scale line 83 7c Voice spectrogram 840 Display state change part 841 Zoom bar 842 Summary bar 850 Toolbar display part 860 Search bar

Claims (13)

A speech recognition result management apparatus for managing a result of speech recognition processing for speech data of conversation,
A screen generation unit that generates a conversation display screen that distinguishes each character string that is a result of the voice recognition processing for each speaker and displays them in the order of speech timing;
A screen output unit for outputting the conversation display screen generated by the screen generation unit,
The conversation display screen is
For each speaker, information indicating an utterance interval and information indicating a silent interval other than the utterance interval are displayed along the same time axis, and the character string is information indicating the utterance interval. And display without matching the direction of the time axis and the direction of arrangement of the character string,
Speech recognition result management device. The conversation display screen is
Information indicating the utterance interval and information indicating the silent interval and the character string are displayed with the direction of the time axis and the arrangement direction of the character string orthogonal to each other.
The speech recognition result management apparatus according to claim 1. The direction of the time axis is a vertical direction.
The speech recognition result management apparatus according to claim 1. The conversation display screen is
For each speaker, information indicating the vocabulary state of the speaker is displayed in association with information indicating the utterance interval.
The speech recognition result management apparatus according to claim 1. The information indicating the state of vocabulary includes a voice waveform of the voice data arranged along the time axis.
The speech recognition result management apparatus according to claim 4. The information indicating morale includes information that emphasizes the character string,
The speech recognition result management apparatus according to claim 4. The screen output unit displays the conversation display screen,
An operation receiving unit including a control icon for receiving an editing operation on the character string corresponding to each of the character strings displayed on the conversation display screen;
The speech recognition result management apparatus according to claim 1. The screen output unit displays the conversation display screen,
An operation accepting unit that accepts a vertical scroll operation on the conversation display screen;
The speech recognition result management apparatus according to claim 3. The screen output unit displays the conversation display screen,
An operation accepting unit that accepts a change operation for a time width of a range displayed at a time on the conversation display screen;
The screen generator is
Generating the character string summarized at a summary level corresponding to the time width and displaying the character string on the conversation display screen;
The speech recognition result management apparatus according to claim 1. The screen output unit displays the conversation display screen,
An audio reproduction unit for reproducing the audio data;
An operation accepting unit that accepts designation of a reproduction location of the audio data and start and stop of reproduction;
The conversation display screen shows a playback location during playback of the audio data.
The speech recognition result management apparatus according to claim 1. The screen output unit displays the conversation display screen,
The conversation is a conversation between two speakers,
The conversation display screen displays the voice waveforms of the two speakers divided into left and right,
The audio data of the two speakers in the direction corresponding to the lateral direction of the display of the audio waveform, further comprising sound reproduction unit for reproducing divided into right and left, and
The speech recognition result management apparatus according to claim 3. The conversation display screen is
For each utterance section, information indicating the type of utterance content of the utterance section is displayed in association with the utterance section.
The speech recognition result management apparatus according to claim 1. A speech recognition result display method for displaying a result of speech recognition processing for speech data of a conversation,
Generating a conversation display screen that distinguishes the character string that is the result of the voice recognition processing for each speaker and displays the sequence in the order of utterance timing;
Outputting the generated conversation display screen,
The conversation display screen is
For each speaker, information indicating an utterance interval and information indicating a silent interval other than the utterance interval are displayed along the same time axis, and the character string is information indicating the utterance interval. And display without matching the direction of the time axis and the direction of arrangement of the character string,
Speech recognition result display method.