JP5206553B2 - Browsing system, method, and program - Google Patents

Description

  The present invention relates to a browsing system, method, and program.

  In recent years, use of speech recognition technology has been studied for the purpose of facilitating the creation of minutes, for example, when there are a plurality of speakers such as a conference. After creating the minutes etc. using the voice recognition technology, the user may listen to the voice while displaying the text of the voice recognition result, and discuss the contents of the agenda while confirming the reproduction.

  Patent Document 1 describes an example of a conference system that automatically creates and stores minutes of meetings held by a plurality of people. The conference system of Patent Literature 1 includes a plurality of terminal devices provided with microphones for inputting a voice of a speaker, and a server connected to the plurality of terminal devices via a network and storing information related to the speaker. A speaker specifying means for specifying a speaker and a speaking time based on terminal specifying information transmitted from at least one terminal device, and converting the voice of the specified speaking person into text and speaking the voice text Audio text storage means for storing together, image storage means for storing image files used as network conference materials and image file display times, and minutes associated with audio text, speech time, image files and display times And minutes creation means for creating a file. Further, it is described that the apparatus includes a minutes display means for displaying an audio text according to the utterance time and displaying an image file according to the display time.

  Accordingly, it is possible to realize a conference proceeding similar to the actual conference proceeding and to create a minutes that allows a third party to realistically experience the content of the conference.

  Patent Document 2 describes an example of a speech recognition system that recognizes speech recorded in a special situation such as a trial or a meeting and creates a document. The speech recognition system of Patent Document 2 specifies a speech section for the speech data acquired by the microphone for acquiring speech data and the speech data acquired by the microphone, and performs speech recognition for each speech section. A speech recognition processing unit for associating character strings and speech data of recognition data for each section with speech time information; a text creation unit for creating text from recognition data for each speech section obtained by processing of the speech recognition processing unit; And an output control unit for displaying and outputting the text created by the text creation unit.

  As a result, even when the text transcribed by voice recognition is corrected, it is possible to make an appropriate association between the corrected text and the original voice, especially in trials and meetings. It is said that it is suitable for the transcription of the sound acquired in special situations.

  Patent Document 3 describes an information processing apparatus that associates information describing conference attributes (conference name, participants, date and place, etc.) with grouped conference audio and images. In this information processing apparatus, the history display matrix MS displays a material link display LS, which is a display representing a material group, and a conference name NM, which is information related to a conference, on a history display screen in chronological order based on the date of the conference. It is described to display.

  Patent Document 4 describes a minutes management system that associates and manages a document that is created or modified as a development project progresses and the minutes of a review meeting of the document. Thereby, it is supposed that the version of the document created or modified with the progress of the development project and the meeting reviewed about the version can be associated with each other and arranged and displayed in time series. In addition, it is described that a document creation history list screen can be displayed indicating that a document is created or modified each time a review meeting is held several times, and that the minutes are linked to the minutes by the document information table. ing.

JP 2007-180828 A JP 2005-165066 A JP 2008-158812 A JP 2001-5874 A

  When discussing the contents of a meeting again while looking at the minutes, there are many cases where the discussion points are limited. In this case, the other minutes are unnecessary. Therefore, when discussing the same point in the minutes many times, if it is not easy to identify or search the location from the entire minutes, it will take time to find the location from the entire minutes of the long meeting. In addition, there is a problem that the same part must be searched many times and time is wasted.

  In the systems described in Patent Documents 1 and 2 described above, it is described that a meeting minutes is created by converting audio into text, but it is repeated for a specific part while partially reproducing the minutes. There is no description of intensive discussions or recording and displaying browsing history referring to the minutes.

  In addition, in the systems described in Patent Documents 3 and 4 described above, a method in which conference materials and conference information are associated with each other is arranged on a schedule in chronological order based on the date of the conference. Although the points to be displayed in the document creation history list along with the date in the order in which the review minutes were discussed are listed, the browsing history cannot be recorded for each series of minutes. It takes time to find the points.

  The object of the present invention is to provide a browsing system, method, and solution for waste of time and labor to search for a specific location every time when reviewing the contents repeatedly while discussing the minutes and the like, which are the above-mentioned problems. And to provide a program.

The browsing system of the present invention performs speech recognition processing on a series of speech sounds, converts speech recognition results into text data, and the obtained speech speech data and the text data are associated with time information for each speech. A speech list display means for displaying the speech recognition results of the speech of the series of speech on a speech list screen in chronological order with reference to the speech storage device and the speech recognition result storage device stored respectively.
Selection accepting means for accepting a choice of a comment from the series of comments displayed on the message list screen;
A selection history recording unit that records the text data of the voice for the utterance selected by the selection receiving unit as a selection history in association with selection identification information that identifies a selection instruction order;
Selection history display means for displaying the recorded text data of the selection history on the selection history list screen side by side based on the selection identification information.

The browsing method of the present invention performs speech recognition processing on the speech of a series of speech, converts the speech recognition result into text data, and the obtained speech speech data and the text data are associated with time information for each speech. The speech recognition results of the voices of the series of utterances are displayed in a chronological order on the speech list screen, with reference to the speech storage device and the speech recognition result storage device respectively stored.
Accepting a selection of comments from the series of messages displayed on the message list screen,
The text data of the voice for the selected utterance is recorded in a selection history recording device as a selection history in association with selection identification information that identifies the selection instruction order,
The recorded text data of the selection history is arranged on the basis of the selection identification information and displayed on the selection history list screen.

The computer program according to the present invention performs speech recognition processing on a series of speech sounds, converts speech recognition results into text data, and the obtained speech speech data and the text data are associated with time information for each speech. A procedure for displaying the speech recognition results of the speech of the series of utterances in a chronological order on the speech list screen with reference to the stored speech storage device and the speech recognition result storage device, respectively.
A procedure for accepting selection of a message from the series of messages displayed on the message list screen;
A procedure for recording the text data of the voice for a selected utterance as a selection history in association with selection identification information for identifying a selection instruction order;
A computer program for causing a computer to execute the procedure of displaying the recorded text data of the selection history on the selection history list screen side by side based on the selection identification information.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  Moreover, although the several procedure is described in order in the method and computer program of this invention, the order of the description does not limit the order which performs a several procedure. For this reason, when implementing the method and computer program of this invention, the order of the several procedure can be changed in the range which does not interfere in content.

  Furthermore, the plurality of procedures of the method and the computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  ADVANTAGE OF THE INVENTION According to this invention, the browsing system, method, and program which can search or specify a desired location easily from a reproduction | regeneration log | history easily are provided.

It is a functional block diagram which shows the structure of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a figure which shows an example of the structure of the memory | storage part of the audio | voice reproduction system which concerns on embodiment of this invention. It is a figure which shows an example of the screen of the audio | voice reproduction system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a figure which shows an example of the screen of the audio | voice reproduction system which concerns on embodiment of this invention. It is a figure which shows an example of the screen of the audio | voice reproduction system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a figure which shows the example of the reproduction history list screen of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a figure which shows an example of the search screen of the audio | voice reproduction system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a figure which shows an example of the structure of the memory | storage part of the audio | voice reproduction system which concerns on embodiment of this invention. It is a figure which shows an example of the reproduction | regeneration list | wrist of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the audio | voice reproduction | regeneration system which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

(First embodiment)
FIG. 1 is a functional block diagram showing a configuration of an audio reproduction system 100 according to the embodiment of the present invention.
The browsing system (sound reproduction system 100) of the present embodiment performs speech recognition processing on a series of speech sounds, converts speech recognition results into text data, and obtains the speech data and the text data obtained for each speech. A speech list display unit that displays speech recognition results of a series of speeches on a speech list screen in time series with reference to the speech storage unit 104 and the speech recognition result storage unit 114 stored in association with each other's time information 120, a selection reception unit (first reproduction instruction reception unit 122) that receives a selection of a message from a series of messages displayed on the message list screen, and a voice for the message selected by the first reproduction instruction reception unit 122 Text data is recorded as selection history (reproduction history) in association with selection identification information (reproduction identification information) that identifies the selection instruction order (reproduction instruction order) The selection history recording unit (playback history recording unit 130) and the text data of the recorded selection history (playback history) are arranged on the basis of the selection identification information (playback identification information) and displayed on the selection history list screen (playback history list screen). A selection history display section (playback history display section 140) to be displayed.

In the browsing system (sound reproduction system 100) of this embodiment, referring to the voice storage unit 104 and the voice recognition result storage unit 114, the time selected from the remarks selected by the selection instruction receiving unit (first playback instruction receiving unit 122) A first reproduction unit 124 is further provided that associates audio data and text data based on the information and reproduces them in synchronization.
In the present embodiment, the audio reproduction system 100 will be described as an example of the browsing system according to the embodiment of the present invention. However, a comment selected from a series of comments displayed on the comment list screen is not necessarily limited. It does not need to be played back, and may be browsed as at least a selection history list screen.

  Furthermore, the audio reproduction system 100 according to the present embodiment includes a second reproduction instruction accepting unit 142 that accepts selection of a message to be reproduced from the reproduction histories displayed on the reproduction history list screen, a voice recognition result storage unit 114, and a reproduction. A second playback unit 144 that refers to the history recording unit 130 and associates the audio data with the text data based on the playback identification information and the time information and plays back in synchronization with the selected message.

  In each of the following drawings, the configuration of parts not related to the essence of the present invention is omitted, and is not shown, for example.

  Each component of the audio reproduction system includes a CPU (Central Processing Unit) of an arbitrary computer, a memory, a program that realizes the components of this figure loaded in the memory, a storage unit such as a hard disk that stores the program, It is realized by any combination of hardware and software, centering on the network connection interface. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Each figure described below shows functional unit blocks, not hardware unit configurations.

  The audio reproduction system 100 is a computer to which an input device such as a keyboard and a mouse and an output device such as a display and a printer can be connected. For example, each of the audio reproduction systems 100 can be executed by a computer that executes processing according to a computer program stored in a recording medium. Each function of the component is realized.

  The computer program according to the present embodiment performs speech recognition processing on a series of speech sounds, converts speech recognition results into text data, and the obtained speech speech data and text data are associated with time information for each speech. With reference to the stored voice storage unit 104 and voice recognition result storage unit 114 respectively, a procedure for displaying the speech recognition results of a series of speeches in a time series on the speech list screen, and a series of sequences displayed on the speech list screen A procedure for accepting the selection of a speech from among the utterances, a procedure for recording audio text data corresponding to the selected utterance as a playback history in association with playback identification information for identifying the playback instruction order, and a recorded playback history To cause the computer to execute a procedure for displaying text data on the playback history list screen side by side based on the playback identification information. It is predicates.

  Further, the computer program of the present embodiment refers to the voice storage unit 104 and the voice recognition result storage unit 114, associates voice data and text data based on time information from the remarks selected in the selection instruction reception procedure, It is described to cause a computer to execute a playback procedure for playback in synchronization.

  Specifically, the audio reproduction system 100 includes a message list display unit 120, a first reproduction instruction reception unit 122, a first reproduction unit 124, a reproduction history recording unit 130, a reproduction history display unit 140, a second A playback instruction receiving unit 142 and a second playback unit 144 are provided.

  The voice reproduction system 100 can be further connected to a voice recognition system 110 having a voice acquisition unit 102, a voice storage unit 104, a voice recognition unit 112, and a voice recognition result storage unit 114. In the voice recognition system 110, the voice acquisition unit 102 acquires a voice of a series of speeches of a speaker input from a voice input unit (not shown) such as a microphone. The voice recognition unit 112 performs voice recognition processing for each utterance associated with the time information, and converts the voice recognition result into text data.

  As shown in FIG. 2, the voice storage unit 104 stores the voice acquired by the voice acquisition unit 102 as voice data for each utterance associated with time information. The speech recognition result storage unit 114 stores the text data converted by the speech recognition unit 112 as a text section for each utterance associated with time information. As a result, voice voice data and text data are stored in the voice storage unit 104 and the voice recognition result storage unit 114 in association with the time information for each utterance.

  In the example of FIG. 2, the speech recognition result storage unit 114 stores, for each time information, a text section serving as text data identification information, speaker identification information, and text data in association with each other.

Returning to FIG. 1, the speech list display unit 120 refers to the speech storage unit 104 and the speech recognition result storage unit 114 and displays the speech recognition results of a series of speeches on the speech list screen in time series. As an example of this screen, the message list screen 150 in FIG. 3A includes a message list 152. The speech list 152 displays a speaker name 154 for distinguishing speakers such as names and names, and text data indicating the speech content 156 that has been subjected to speech recognition processing. In the speech list 152, these pieces of information are displayed in time series for each speech.
In the speech list 152, for example, the sorting order of the speech time can be reversed, displayed for a limited period, or sorted by the speaker.

  Returning to FIG. 1, the first reproduction instruction accepting unit 122 accepts a user's operation to select a message to start reproduction from a series of messages displayed on the message list screen 150 (FIG. 3A). For example, as shown in the message list screen 160 in FIG. 3B, when the user selects and clicks on a message line 162 to be reproduced from the message list 152, the selection is made by color change or reverse display. Is visible. Note that a row 164 indicates a location selected next to the row 162.

  Returning to FIG. 1, the first playback unit 124 refers to the voice storage unit 104 and the voice recognition result storage unit 114, and associates the corresponding voice data and text data based on the time information from the selected utterance, and synchronizes them. And play it. The audio data is output from a speaker (not shown) by an audio output control unit (not shown). In addition, the text data is remarked so that the voice output part can be visually recognized by changing the color of the part of the voice output part in the comment list 152 of the comment list screen 150 described above or by reversing the text. Displayed by the list display unit 120. In FIG. 1, the connection between the first playback unit 124 and the voice storage unit 104, the voice recognition result storage unit 114, and the playback history recording unit 130 is omitted. In addition, although reproduction | regeneration is not specifically limited, It can reproduce | regenerate continuously from a starting position.

  As shown in FIG. 2, the playback history recording unit 130 associates a text section that is identification information of audio text data played back by the first playback unit 124 with a playback ID that identifies the playback instruction order as a playback history. Record. The reproduction ID is, for example, a serial number assigned in the order in which reproduction is instructed or reproduction time information. As described above, the reproduced speech continuously outputs the next utterance, but the reproduction history recording unit 130 discusses only the utterance instructed to be reproduced by the first reproduction instruction accepting unit 122. Is done.

  The playback history display unit 140 displays the recorded playback history on the playback history list screen. FIG. 3C shows an example of the reproduction history list screen 170. For example, on the message list screen 160 of FIG. 3B, when the user first selects and reproduces the line 162, then selects and reproduces the line 164, and then selects and reproduces the line 162 again. Thus, the reproduction history display screen 140 displays the reproduction history list screen 170 in which the messages are sorted in the order of reproduction.

  The second playback instruction receiving unit 142 receives a selection of a message to be played from the playback history displayed on the playback history list screen 170 by a user operation. That is, in the audio reproduction system 100 according to the present embodiment, not only the utterance is selected and reproduced from the utterance list 152 of a series of utterances, but the utterance is selected again from the list on the reproduction history list screen 170 of the utterances once reproduced. Can be played. Since the list on the playback history list screen 170 is usually much smaller than the speech list 152, it is easy to search for the speech. In addition, when replaying repeatedly, the operability is greatly improved by instructing playback from the list on the playback history list screen 170.

  Returning to FIG. 1, the second playback unit 144 refers to the speech recognition result storage unit 114 and the playback history recording unit 130, and associates speech data and text data from the selected utterance based on the playback identification information and time information. Play in sync. This reproduction method and the like are the same as those of the first reproduction unit 124.

  The operation of the sound reproduction system 100 of the present embodiment configured as described above will be described below. FIG. 4 is a flowchart showing an example of the operation of the audio reproduction system 100 according to the embodiment of the present invention. Hereinafter, a description will be given with reference to FIGS.

  In the browsing method according to the embodiment of the present invention, the speech of a series of speeches is subjected to speech recognition processing, the speech recognition result is converted into text data (step S105), and the obtained speech speech data and text data are obtained for each speech. The speech storage unit 104 and the speech recognition result storage unit 114 stored in association with each other's time information are referred to, and the speech recognition results of a series of speeches are displayed on the speech list screen 150 in time series (step S109). ), Receiving a selection of a message from a series of messages displayed on the message list screen 150 (step S111), and associating the voice text data corresponding to the selected message with reproduction identification information for identifying the reproduction instruction order. The reproduction history is recorded in the reproduction history recording unit 130 (step S115), and the text data of the recorded reproduction history is reproduced. Side-by-side to display the playback history list screen 170 on the basis of the information (step S117).

  Furthermore, the browsing method according to the embodiment of the present invention further refers to the voice storage unit 104 and the voice recognition result storage unit 114 and associates voice data and text data based on time information from the selected utterance. The data is reproduced in synchronism (step S113).

In FIG. 4, steps S101 to S107 are performed by the speech recognition system 110. Thereby, it can reproduce | regenerate by synchronizing with the audio | voice data and text data for every meeting and trial which are discussion and analysis object.
Specifically, first, in the speech recognition system 110, the speech acquisition unit 102 acquires the speech A of an example utterance (step S101), the speech recognition unit 112 performs speech recognition processing, and converts the speech recognition result into text data. (Step S103). Then, the obtained voice data and text data are stored in the voice storage unit 104 and the voice recognition result storage unit 114 in association with the time information for each utterance (steps S105 and S107).

  The voice data and the voice recognition result obtained in this way can be recorded in a recording medium or a storage unit such as a hard disk corresponding to the voice storage unit 104 and the voice recognition result storage unit 114 in FIG. Read and processed. The information is stored in a file or folder for each meeting or court hearing, and the audio reproduction system 100 selects and processes a file corresponding to a meeting or trial for which the contents of the proceedings are to be browsed.

  First, the speech list display unit 120 refers to the speech storage unit 104 and the speech recognition result storage unit 114, and displays the speech recognition results of a series of speeches on the speech list screen 150 in time series (step S109). Then, the first reproduction instruction accepting unit 122 accepts a selection of an utterance to be reproduced from a series of utterances displayed on the utterance list screen 150 by a user operation (step S111). In response to the reproduction start instruction from the user, the first reproduction unit 124 refers to the voice storage unit 104 and the voice recognition result storage unit 114, and based on the selected utterance, the first reproduction unit 124 generates voice data and text data based on the time information. In association with each other, reproduction is performed (step S113). As described above, sound is output and the corresponding part of the speech list 152 is displayed in a color-changed manner so that it is possible to visually recognize which speech is being reproduced.

  The message list display unit 120 or the first reproduction unit 124 records the reproduced audio text data in the reproduction history recording unit 130 as a reproduction history in association with the reproduction identification information for identifying the reproduction instruction order (step S115). The reproduction history display unit 140 arranges the reproduction history text data recorded in the reproduction history recording unit 130 on the reproduction history list screen 170 based on the reproduction ID (step S117). In FIG. 1, the connection between the first reproduction unit 124, the voice recognition system 110, and the reproduction history recording unit 130 is omitted.

  Then, in a state where the reproduction history list screen 170 is displayed, the second reproduction instruction accepting unit 142 accepts selection of a message to be reproduced from the reproduction history displayed on the reproduction history list screen 170 by a user operation (step S119). ). In response to the reproduction start instruction from the user, the second reproduction unit 144 refers to the voice recognition result storage unit 114 and the reproduction history recording unit 130, and from the remarks selected by the reproduction start instruction, reproduction identification information and time information The voice data and the text data are associated with each other based on the above and are reproduced synchronously (step S121). This reproduction method and the like are the same as those of the first reproduction unit 124. In FIG. 1, the connection between the second playback unit 144, the voice recognition system 110, and the playback history recording unit 130 is omitted.

  As described above, according to the audio reproduction system 100 of the embodiment of the present invention, it is possible to easily search and specify a desired location from the reproduction history and reproduce it. That is, it is possible to solve the labor and time waste of searching for a specific part each time when reviewing the contents repeatedly while discussing the contents while browsing the minutes.

  For example, a voice recognition result created by the voice reproduction system 100 can be used as a minutes of a meeting or a trial. For example, by providing a playback history display unit 140 that displays a history of only the necessary text sections in the minutes in the order of selection seen in the past, it is arranged in chronological order when discussions are again made based on the minutes. , Easy to find only the necessary parts. In particular, even when discussing over a plurality of days, it is possible to quickly find the place seen the previous day.

  The voice reproduction system 100 according to the present embodiment can reproduce, for example, voice data in which communication such as a meeting or a trial is recorded and text data in synchronization. The playback start position can be specified at any position, and a list of utterances arranged in chronological order is displayed. In order to discuss the contents of exchanges such as meetings and trials from within the list, utterances can be made from any position. You can discuss while listening to. For example, if the speaker's voice is difficult to hear or if there is a contradiction in the speaker's argument, it may be rehearsed, or the tone and breathing may be repeated many times to infer the speaker's psychological state. In various situations, such as when confirming, etc., it is possible to repeatedly reproduce some of the parts of the entire recording.

(Second Embodiment)
FIG. 5 is a functional block diagram showing the configuration of the audio reproduction system 200 according to the embodiment of the present invention.
The audio reproduction system 200 according to the present embodiment is different from the above-described embodiment in that a video of a speaker at that time can be synchronized and reproduced along with a series of speech sounds. Furthermore, in the present embodiment, a specific example of a playback history list screen displayed by the playback history display unit 140 is shown.

  5, the voice recognition system 210 includes a video acquisition unit 202, a video acquisition unit 102, a voice storage unit 104, a voice recognition unit 112, and a voice recognition result storage unit 114 of the voice recognition system 110 of FIG. And a storage unit 204. Further, the audio reproduction system 200 of this embodiment has the same configuration as the audio reproduction system 100 of the above-described embodiment of FIG. 1 except for the statement list display unit 120, the first reproduction unit 124, and the second reproduction unit 144. In addition, the message list display unit 120, the first reproduction unit 124, and the second reproduction unit 144 are replaced with a message list display unit 220, a first reproduction unit 224, and a second reproduction unit 244.

  The video acquisition unit 202 acquires a video V obtained by imaging a speaker using a video camera (not shown) or the like. The video storage unit 204 stores the video acquired by the video acquisition unit 202 in association with time information as video data. In the present embodiment, the voice data stored in the voice storage unit 104, the text data stored in the voice recognition result storage unit 114, and the video data stored in the video storage unit 204 are Associated with time information.

  The speech list display unit 220 refers to the speech storage unit 104, the speech recognition result storage unit 114, and the video storage unit 204, and displays the speech recognition results of a series of speeches in a time series on the speech list screen. The first playback unit 224 refers to the voice storage unit 104 and the voice recognition result storage unit 114, and from the selected utterance, associates the corresponding voice data and text data based on the time information and plays back in synchronization. The video is displayed on the screen in synchronism with the voice output of the speech specified by the user operation.

  FIG. 6 is a diagram illustrating an example of the message list screen 250 displayed by the message list display unit 220 of the present embodiment. The message list display unit 220 can open a video display window 280 in addition to the message list 252 on the message list screen 250, and the video stored in the video storage unit 204 in the video display window 280 is reproduced in synchronization with the audio. Is displayed.

  Returning to FIG. 5, the first playback unit 224 refers to the voice storage unit 104, the voice recognition result storage unit 114, and the video storage unit 204, and selects the corresponding voice data and text based on the time information from the selected utterance. Data and video data are linked and played back in synchronization. The method for reproducing audio data and text data is the same as that of the first reproducing unit 124 in FIG. The video data is displayed in the video display window 280 of the message list screen 250 described above. In FIG. 5, the connection between the first playback unit 224, the voice recognition system 210, and the playback history recording unit 130 is omitted. In addition, although reproduction | regeneration is not specifically limited, It can reproduce | regenerate continuously from a starting position.

  As shown in FIG. 7, the playback history display unit 140 can open the playback history list window 270 when the playback history button 274 is pressed by a user operation. The playback history list window 270 includes a playback list. 272 is displayed. In the example of FIG. 7, the reproduction list 272 displays the messages arranged in the order of reproduction in time series. The reproduction list 272 includes a speech time, a speaker, and the content of the speech.

  Returning to FIG. 5, the second playback unit 244 refers to the voice storage unit 104, the voice recognition result storage unit 114, the video storage unit 204, and the playback history recording unit 130, and determines the playback ID and time information from the selected remarks. The audio data, the text data, and the video data are associated with each other and reproduced in synchronization. The playback method and the like are the same as those of the first playback unit 224. In FIG. 5, the connection between the second playback unit 244, the voice recognition system 210, and the playback history recording unit 130 is omitted.

  The operation of the audio reproduction system 200 of the present embodiment configured as described above will be described below. FIG. 8 is a flowchart showing an example of the operation of the audio reproduction system 200 according to the embodiment of the present invention. Hereinafter, description will be made with reference to FIGS.

  The flowchart in FIG. 8 includes the same steps as those in the flowchart in FIG. 4 of the above embodiment, and further includes steps S201, S203, S211, and S213.

  First, when S101 to S107 are executed by the voice acquisition unit 102 and the voice recognition unit 112, at the same time, the video acquisition unit 202 acquires the video V (step S201), and the corresponding voice data and voice recognition as the video data. The text data as the speech recognition result of the unit 112 and the time information for each utterance are stored in association with each other (step S203).

  In step S111, when the first reproduction instruction accepting unit 122 accepts selection of an utterance to start reproduction from a series of utterances displayed on the utterance list screen 250 (FIG. 7), the first reproducing unit 224 The speech storage unit 104, the speech recognition result storage unit 114, and the video storage unit 204 are referred to, and the speech recognition result of a series of speeches is displayed on the speech list screen 250 in time series on the speech list display unit 220. Then, the video is displayed on the video display window 280 (FIG. 7) in synchronization with the voice output of the speech specified by the user operation (steps S211 and S113). In step S109, the message list display unit 220 displays the message list screen 250 of FIG.

  In step S119, when the selection of the message to be reproduced is received from the reproduction history (reproduction history list window 270) displayed in the reproduction history list window 270, the message list display unit 220 displays the voice storage unit 104, The speech recognition result storage unit 114 and the video storage unit 204 are referred to, and speech recognition results of a series of speech are displayed on the speech list screen 250 (FIG. 6) in time series. The second playback unit 244 refers to the voice storage unit 104 and the voice recognition result storage unit 114, and from the selected utterance, the voice data and the text data are associated with each other based on the time information and played back synchronously. The video is displayed on the video display window 280 in synchronization with the voice output of the speech specified by the operation (steps S213 and S121).

  In the present embodiment, a method for displaying a playback history will be described below. FIG. 9 is a diagram showing an example of a playback history list screen of the audio playback system according to the embodiment of the present invention. FIG. 9A shows an example of a screen that sorts and displays the playback history in the order of the number of playbacks, and FIG. 9B shows an example of a screen that displays the playback history in the order of the number of playbacks. Note that the example of FIG. 7 is a screen that displays the playback history in chronological order in the order of playback time.

  In the audio reproduction system 200 of the present embodiment, the reproduction history recording unit 130 records the audio text data reproduced by the first reproduction unit 224 as a reproduction history in association with the reproduction time information, and the reproduction history display unit 140 Based on the reproduction identification information (reproduction ID), the recorded reproduction history is displayed in the reproduction history list window 270 in time series. In the audio reproduction system 200 of the present embodiment, the reproduction history recording unit 130 includes a reproduction number counting unit (not shown) that counts the number of reproductions of the audio text data reproduced by the second reproduction unit 244. The history display unit 140 displays the recorded playback history on the playback history list screen based on the number of times of playback.

  For example, in the example of FIG. 9A, the reproduction history list screen 275 is sorted and displayed in descending order of the number of times of reproduction, and only those that are a predetermined number of times or more are displayed. As a result, the replayed message is clearly recognized at a glance compared to the reproduction history list screen 170 of the above embodiment that displays the reproduction history in the order of the reproduction time. Although not shown, the number of reproductions may be displayed.

  Further, in the example of FIG. 9B, the playback history list screen 285 displays the speech color-changed according to the number of playbacks or the frequency. For example, when the number of times of utterances in the row 284 is greater than the number of times of utterances in the row 282, the color in the row 284 can be changed to a more prominent color.

  In other words, for the most frequently viewed items in the playback history (points that are discussed many times), only the points of discussion can be extracted based on the labels by automatically attaching labels according to the number of playbacks. Efficiency will increase.

As described above, the audio reproduction system 200 according to the embodiment of the present invention achieves the same effect as the above embodiment, and can reproduce not only audio but also video and text data in synchronization. This makes it possible to check the facial expression and gesture when the speaker is speaking.
Further, according to this configuration, since the time at the time of discussion can be registered as the playback time in association with the text section in the playback history, the text section and the video associated with the text section are displayed based on the playback time. It is possible to search for “whether the argument was made” as a key, and to find a desired remark part more quickly.

  Furthermore, since the playback history can be displayed with a high number of playbacks in an easy-to-view manner, the points that have been repeatedly discussed are obvious.

  For example, when discussing whether or not the defendant was murderous in a trial, it is assumed that the defendant's real intention is searched by repeatedly looking at the relevant minutes and videos. It is assumed that the point of discussion will be seen many times, and may be seen in the future. In such a case, discussion points can be easily extracted and searched by performing a marking process such as changing the color of the corresponding text section in the reproduction history. You may enable it to display on another screen, and you may enable it to search by marking.

  Also, for example, by changing the color according to the frequency of viewing, you can see at a glance what points have been discussed when viewing the playback history, whether there are any points that have not been discussed yet, or whether the discussion is biased There is also an effect that can be easily grasped visually. In addition, since the high frequency means that it is a priority discussion point ("argument" in the trial), it is possible to arrange the history separately in an important order, or to extract only the important argument. . Thus, the playback history is much easier to use and convenient.

(Third embodiment)
FIG. 10 is a functional block diagram showing the configuration of the audio reproduction system 300 according to the embodiment of the present invention.
The audio reproduction system 300 of this embodiment is different from the above embodiment in that it can search and narrow down the messages displayed on the message list screen 150 according to the conditions, and can receive a reproduction instruction from them.

  The voice reproduction system 300 according to the present embodiment refers to the condition receiving unit 302 that receives a search condition for narrowing down the messages displayed on the message list screen, and the voice recognition result storage unit 114, and searches for a message according to the search condition. And a search result display unit 306 that displays the searched search results as a search result list screen. The first reproduction instruction receiving unit 322 includes a search result list screen that includes a search result list screen. The selection of the speech to start playback from is accepted.

  Specifically, the audio reproduction system 300 of this embodiment further includes a condition receiving unit 302, a search unit 304, and a search result display unit 306 in addition to the configuration of the audio reproduction system 200 of the above embodiment. Further, a first reproduction instruction receiving unit 322 is provided instead of the first reproduction instruction receiving unit 122 of the audio reproduction system 200 of the above embodiment. In addition, it is good also as a structure which combined these with the structure of the audio | voice reproduction | regeneration system 100 of the said embodiment of FIG.

  The condition receiving unit 302 receives a search condition for narrowing down the messages displayed on the message list screen 250. For example, as shown in FIG. 11, in this embodiment, a search button 340 is provided on the message list screen 250, and the search result display unit 306 displays the search screen 350 when the search button 340 is pressed. The search screen 350 has a search condition input field 352 and a search result display field 354.

  The condition receiving unit 302 receives various search conditions input or selected in the search condition input field 352. Here, for example, as a search condition, a character string of the content of a statement, a type of speaker, a speech time, a time zone, and the like can be designated.

  The search unit 304 searches the speech recognition result storage unit 114 according to the search condition specified by the user operation in the search condition input field 352 and searches for a statement.

  The search result display unit 306 displays the searched search results in the search result display field 354 of the search screen 350.

  In the present embodiment, the first reproduction instruction accepting unit 322 accepts selection of an utterance for starting reproduction from utterances displayed in the search result display field 354 of the search screen 350.

  The operation of the sound reproduction system 300 of the present embodiment configured as described above will be described below. FIG. 12 is a flowchart showing an example of the operation of the audio reproduction system according to the embodiment of the present invention. Hereinafter, a description will be given with reference to FIGS.

  In step S109 of the flowchart of FIG. 4 or FIG. 8 of the voice reproduction system of the above embodiment, after the message list display unit 120 or the message list display unit 220 displays the message list screen 150 or the message list screen 250, for example, FIG. This processing starts when the user presses the search button 340 of the 11 message list screen 250. A search screen 350 is displayed by the search result display unit 306, and the condition receiving unit 302 receives the search condition input in the search condition input field 352 (step S301).

  Then, the search unit 304 refers to the voice recognition result storage unit 114 and searches for a corresponding statement according to the search condition received in step S301 (step S303). The search result display unit 306 displays the search result extracted in step S303 in the search result display field 354 (step S305). Returning to FIG. 4 or FIG. 8, in step S <b> 111, the first reproduction instruction accepting unit 322 accepts selection of a message to start reproduction. The following is the same as in the above embodiment.

  As described above, according to the audio reproduction system 300 of the present embodiment, the same effect as the audio reproduction system of the above embodiment can be obtained, and a reproduction instruction can be given after narrowing down the content of a statement by search conditions. It becomes possible to find a desired remark efficiently.

(Fourth embodiment)
FIG. 13 is a functional block diagram showing the configuration of the audio reproduction system 400 according to the embodiment of the present invention.
The audio reproduction system 400 according to the present embodiment is different from the above embodiment in that, for example, an arbitrary operation is accepted in response to a message displayed in the reproduction list 272 of the reproduction history list window 270 in FIG. Is different.

The audio reproduction system 400 of the present embodiment further includes an operation accepting unit 402 that accepts an arbitrary operation on a message displayed on the reproduction history list screen 170, and an execution unit 404 that executes the accepted operation.
For example, a marker can be given to the utterance, and a memo or comment can be input to the utterance.

  In this embodiment, as shown in FIG. 14, a playback history recording unit 430 is provided instead of the playback history recording unit 130 of the above embodiment. In addition to the information recorded by the playback history recording unit 130, the playback history recording unit 430 further performs level classification based on the number of playbacks in the above embodiment, remarks, and remarks for each playback ID. The memo attached to can be recorded.

  In the present embodiment, the playback history list screen 480 that displays information of the playback history recording unit 430 can include a memo field 482 as shown in FIG.

  Note that the audio reproduction system 400 of this embodiment can be combined with any of the configurations of the audio reproduction system 100, the audio reproduction system 200, and the audio reproduction system 300 of the above-described embodiment shown in FIGS.

  For example, the operation receiving unit 402 can receive an instruction to delete unnecessary messages from the messages displayed on the reproduction history list screen, and the execution unit 404 can delete the messages that have received the deletion instruction from the reproduction history.

  Alternatively, the playback history display means further includes an extraction unit (not shown) that extracts a message that satisfies a predetermined condition from the recorded playback history, and adds predetermined identification information to the extracted message. Can display the extracted utterances in an identifiable manner.

  For example, it is possible to extract those whose number of reproductions is equal to or greater than a predetermined number, those having a predetermined marker, those having a predetermined keyword in a memo or comment, and those having a predetermined date and time. The extracted remarks are preferably displayed with improved visibility by color change, blinking, reversal, mark assignment, separate window (pop-up) display, and the like.

  The operation of the sound reproduction system 400 of this embodiment configured as described above will be described below. FIG. 16 is a flowchart showing an example of the operation of the audio reproduction system according to the embodiment of the present invention. This will be described below with reference to FIGS.

  After the reproduction history list screen is displayed in step S117 in the flowcharts of the respective drawings of the audio reproduction system of the above embodiment, the operation reception unit 402 receives various operations (step S401). When deletion of an utterance is received by a user operation (deletion in step S401), the execution unit 404 deletes the specified utterance from the reproduction history of the reproduction history recording unit 430 (step S403).

  When a memo input for a utterance is received by a user operation (a memo in step S401), the execution unit 404 adds a memo to the specified utterance. Here, the memo is stored in the information of the corresponding reproduction ID of the reproduction history of the reproduction history recording unit 430 (step S405).

  As described above, according to the audio reproduction system 400 according to the embodiment of the present invention, the same effects as those of the above embodiment can be obtained, and operations can be performed on each message in the reproduction list on the reproduction history list screen. For example, you can delete a comment that you tried to play but were not needed from the playback history, or if you want to add a comment to a comment during a discussion, you can add a note, The playback history can be used more efficiently.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

For example, in another embodiment, the audio reproduction system can further include an output unit (not shown) that outputs the reproduction history recorded in the reproduction history recording unit of the audio reproduction system of the above embodiment to the outside.
According to this configuration, it is possible to use the reproduction history as a minutes of a meeting or the like in which a discussion is performed while watching a series of statements.

  In another embodiment, a time designation receiving unit (not shown) that receives designation of a reproduction time of a reproduction history displayed on the reproduction history list screen may be further provided. The reproduction history display unit can display the recorded reproduction history on the reproduction history list screen based on the designated reproduction time.

While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
<< Appendix >>
<Appendix 1>
A speech memory in which speech of a series of speeches is speech-recognized, speech recognition results are converted into text data, and the obtained speech speech data and the text data are stored in association with time information for each speech Referring to a device and a speech recognition result storage device, a speech list display means for displaying the speech recognition results of the speech of the series of speech on a speech list screen in time series;
Selection accepting means for accepting a choice of a comment from the series of comments displayed on the message list screen;
A selection history recording unit that records the text data of the voice for the utterance selected by the selection receiving unit as a selection history in association with selection identification information that identifies a selection instruction order;
A browsing history system comprising: a selection history display unit configured to display the recorded text data of the selection history on the selection history list screen side by side based on the selection identification information.
<Appendix 2>
In the browsing system described in Appendix 1,
The selection receiving unit is a browsing system that receives a selection of a statement from the selection history displayed on the selection history list screen.
<Appendix 3>
In the browsing system according to appendix 1 or 2,
The selection history recording unit records the text data of the voice for the utterance selected by the selection receiving unit as a selection history in association with the selection time information,
The browsing history display unit displays the recorded selection history on the selection history list screen based on the selection time information.
<Appendix 4>
In the browsing system according to any one of appendices 1 to 3,
The selection history recording unit further includes a selection number counting unit that counts the number of selections of the text data of the voice for the utterance selected by the selection receiving unit,
The selection history display means displays the recorded selection history on the selection history list screen based on the number of times of selection.
<Appendix 5>
In the browsing system described in Appendix 4,
A time designation accepting unit for accepting designation of the selection time information of the selection history displayed on the selection history list screen;
The browsing history system, wherein the selection history display means displays the recorded selection history on the selection history list screen based on the designated selection time information.
<Appendix 6>
In the browsing system according to any one of appendices 1 to 5,
Operation accepting means for accepting an arbitrary operation on the remarks displayed on the selection history list screen;
A browsing system further comprising execution means for executing the received operation.
<Appendix 7>
In the browsing system described in appendix 6,
The operation accepting unit accepts an instruction to delete unnecessary utterances from the utterances displayed on the selection history list screen,
The browsing system in which the execution means deletes the message that has received the deletion instruction from the selection history.
<Appendix 8>
In the browsing system according to any one of appendices 1 to 7,
Extraction means for extracting a utterance that satisfies a predetermined condition from the recorded selection history, and adding predetermined identification information to the extracted utterance,
The selection history display means is a browsing system that displays the extracted utterances so that they can be identified.
<Appendix 9>
In the browsing system according to any one of appendices 1 to 8,
A browsing system further comprising output means for outputting the selection history recorded in the selection history recording means to the outside.
<Appendix 10>
In the browsing system according to any one of appendices 1 to 9,
Condition accepting means for accepting a search condition for narrowing down the comments displayed on the comment list screen;
Search means for referring to the voice recognition result storage device and searching for the utterance according to the search condition;
Search result display means for displaying the searched search results as a search result list screen;
The selection receiving unit is a browsing system that receives a selection of a message from among messages displayed on the search result list screen.
<Appendix 11>
In the browsing system according to any one of appendices 1 to 10,
Playback that refers to the voice storage device and the voice recognition result storage device, and plays back the speech data and the text data in association with each other based on the time information from the utterance selected by the selection receiving means. A browsing system further comprising means.
<Appendix 12>
A speech memory in which speech of a series of speeches is speech-recognized, speech recognition results are converted into text data, and the obtained speech speech data and the text data are stored in association with time information for each speech Referring to a device and a speech recognition result storage device, displaying the speech recognition results of the speech of the series of speech on a speech list screen in time series,
Accepting a selection of comments from the series of messages displayed on the message list screen,
The text data of the voice for the selected utterance is recorded in a selection history recording device as a selection history in association with selection identification information that identifies the selection instruction order,
A browsing method for displaying the recorded text data of the selection history on the selection history list screen side by side based on the selection identification information.
<Appendix 13>
In the browsing method described in appendix 12,
A browsing method of referring to the voice storage device and the voice recognition result storage device, and reproducing the synchronized voice data and the text data in association with the voice data based on the time information from the selected utterance.
<Appendix 14>
A speech memory in which speech of a series of speeches is speech-recognized, speech recognition results are converted into text data, and the obtained speech speech data and the text data are stored in association with time information for each speech Referring to a device and a speech recognition result storage device, and displaying the speech recognition result of the speech of the series of speeches on a speech list screen in time series;
A procedure for accepting selection of a message from the series of messages displayed on the message list screen;
A procedure for recording the text data of the voice for a selected utterance as a selection history in association with selection identification information for identifying a selection instruction order;
A computer program for causing a computer to execute the procedure of displaying the recorded text data of the selection history on the selection history list screen side by side based on the selection identification information.
<Appendix 15>
In the computer program according to attachment 14,
Referring to the voice storage device and the voice recognition result storage device, the voice data and the text data are associated with each other and reproduced based on the time information from the utterance selected in the procedure of accepting the selection. A computer program for causing a computer to further execute a playback procedure.

100 voice reproduction system 102 voice acquisition unit 104 voice storage unit 110 voice recognition system 112 voice recognition unit 114 voice recognition result storage unit 120 speech list display unit 122 first playback instruction receiving unit 124 first playback unit 130 playback history recording unit 140 playback History display unit 142 Second playback instruction accepting unit 144 Second playback unit 150 Statement list screen 152 Statement list 154 Speaker name 156 Message content 160 Message list screen 170 Playback history list screen 200 Audio reproduction system 202 Image acquisition unit 204 Image storage unit 210 Speech recognition system 220 Statement list display unit 224 First playback unit 244 Second playback unit 250 Message list screen 252 Message list 270 Playback history list window 272 Playback list 274 Playback history button 275 Playback history list screen 280 Video display window 285 Playback history List screen 300 Audio playback system 302 Condition reception unit 304 Search unit 306 Search result display unit 322 Playback instruction reception unit 340 Search button 350 Search screen 352 Search condition input column 354 Search result display column 400 Audio playback system 402 Operation reception unit 404 Execution unit 430 Playback history recording unit 480 Playback history list screen 482 Memo field

Claims (15)

A speech list display means for displaying a speech list screen for displaying a list of text data of a plurality of the speeches obtained by performing speech recognition processing on speech data including a plurality of speeches ;
Selection accepting means for accepting selection of one of the plurality of the comments displayed on the comment list screen;
Playback means for playing back the audio data from the position of the statement received by the selection receiving means;
Selection history display means for displaying the history of the selected utterance on a selection history list screen;
A browsing system comprising: The browsing system according to claim 1,
The selection receiving means further includes a browsing system for receiving selection of one of the utterances from the utterances displayed on the selection history list screen. In the browsing system according to claim 1 or 2,
  The browsing history system, wherein the selection history display means displays the selected utterances on the selection history list screen in order of increasing selection frequency. In the browsing system according to any one of claims 1 to 3,
  The browsing history system, wherein the selection history display means displays, on the selection history list screen, the remarks that are selected a predetermined number of times or more.   The browsing system according to any one of claims 1 to 4,
  The speech list display means performs speech recognition processing on the speech of a series of speech, converts the speech recognition result into text data, and the obtained speech data of the speech and the text data are associated with time information for each speech. The speech recognition results of the voices of the series of utterances are displayed on the utterance list screen in chronological order, with reference to the speech storage device and the speech recognition result storage device stored respectively.
  A selection history recording unit that records the text data of the voice for the utterance selected by the selection receiving unit as a selection history in association with selection identification information that identifies a selection instruction order;
  The browsing history system, wherein the selection history display means displays the selection history text data recorded by the selection history recording means on the selection history list screen side by side based on the selection identification information. In the browsing system according to claim 5 ,
The selection history recording unit records the text data of the voice for the utterance selected by the selection receiving unit as a selection history in association with the selection time information,
The browsing history display unit displays the recorded selection history on the selection history list screen based on the selection time information. In the browsing system according to claim 5 or 6 ,
The selection history recording unit further includes a selection number counting unit that counts the number of selections of the text data of the voice for the utterance selected by the selection receiving unit,
The selection history display means displays the recorded selection history on the selection history list screen based on the number of times of selection. The browsing system according to claim 7 ,
A time designation accepting unit for accepting designation of the selection time information of the selection history displayed on the selection history list screen;
The browsing history system, wherein the selection history display means displays the recorded selection history on the selection history list screen based on the designated selection time information. The browsing system according to any one of claims 5 to 8 ,
Operation accepting means for accepting an arbitrary operation on the remarks displayed on the selection history list screen;
A browsing system further comprising execution means for executing the received operation. The browsing system according to claim 9 , wherein
The operation accepting unit accepts an instruction to delete unnecessary utterances from the utterances displayed on the selection history list screen,
The browsing system in which the execution means deletes the message that has received the deletion instruction from the selection history. The browsing system according to any one of claims 5 to 10 ,
Extraction means for extracting a utterance that satisfies a predetermined condition from the recorded selection history, and adding predetermined identification information to the extracted utterance,
The selection history display means is a browsing system that displays the extracted utterances so that they can be identified. The browsing system according to any one of claims 5 to 11 ,
A browsing system further comprising output means for outputting the selection history recorded in the selection history recording means to the outside. The browsing system according to any one of claims 5 to 12 ,
Condition accepting means for accepting a search condition for narrowing down the comments displayed on the comment list screen;
Search means for referring to the voice recognition result storage device and searching for the utterance according to the search condition;
Search result display means for displaying the searched search results as a search result list screen;
The selection receiving unit is a browsing system that receives a selection of a message from among messages displayed on the search result list screen.   Computer
  A speech list display step for displaying a speech list screen for displaying a list of a plurality of text data of the speech obtained by performing speech recognition processing on speech data including a plurality of speeches;
  A selection accepting step for accepting selection of one of the plurality of the comments displayed on the comment list screen;
  A reproduction step of reproducing the audio data from the selected position of the utterance;
  A selection history display step for displaying the history of the selected utterance on a selection history list screen;
Browsing method to execute. Computer
  A speech list display means for displaying a speech list screen for displaying a list of text data of the plurality of speeches obtained by performing speech recognition processing on speech data including a plurality of speeches;
  Selection accepting means for accepting selection of one of the plurality of the comments displayed on the comment list screen;
  Playback means for playing back the audio data from the selected position of the utterance;
  Selection history display means for displaying the history of the selected utterance on a selection history list screen;
Program to function as.

Images