JP2013117879A - Material display and audio reproduction system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system, associated mainly with materials of a conference, a lecture class, etc., for efficiently reproducing and understanding details of conference and lecture class materials by not only enabling a person who did not attend to reproduce the materials and voice of the conference or lecture class later as he or she were there, but also devising how to make people hear data of recorded voice and how to show the materials.SOLUTION: A material voice reproduction system for material sharing includes means of listing recorded voice data on materials being displayed in a conference or lecture class along the time base, and means of displaying recorded parts close to details of the material as a voice list when they are available. Further, the system includes means of displaying all material screens related to voice when the displayed recording list is selected and the voice in the recording is heard.

Description

  The present invention relates to a material display and sound reproduction system, and more particularly to a material display and sound reproduction system in material sharing.

  As a background technology in this technical field, paper is saved by using a system in which materials are stored in a server at a conference or seminar, a plurality of terminals are interconnected by a wireless LAN or a wired network, and the materials are electronically shared. It is illustrated. In a meeting or a workshop using such a material sharing system, it is generally performed to record a voice using a voice recording device in order to record the contents. For example, there exists Unexamined-Japanese-Patent No. 2009-187462 (patent document 1). In this gazette, a meeting or a lecture is started using materials stored in a PC, and recording is performed at the same time. It has been proposed to display the material of the corresponding page on the screen of the PC in accordance with the time for reproducing the audio data and the timing of turning the page. Moreover, there exists Unexamined-Japanese-Patent No. 2009-098882 (patent document 2). This gazette states that “when a document related to a conference is searched, an information processing system that outputs information related to the document and generated in the conference is provided”.

JP 2009-186462 A JP 2009-098882 A

The technology so far has been the display of materials mainly considering voice. For this reason, the audio data time and the page turning time are often displayed together to reproduce the conference as it is. In addition, the technology for searching for materials focusing on improving the quality of discussions in a conference displays only the searched materials and does not reproduce the conference or workshop. Furthermore, if the recorded content is related to any number of pages of the material, and there are multiple pages to be referred to, the recorded content and the content of the displayed material did not necessarily match. In addition, there is a high possibility that the progress of the conference will proceed regardless of the materials, such as when you skip a part of the abandonment or recording that is not related to the conference or workshop at once, or if you want to hear only part of it or pinpoint It took a lot of effort to find a recording that matched the material.
In view of the above points, the present invention is mainly based on materials such as conferences and workshops, and even if those who could not participate in the meeting, the materials and audio of the conference and workshop were reproduced as they were as if they were there at a later date. It aims to provide a system that improves the understanding of users by efficiently reproducing the contents of conference and seminar materials by devising how to listen to recorded audio data and how to display the materials.

In the material sound reproduction system for sharing this material, for example, there are means for displaying recorded sound data of a material being displayed in a meeting or a seminar in a list on a time axis, and if there is a recording portion close to the content of the material, the sound list is used. One of the features is that it has a display means.
In addition, this material sound reproduction system has, for example, a means for displaying all the material screens associated with the sound when selecting and listening to the displayed recording list.

According to the solution of the present invention,
A plurality of terminals including a first terminal and a second terminal and a server are provided, and in a recording stage, a material displayed on the first terminal, an operation event specifying the operation of the first terminal, and audio data are used as a time stamp. Correspondingly recorded on the server, a material display and sound reproduction system for reproducing the sound and displaying the material on the second terminal in the reproduction stage,
The server
A material analysis data storage unit in which keywords in pre-registered materials are stored;
A recorded voice storage unit for storing recorded voice data;
A recording character list in which the recorded voice data is converted into text, a recording analysis data storage unit stored corresponding to the time stamp, and
An operation data storage unit in which the material displayed on the first terminal and the operation event for specifying the operation of the first terminal are stored corresponding to the time stamp;
A processing unit that reproduces the audio data stored in the recorded audio storage unit and the display of the material according to the operation event stored in the operation data storage unit on the second terminal;
In the reproduction stage, the processing unit
The material keyword reproduced and displayed on the second terminal is identified with reference to the material analysis data storage unit,
Search the recording analysis data storage unit based on the identified keyword, extract a recording character list including the keyword, and a corresponding time stamp,
Transmitting a recording list including the extracted one or more recording character lists to the second terminal;
The second terminal is
When the received recording list is displayed and one of the recording character lists included in the displayed recording list is selected, the sound corresponding to the selected recording character list is displayed according to the time stamp corresponding to the selected recording character list. A material display and sound reproduction system for reproducing data is provided.

In the above-mentioned material display and sound reproduction system, not only fast-forward and rewind as a method of selecting sound data, but also a recording list display and selective listening, and a recording list close to the content based on the displayed material are displayed. Therefore, it is possible to find the recording content that matches the material.
In the above-described document display and sound reproduction system,
Further comprising a material storage unit in which data of materials to be displayed on the plurality of terminals is registered in advance,
The recording analysis data storage unit stores the keyword included in the recording character list in correspondence with the recording character list,
In the reproduction stage, the processing unit
Referring to the recording analysis data storage unit, identify a keyword corresponding to the selected recording character list,
Refer to the material analysis data storage unit based on the identified keyword, identify the material containing the keyword,
The data of the material is transmitted to the terminal and displayed.

  In this material display and sound reproduction system, when the displayed recording list is selected and listened to, a material screen associated with the sound can be displayed, so that the user's effort to search for himself is halved. In this way, when reproducing recorded conferences and workshops using materials and listening to recorded audio while viewing the materials, the materials you want to see and the sounds you want to hear are reproduced more accurately than before, improving the user's understanding. Can be made.

  According to the present invention, not only the materials such as conferences and workshops, but also those who could not participate in the place, as well as reproducing the materials and audio of the conference and workshops as if they were there at a later date, It is possible to provide a system that improves the user's understanding by efficiently reproducing the contents of the conference and workshop materials by devising how to listen to the recorded audio data and displaying the materials.

It is an example of a block diagram of a document sharing system. It is an example of the block diagram of a server. It is an example of the block diagram of a terminal. It is an example of the data registration screen of a server. It is a structural example of the meeting material storage part of a server. It is an example of data analysis data. It is a creation flowchart figure of data analysis data. It is a creation flowchart figure of operation data. It is a recording analysis data creation flowchart figure. It is a display method flowchart figure of a time-sequential recording list. It is a display method flowchart figure of a candidate recording list. FIG. 10 is a flowchart of another screen display method when a candidate recording list is selected. It is an example of a meeting reproduction data storage part. It is an example of operation data. It is an example of recording analysis data. It is an example of the screen of a terminal. It is an example of the reproduction screen 1 of a terminal. It is an example of the reproduction screen 2 of a terminal. It is an example of the sequence of the meeting using a document sharing system. It is an example of the sequence diagram of meeting reproduction. This is a candidate recording content that is close to the content of the material being displayed.

  Hereinafter, examples will be described with reference to the drawings.

In the present embodiment, a conference using a document sharing system (document display and voice reproduction system) and an example of conference reproduction will be described.
FIG. 17 shows an example of a conference sequence using the document sharing system. First, in the sequence of FIG. 17, the server 11 registers an example in which the server 11 registers materials necessary for the meeting and the workshop, an example of the progress of the meeting and the workshop, and data necessary for reproducing the meeting and the workshop. An example of creation will be described.

(System configuration)
First, FIG. 1 shows an example of devices constituting the document sharing system.
The document sharing system includes a server 11, an access point 12, and a plurality of terminals 13. The server 11 and the terminals 13 to 14 are connected by a LAN via the access point AP12, and can perform communication such as message transmission and reception using a protocol such as HTTP. The terminals 13 to 14 may be wireless terminals, and the accelerator point 12 and the terminals may communicate wirelessly. The terminals 13 to 14 may be, for example, a smartphone terminal, a tablet terminal, a netbook, a terminal such as a PC having a wireless LAN function, or a terminal such as a PC that can communicate with a server by wire.
FIG. 2 shows a configuration example of the server 11. The server 11 is a personal computer that has a CPU (processing unit) 21 that performs processing, a storage unit 22, an input device 23, an output device 24, and a communication line interface 25 such as a LAN, and can be constructed as a server. The storage unit 22 stores, for example, a Web application 221 and a server application 228. The storage unit 22 includes, for example, a conference material storage unit 222, a material analysis data storage unit 223, a conference reproduction data storage unit 224, a recording voice storage unit 225, an operation data storage unit 226, and a recording analysis data storage unit 227. Including. The input device 23 includes, for example, a keyboard 231, a microphone 232, a mouse 233, a CD, and a USB drive 234. The output device 24 includes, for example, a monitor 241 and a speaker 242.
FIG. 3 shows a configuration example of the terminals 13 and 14. The terminals 13 and 14 have a recordable microphone 331 and a recording function. The terminals 13 and 14 include, for example, a CPU 31, a storage unit 32, an input device 33, an output device 34, and a communication line interface 35 such as a LAN. For example, a terminal such as a personal computer or an Android terminal is suitable. The storage unit 32 includes, for example, a recorded voice storage unit 321 and an Android application 322. The input device 33 includes, for example, a keyboard 331, a microphone 332, and a mouse 333. The output device 34 includes a monitor 341 and a speaker 342, for example.

(Operation at the recording stage of the meeting)
First, the server 11 stores materials used for a meeting or a seminar (171 in FIG. 17). For this purpose, the server 11 activates the Web application 221 shown in FIG. 2 and displays a screen for registering materials to be used for the meeting and seminar shown in FIG. The example of FIG. 4 displayed on the screen of the server 11 is an example in which the conference C is input in the conference name column 48 and 2011/9/5 is input in the date / time column. The server 11 reads the material C1 from these storage media or from the hard disk using a CD or USB input device indicated by reference numeral 234 in FIG. Identification information such as the name of the material C1 is displayed in the material name column 49. The user operates the keyboard 231 and the mouse 233 to select the material C1 and inputs the selected information from the input device 23. The server 11 recognizes that the registration 41 in FIG. 4 has been pressed, and registers the selected material in the conference material storage unit 222 in FIG. Registration of the material data in the server 11 can also be performed by the terminals 13 and 14 accessing the server 11.
FIG. 5 is a configuration diagram of the conference material storage unit 222 of FIG. As shown in FIG. 5, the server can register a plurality of materials for each conference. For example, the meeting material storage unit 222 stores materials corresponding to the meeting date and meeting name. There may be multiple documents or one document for a meeting.
Next, the server 11 registers the keyword used for reproducing the meeting C of the registered material C1 automatically or manually, or both. In the case of automatic, the server 11 recognizes that the start 42 in FIG. 4 has been pressed and performs a keyword search. For example, the title of the material, characters / words included in the title, frequently used words in the material, etc. are searched from one page, and if they match, the data is stored in the material analysis data 223 of FIG. The title can also be input from the input device 23. In addition, as for the usage frequency, for example, some of the top usage frequencies may be extracted, or a threshold value for the number of times of use or a threshold value for the usage ratio with respect to the whole may be determined in advance, and more than that may be extracted. . The keyword may be an appropriate character string such as a word, idiom, or phrase. A configuration example of the material analysis data is shown in FIG. For example, the analysis data 64 of the material C1 is stored for the material C1 of the conference C. The analysis data includes a correspondence between a page number and a keyword included in the page for each meeting material.
Further, the server 11 displays the registered keyword in the list 47 shown in FIG. The server 11 automatically updates the page, searches all pages, and creates material analysis data for all pages. If more keywords are required, the server 11 detects the keyword input from the input position indicated by reference numeral 43 in FIG. 4 and that the search 744 has been pressed, and searches all pages in the same manner as the automatic keyword search described above. Then, the material C1 of the meeting C is additionally stored in the analysis data 64 of the material C1, and is additionally displayed in the list 47 in FIG.
When the server 11 detects that the material addition 46 is pressed, the server 11 returns to the operation of reading another material from these storage media or from the hard disk by the CD or USB input device 231 in FIG. Register another document in the procedure. As described above, the server 11 stores the material C1 in the portion of the material C1 of 53 in FIG. 5, stores the analysis data created by the material C1 in 641 of FIG. And manage each analysis data.

FIG. 7 shows a flowchart for creating the material analysis data. Since each process is as described above, a description thereof will be omitted.
After the server 11 registers the material in this way, the terminals 13 and 14 activate the application 322 in FIG. Of the plurality of terminals, for example, one terminal for proceeding with a conference or a seminar is a presenter terminal (first terminal) 13. The presenter terminal 13 displays the screen of FIG. When the presenter terminal 13 detects that the material list 152 is pressed, the presenter terminal 13 outputs a conference material list display request (17-1) to the communication line interface 35 of FIG. The server 11 receives the message transmitted by the presenter terminal 13 from the communication line interface 25 in FIG. 2, and transmits the conference material list data to the conference material storage unit 222 in FIG. 5 (17-2). The conference material list data includes, for example, a conference name and a material name. The message transmission / reception is similarly performed via the communication line interface 35 in FIG. 3 on the presenter terminal 13 side and 25 in FIG. 2 on the server side. The presenter terminal 13 displays a material list on the monitor 342 of FIG. 3 of the presenter terminal 13 based on the received meeting material list data. For example, it is displayed in the area 151 of FIG. The presenter terminal 13 detects that the material C1 is selected by the input device such as the mouse 332 of FIG. 3 and transmits a material C1 request (17-3) to the server 11.
The server 11 transmits the data of the first page of the material C1 stored in the conference material storage unit 222 to the presenter terminal 13 (17-4). Further, the server 11 transmits the first page of the document C1 to all terminals participating in the conference (17-5). Of the terminals 13 and 14, terminals participating in the conference can be registered in advance. Each terminal 13 and 14 that has received the data of the first page of the document C1 displays the first page of the same document C1 on its own screen. The presenter terminal 13 detects that the recording conference 153 in FIG. 5 has been pressed, performs a recording conference designation 174, and transmits a recording designation message (17-6) to the server 11. The server 11 receives the recording designation message and starts creating the operation data 175. For example, an operation event and a time stamp, which are messages transmitted from the presenter terminal 13, are started to be stored in the operation data storage unit 226 in FIG. 2 with the operation data configuration in FIG. 13, for example. In addition, the presenter terminal 13 inputs a voice through the microphone 332 of the input device 33 in FIG. 3 and starts recording the voice of the conference and the seminar in the recording voice storage unit 321.

FIG. 8 shows a flowchart for creating operation data. If the message from the presenter terminal 13 is a recording designation (81-83), the server 11 initializes the operation data and starts data creation (87). If the message from the presenter terminal 13 is the conference end (81 to 84), the server 11 ends the operation data creation. If the message from the presenter terminal 13 is an operation event and the conference has started (81-85), the time stamp and the operation event are stored in the operation data storage unit 227 in association with each other.
Returning to FIG. 17, the description of the processing is continued. When the conference progresses and the page is changed, the presenter terminal 13 performs page turning by pressing a button 156, for example, and transmits the 2-page designation (17-7) of the material C1 to the server 11. The server 11 transmits the data on the second page of the material C1 in FIG. 5 to the presenter terminal 13 (17-8). At the same time, the server 11 transmits the second page of the document C1 to all the terminals participating in the conference (17-9). Further, the time stamp 131 and the page feed 1322 of the operation event 132 are added to the operation data storage unit 226 of FIG. Each terminal that has received the data on the second page of the document C1 displays the second page of the same document C1 on its own screen.
Hereinafter, the presenter terminal 13 transmits a message to the server 11 for each event such as page feed or page return. The server 11 analyzes the message and transmits the designated page of the material data stored in the conference material storage unit 222 of FIG. Then, the terminals 13 and 14 receive the designated page of the material data and display them simultaneously on the screen.
In addition, when a plurality of materials for the conference C are registered on the material registration screen of the server 11 of FIG. 4, other materials can be designated. In the example of FIG. 17, the presenter terminal 13 displays the material list again by an appropriate operation, and when the material C2 is selected, a material C2 request (17-13) is transmitted to the server 11.

  The server 11 transmits the data of the first page of the material C2 in FIG. 5 to the presenter terminal 13 (17-14). At the same time, the server 11 transmits the first page of the document C2 to all the terminals participating in the conference (17-15). Each terminal that has received the data of the first page of the document C2 displays the first page of the same document C2 on its own screen. When performing page feed, the two-page designation (17-17) of the material C2 is transmitted to the server 11. In response to the page designation, the server 11 transmits the data on the second page of the material C2 in FIG. 5 to the presenter terminal 13 (17-18). At the same time, the server 11 transmits the second page of the document C2 to all terminals participating in the conference (17-19). Further, the time stamp 131 and the current operation event 132 “open document C2” 1325 are added to the operation data storage unit 226 of FIG. Each terminal that has received the data of the second page of the document C2 displays the second page of the same document C2 on its own screen. Hereinafter, as described above, a message is transmitted to the server 11 for each event that occurred in the presenter terminal 13, and the server 11 analyzes the message and stores it in the meeting material storage unit 222 in FIG. The data designation page is transmitted to all the terminals participating in the conference, the operation data storage unit 226 in FIG. 13 is updated, and the terminals 13 and 14 receive the data of the material and repeatedly display them on the screen. In this way, materials are shared and conferences are held.

The presenter terminal 13 detects that the conference end button 154 of FIG. 15 has been pressed when the conference is ended, and transmits a conference end notification (17-20) to the server 11. The server 11 transmits a conference end notification to the presenter terminal 13 (17-21), and also transmits a conference end notification (17-22) to all conference participant terminals. Subsequently, the server 11 stores the stop and time stamp indicated by 13211 in the operation data storage unit 226 of FIG. 13, ends the creation of the operation data (177), and completes the operation data of the conference C. Upon receiving the conference end notification sent from the server 11, the presenter terminal 13 stops the recording operation and creates a recorded audio file (178). The presenter terminal 13 uploads the recording file created here to the server 11 (17-23). The server 11 stores the received recorded sound file in the recorded sound storage unit 225 of FIG. 2 (179).
Next, the server 11 creates recording analysis data shown in FIG. 14 using the recorded voice data. The server 11 converts the recorded voice data into text by speech recognition, etc., divides the voice characters into sentences by general techniques of natural language search and syntax analysis, and uses them as data together with time stamps. The recording analysis data storage shown in FIG. The time stamp of the part 227 and the recorded character list are stored. In addition to dividing the voice character for each sentence, it may be divided by time, or an appropriate division method such as dividing by an appropriate sentence length may be used. Further, the server 11 searches the keyword 47 registered in FIG. 4 for each recording character list, and if the keyword 47 is included, stores the word in the keyword 143 of the recording analysis data in FIG. This operation is performed until the voice character is completed, and the recording analysis data is completed (1710). FIG. 9 shows a flowchart of recording analysis data creation. Since each process is as described above, a description thereof will be omitted.

  As shown in the example of FIG. 12, the server 11 links the above-described recorded audio file, operation data, and recording analysis data, and stores them as conference reproduction data in the conference reproduction data storage unit 224 of FIG. For example, the identification information and storage location of each data may be stored. Further, as shown in FIG. 5, the conference material and the material analysis data are managed for each conference, and the conference C can be reproduced from any of the terminals 13 and 14 in FIG. In addition, the reproduction is synchronized with other terminals in the same way as the document sharing, and it is also possible to reproduce a meeting or a workshop on a plurality of terminals all at once. Let it be a reproduction C file (123 in FIG. 12).

(Meeting reproduction)
FIG. 18 is a diagram showing a sequence of a conference reproduction flow. The reproduction file (123 in FIG. 12) created by the document sharing conference as described above with reference to FIG. 17 is reproduced on the terminal (reproduction terminal, second terminal) 15, and the user of the terminal 15 is viewing the conference material. Explain how to reproduce a meeting to hear the recorded audio. The terminal 15 may be the presenter terminal 13 or the terminal 14.
The reproduction of the conference can be performed by the same device configuration as in FIG. Here, at least one terminal is sufficient. In FIG. 18, the terminal 15 that wishes to reproduce the conference displays the screen shown in FIG. 15, and upon detecting that the conference reproduction 155 has been pressed, the server 11 requests the conference reproduction file list (18-1). Send. The server 11 creates conference reproduction file list data based on the conference reproduction data storage unit 122 of FIG. 12, and transmits the conference reproduction list data to the terminal 15 (18-2). The meeting reproduction file list data can include, for example, a meeting name and a reproduction file name stored in the meeting reproduction data storage unit 122.
The terminal 15 displays the screen of FIG. 16A based on the received conference reproduction file list data, and displays the conference reproduction file list in the area 166, for example. Next, for example, when the terminal 15 detects that the reproduction C file is designated and the start 163 in FIG. 16A is pressed, the terminal 15 transmits a message requesting the reproduction C file (18-3) to the server 11. The server 11 identifies the audio file, operation data, and recording analysis data corresponding to the requested reproduction file with reference to the meeting reproduction data storage unit 122. The server 11 reads the recorded audio file of the conference to be reproduced from the recorded audio storage unit 225 and transmits it to the terminal 15 (18-4). The terminal 15 stores the audio file in the storage unit 32 (recording unit) of FIG. It is stored in the voice storage unit 321). Note that the recorded audio file includes a time stamp and audio data.

  The server 11 monitors the time stamp 131 of the operation data in FIG. 13 and performs processing according to the operation event 132. For example, in the operation event 1321, the server 11 transmits one page of the information material C1, which is the first page of the material, to the terminal 15 (18-5). The terminal 15 displays an image of the document on the area 166 of the terminal screen in FIG. 16A based on the received information. The server 11 creates data (recording list data) necessary for the recording list that matches the displayed material page from the recording analysis data of FIG. 14 associated with the requested reproduction C file, and stores it in the terminal 15. Transmit (18-6). Here, in the recording list data, the keyword of the currently opened material page is specified from the material analysis data in the material analysis data storage unit 223 (63, 66, 69 in the figure), and the sound recording analysis of FIG. 14 is performed based on the keyword. It is obtained by searching the data keyword 143 and extracting the recorded character list 142 and the time stamp 141 corresponding to the matched keyword. In one example, one or more extracted recorded character lists 142 may be transmitted to the terminal 15. If it cannot be sent all at once, it may be sent separately. At this time, the terminal may be controlled including the scroll display data.

Based on the received recording list data, the terminal 15 displays the characters of each list (for example, recording character list) 162 in the recording list 161 on the terminal screen of FIG. Further, the terminal 15 starts reproduction of the audio file stored in the storage unit 32 of FIG. At this time, there may be a plurality of terminals synchronized with the terminal 15.
FIG. 10 is a flowchart when the terminal 15 displays the material, receives the data necessary for the recording list, and displays it on the screen for the first time. When it cannot be displayed on the screen, a scroll mark is displayed (scroll display). Since the operation is as described above, the description is omitted.
During playback, the server 11 monitors the operation data shown in FIG. 13 and prepares for an operation event. When the operation event 132 of the server 11 occurs, the server 11 transmits a message, a material, and the like corresponding to the event to the terminal 15, and the terminal 15 displays the material accordingly. For example, if the operation event is a two-page display of the document C2, the server 11 transmits the second page (18-7) of the document C2 to the terminal 15. In addition, the server 11 extracts necessary data from the recording list on the second page of the material C2 and transmits it (18-8). As described above, the server 11 detects the operation event 132, and if detected, transmits the display material data and the recording list message corresponding to the operation event 132 to the terminal 15, and the terminal 15 displays the data on the screen based on this data. Is displayed. The meeting is reproduced by repeating this process.
Here, when reproducing a meeting or the like, there is a case where it is desired to skip audio or the like that is not related to the material. In that case, assuming a place where the user wants to hear from the recording list 162 on the screen of the terminal 15 in FIG. 16A, for example, when the recording character list 162 of the list 161 is selected, the terminal 15 selects the selected recording list 162. The designated message (18-9) is transmitted to the server 11. The server 11 refers to the recording analysis data 141 of FIG. 14 to identify the time stamp of the selected recording character list and transmits a message (18-10) including the identified time stamp to the terminal 15. The terminal 15 detects a playback position based on the received time stamp, and plays back (jumps) from the corresponding audio position. The recording list transmitted by the server 11 may include a recording character list and a corresponding time stamp. In this case, when the recording character list is selected, the terminal 15 identifies the corresponding time stamp on its own terminal. Then, the audio data may be reproduced. In this way, it is possible to skip aside talks and recordings that are not related to a conference or a workshop, or to select a voice at a pinpoint.

Furthermore, there are cases where the recorded content played back does not necessarily match the content of the displayed material even after listening to the entire recording list. In that case, the user presses the key of the candidate 165 on the screen in FIG. When the terminal 15 detects that the candidate 165 has been pressed, the terminal 15 transmits a candidate list request (18-11) to the server 11. In response to the candidate list request from the terminal 15, the server 11 extracts the keyword of the displayed page with reference to the material analysis data of FIG. 6, searches the keyword 143 of the recording analysis data of FIG. 14 based on the keyword, A candidate list is created based on the matching recorded character list 142.
FIG. 19 shows an example of the contents of the candidate list. The candidate list includes a keyword included in the displayed page, a recording character including the keyword, a time stamp corresponding to the recording character, and information on the material name and page displayed at the time of the time stamp. Including. In addition, you may attach the identification information which identifies each candidate. For example, as for the candidate for the second page of the document C2 of the conference C, the keyword word1 has two places with time stamps 0:05:40 and 0:32:11, and the respective character lists become data (194). There are three keyword words 2 of time stamps 0:23:32, 0:48:40, and 0:51:32, and each character list becomes data (194). At that time, scroll information may be added as shown in FIG.
The candidate list created in the server 11 in this way is transmitted to the terminal 15 (18-12). The terminal 15 displays the other candidate list 169 of the reproduction screen shown in FIG. 16B based on the received candidate list information. As described above, if there are recorded portions close to the content of the page of the material displayed in the area 166 of FIG. 16A, a candidate list for displaying them as an audio list can be displayed.

FIG. 11A shows an outline of the processing flow of the server 11 and the terminal 15 when another candidate key is pressed. Since the operation is as described above, the description is omitted.
With this candidate list, the user reviews the voice he / she wants to hear (or related material he / she wishes to refer to), and selects and pushes the candidate list 169 of FIG. 16-2, which is one of the candidate lists. When the terminal 15 detects that the candidate list has been pressed, the terminal 15 transmits a designation message (18-13) of the selected candidate list 169 to the server 11. For example, the identifier of the selected candidate may be transmitted. The server 11 first obtains a time stamp corresponding to the designated candidate list. For example, the candidate list of FIG. 19 may be stored in an appropriate storage unit, and the time stamp corresponding to the specified candidate identifier may be obtained. Alternatively, the recording analysis data may be searched to obtain a corresponding time stamp. The server 11 transmits a message (18-14) as information on where the terminal 15 reproduces the obtained time stamp value from the voice. The terminal 15 detects the position of the audio data to be reproduced based on the information, and performs reproduction from the corresponding position. The time stamp may be specified on the terminal side as described above.

The server 11 further specifies the keyword 143 of the recording character list corresponding to the recording analysis data of FIG. 14 based on the candidate list 169 pressed above, and searches the keyword 62 of the material analysis data of FIG. If they match, the material and page display information are transmitted to the terminal 15. For example, the corresponding page of the corresponding material may be transmitted, and when the information has already been transmitted, the identification information of the material and the page may be transmitted. For example, in FIG. 18, assuming that the third page of the material C2 matches, the server 11 transmits the data information of the third page of the material C2 to the terminal 15 (18-15). The terminal 15 receives these pieces of information, reads the data on the third page of the material from the server 11, and displays the material page superimposed on the previous material page 166 as shown in an area 167 of FIG. The display method may be other appropriate methods. In addition to selecting the recording list 169 displayed in this way and listening to the sound, it is also possible to display a material screen linked to the sound with a keyword.
FIG. 11B shows an outline of the processing flow of the server 11 and the terminal 15 when the candidate list is designated. Since the operation is as described above, the description is omitted.
The end of reproduction is automatically ended by an event of the operation data 132, but the terminal 15 detects that the stop 164 of FIG. 16-2 is pressed, and transmits a reproduction end message (18-17) to the server 11. It is also possible to do. The server 11 receives it, returns an end notice (18-18), and ends.
In this way, those who mainly attended meetings and workshops, and who could not attend the meeting, not only reproduced the meeting and workshop materials and audio as if they were there, but also recorded audio. It is possible to provide a system that efficiently reproduces and understands the contents of conference and workshop materials by devising how to let the data be presented and how to display the materials.

  The present invention can be used in a computer system that records and reproduces materials and sounds such as conferences and seminars.

11 server 12 access point 13 terminal (presenter)
14 Terminal n
15 Reproduction terminal 21 CPU of server
22 Server storage unit 23 Server input device 24 Server output device 25 Server communication line interface LAN
31 Terminal CPU
32 Terminal storage unit 321 Recording audio recording file 322 Terminal application 33 Terminal input device 331 Microphone 34 Terminal output device 35 Terminal communication line interface LAN
41 Material Registration Key on Material Registration Screen 42 Automatic Keyword Registration Start Key 43 on Material Registration Screen Manual Keyword Input Position 44 on Material Registration Screen Manual Keyword Search Key 45 on Material Registration Screen 46 Keyword Delete Key 46 on Material Registration Screen Material on Material Registration Screen Additional key 47 Material keyword display section 151 of document registration screen Recording list display section 152 of terminal reproduction screen Recording list key 153 of terminal reproduction screen Candidate recording list key 154 of terminal reproduction screen Material of reproduction screen of terminal Display unit 155 Terminal reproduction screen reproduction start key 156 Terminal reproduction screen reproduction stop key

Claims (9)

A plurality of terminals including a first terminal and a second terminal and a server are provided, and in a recording stage, a material displayed on the first terminal, an operation event specifying the operation of the first terminal, and audio data are used as a time stamp. Correspondingly recorded on the server, a material display and sound reproduction system for reproducing the sound and displaying the material on the second terminal in the reproduction stage,
The server
A material analysis data storage unit in which keywords in pre-registered materials are stored;
A recorded voice storage unit for storing recorded voice data;
A recording character list in which the recorded voice data is converted into text, a recording analysis data storage unit stored corresponding to the time stamp, and
An operation data storage unit in which the material displayed on the first terminal and the operation event for specifying the operation of the first terminal are stored corresponding to the time stamp;
A processing unit that reproduces the audio data stored in the recorded audio storage unit and the display of the material according to the operation event stored in the operation data storage unit on the second terminal;
In the reproduction stage, the processing unit
The material keyword reproduced and displayed on the second terminal is identified with reference to the material analysis data storage unit,
Search the recording analysis data storage unit based on the identified keyword, extract a recording character list including the keyword, and a corresponding time stamp,
Transmitting a recording list including the extracted one or more recording character lists to the second terminal;
The second terminal is
When the received recording list is displayed and one of the recording character lists included in the displayed recording list is selected, the sound corresponding to the selected recording character list is displayed according to the time stamp corresponding to the selected recording character list. Data display and audio reproduction system for reproducing data. Further comprising a material storage unit in which data of materials to be displayed on the plurality of terminals is registered in advance,
The recording analysis data storage unit stores the keyword included in the recording character list in correspondence with the recording character list,
In the reproduction stage, the processing unit
Referring to the recording analysis data storage unit, identify a keyword corresponding to the selected recording character list,
Refer to the material analysis data storage unit based on the identified keyword, identify the material containing the keyword,
The material display and sound reproduction system according to claim 1, wherein data of the material is transmitted to the terminal for display. The processing unit, in the recording stage,
Receiving the voice data from one of the plurality of terminals storing the voice data, storing the voice data in the recorded voice storage unit;
A recorded character list in which the stored voice data is converted into text by voice recognition is created, and the recorded character list is searched based on the keyword stored in the material analysis data storage unit, and the recorded character list is recorded for each sentence or every predetermined time. 3. The material display according to claim 2, wherein a keyword included in the list is obtained, and a time stamp, a recorded character list for each sentence or every predetermined time, and the keyword are stored in the recording analysis data storage unit in association with each other. Voice reproduction system. In the reproduction stage, the processing unit transmits in advance the voice data stored in the recorded voice storage unit and a time stamp to the second terminal,
The material display and sound reproduction system according to claim 1, wherein a time stamp corresponding to the recording list designation message transmitted from the second terminal is returned by selecting one of the recording character lists. The recording list transmitted to the second terminal further includes a time stamp corresponding to the recording character list,
The material display and sound reproduction system according to claim 1, wherein when the second terminal selects one of the recorded character lists, the second terminal reproduces sound data according to a corresponding time stamp. The processor is
The material display and sound reproduction system according to claim 1, wherein the material data is inputted and stored, a predetermined keyword is extracted from the material data for each page of the material, and is stored in the material analysis data storage unit.   The material display and sound reproduction system according to claim 1, wherein the processing unit further inputs a material name and uses the material name or a word included in the material name as a keyword.   The material display and sound reproduction system according to claim 1, wherein the processing unit uses a word that appears frequently in the material data as a keyword. In the recording stage, the first terminal
Voice data is input and stored from the voice data input unit, and an operation event of the terminal is transmitted to the server.
The processing unit of the server is
Store the received operation event in the operation data storage unit corresponding to the time stamp,
The material display and sound reproduction system according to claim 1, wherein the received sound data is stored in the recorded sound storage unit.

Images