JP7176188B2 - Information generation system, information generation method, information processing device, program - Google Patents

Description

The present invention relates to an information generation system, an information generation method, an information processing device, and a program .

Various types of meetings are held in corporations, groups, and the like in which a plurality of people work together, and issues are extracted, decisions are made, and information is shared among related parties. It is generally believed that meetings are effective because they clarify action guidelines, schedules, and the like.

However, the number of meetings often puts pressure on normal work, and there is a latent demand for fewer meetings among the parties concerned. In addition, in reality, "meetings with similar content are often repeated", and one of the reasons for this is that officials from a certain department of the same corporation or organization say "It is difficult to incorporate knowledge from meetings of other departments." ”.

After the meeting, the minutes are generally distributed to the concerned parties. Minutes summarize the main points of the meeting, such as the purpose, agenda, conclusion, and action items of the meeting. be done. However, at meetings, opinions are often exchanged on topics other than the agenda items listed in the minutes. In many cases, the exchange of opinions on topics other than this agenda is important, such as the background and reasons for decisions, but detailed discussions are not recorded in the minutes. In addition, there are cases where the minutes of meetings cannot be viewed or distributed to anyone other than those related to the department. I cannot meet your request.

For this reason, as described above, "meetings with similar content will be repeated", but by using the information in the minutes of meetings of other departments, the participants of the meeting It is expected that the number of meetings can be reduced if similar knowledge can be shared.

As a technique for reducing the trouble of creating minutes, a technique for recognizing speech during a conference, converting it into text data, and automatically generating minutes is known (see, for example, Patent Documents 1 and 2). ). As described in Japanese Patent Application Laid-Open No. 2002-200012, if minutes are generated from texts obtained by speech recognition, it is possible to relatively easily record topics other than agenda items in the minutes. In addition, in Patent Document 2, as in Patent Document 1, voice recognition during a meeting is converted into text data, and a summary is created by linking the text data with automatically classified categories. Disclosed is a technique for generating minutes (summary) from a different point of view from the formal minutes created by .

However, the conventional technology has a problem that it is not assumed that the conference participants and non-participants utilize the minutes for the purpose of sharing knowledge. In other words, it is thought that the minutes have an appropriate format depending on the person who uses them. For example, if non-participants of the meeting use the minutes to reduce the number of meetings, the minutes are generated in a format that allows the non-participants to grasp the details of the meeting in a format suitable for searching. Should. On the other hand, if the participants of the meeting use the minutes to check the summary of the meeting, the minutes should be generated in a format that concisely describes the key points of the meeting.

For example, in the technology disclosed in Patent Document 1, since the minutes creator edits the text data via the user interface, the content of the minutes finally created is changed according to the intention of the minutes creator. . Therefore, there is a risk that information important to persons other than those involved in the meeting will be omitted from the minutes. The minutes may be created in a format that is suitable for those involved in the meeting to understand the main points, but it is not suitable for the purpose of sharing knowledge to the same extent as the participants of the meeting. In other words, in the technique of Patent Document 1, text data converted from voice information uttered during a conference is utilized not only by persons concerned in the conference but also by persons other than persons concerned in the conference for the purpose of sharing knowledge. Not expected.

Similarly, the abstract created by the technique disclosed in Patent Document 2 merely automatically classifies text data, and if there are many voices uttered during a meeting, The text data contains a lot of unnecessary information for the conference participants. For example, it cannot be said that the format is suitable for applications where you want to grasp only the main points such as decisions and action items. In addition, even if the information processing device deletes unnecessary information for conference participants when creating a summary, the search for the summary is intended for use by conference participants. important information may be lost. In other words, even with the technique of Patent Document 2, when both parties involved in the conference and parties other than the parties involved in the conference want to utilize the information of the voice uttered during the conference, minutes are created in a format suitable for both parties. not expected to do so.

SUMMARY OF THE INVENTION In view of the above problems, it is an object of the present invention to provide an information generation system that enables participants and non-participants to utilize meeting minutes for the purpose of sharing knowledge.

The present invention provides an information generation system for generating information about data transmitted or received via a communication session of a conference established between a plurality of terminal devices, the plurality of terminal devices participating in the communication session. text data acquisition means for acquiring text data by recognizing the speech data acquired in any of the above; and data transmission for transmitting the text data acquired by the text data acquisition means to one of the plurality of terminal devices participating in the same communication session via a network. having means and
Any one of the plurality of terminal devices has data receiving means for receiving the text data from the device that generated the text data via a network, and displaying the text data received by the data receiving means on a display screen. display control means; and extraction means for extracting part of the displayed text data according to an input received from a user on the display screen;
The information generation system further generates first minutes data to be transmitted to participants who have participated in the communication session, based on the part of the text data extracted by the extraction means. means, a first providing means for providing information on the generated first minutes data to users of terminal devices participating in the communication session, and a period from the start of the communication session to the end of the communication session. A person other than the participants who participated in the communication session in order to search the second minutes data generated by putting together the plurality of text data stored in the storage means during the A second providing means for transmitting the second minutes data to an in-house search system capable of searching , and generating the second minutes data based on an instruction from a user of a terminal device participating in the communication session a second minutes generating means for determining whether or not to generate the generating the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means; and the second minutes generation means causing the display control means to display The second provision means generates the second minutes data that enables utilization of the text data, and the first providing means transmits the first minutes data to the user of the terminal device participating in the communication session. the second provision means registers the second minutes data in the search system; and the second minutes generation means generates the second minutes data based on the attributes of the users of the plurality of terminal devices. It is determined whether or not to generate the second minutes data, and if it is determined not to generate the second minutes data, the second minutes generating means does not generate the second minutes data .

It is possible to provide an information generation system in which the conference participants and non-participants can utilize the minutes for the purpose of sharing knowledge.

1 is an example of a diagram illustrating a schematic configuration of a conference system; FIG. It is a figure which shows an example of the conference screen displayed on the terminal device which a participant operates. 1 is an example of a schematic configuration diagram of a conference system; FIG. 1 is an example of a hardware configuration diagram of a terminal device, a conference server, a speech recognition server, and a search system; FIG. 1 is an example of a functional block diagram showing functions of a terminal device, a conference server, and a speech recognition server included in a conference system in block form; FIG. FIG. 4 is an example of a functional block diagram showing functions of a text conversion unit in block form; FIG. 10 is an example of a sequence diagram illustrating processing in a phase in which a host registers a conference in the conference server; It is a figure which shows an example of the conference screen which a terminal device displays. FIG. 10 is a diagram showing an example of a conference screen on which the host has entered information about the conference; FIG. 10 is an example of a sequence diagram illustrating processing in a phase in which the conference server sends conference e-mails to participants; It is an example of the figure explaining the meeting screen at the time of transmission of an e-mail. FIG. 10 is an example of a sequence diagram illustrating processing in a phase in which the terminal device transmits voice data to the conference server and displays text data as needed; It is a figure which shows an example of the conference screen explaining correspondence of an action item and a participant. It is a figure which shows an example of the conference screen explaining correspondence of an action item and a participant. FIG. 10 is a diagram showing an example of a conference screen for explaining content switching; FIG. 10 is an example of a sequence diagram illustrating operations when the conference server generates minutes; It is an example of a meeting screen for explaining generation of minutes. FIG. 10 is an example of a flowchart diagram showing a procedure for the minutes generating unit to generate the first minutes and the second minutes. 1 is an example of a schematic configuration diagram of a search system; FIG. It is an example of the figure explaining the reply to the inquiry of a search system. It is an example of the flowchart figure explaining the procedure which a search system answers with respect to an inquiry. It is a figure which shows an example of the screen for a search. It is a figure which shows the modification of the structural example of a conference system.

Hereinafter, the conference system 100 and the information generation method performed by the conference system 100 will be described as an example of a mode for carrying out the present invention.

<Outline of conference system according to the present embodiment>
A system configuration of the conference system 100 will be described with reference to FIG. FIG. 1 is an example of a diagram for explaining a schematic configuration of a conference system 100, and FIG. 2 shows an example of a conference screen 501 displayed on a terminal device operated by a participant. In the conference system 100, the conference server 30 and each participant's terminal device 10 communicate.
(1) A participant's utterance is input as voice data to the terminal device 10 held by the participant, and transmitted to the conference server 30 together with information about which participant's utterance.
(2) The conference server 30 transmits voice data to the voice recognition server 50 .
(3) The speech recognition server 50 converts the speech data into text data and transmits the text data to the conference server 30 .
(4) The conference server 30 transmits the Web page in which the text data is reflected on the conference screen 501 to all or some of the terminal devices 10 . In this way, each terminal device 10 operated by a participant can display the speech of each participant on the terminal device 10 in real time during the conference.

Next, the conference screen 501 will be described. Browser software is running on the terminal device 10, and a conference screen 501 of a web application provided by the conference server 30 is displayed. Since the details of the conference screen 501 will be described later, an outline thereof will be described with reference to FIG. The conference screen 501 is roughly divided into a text display portion 502 and a content display portion 503. The text display portion 502 displays the purpose of the conference, agenda, action items, participants, and conference content. Contents (materials) referred to in the conference are displayed in the content display section 503 .

At the end of the conference, the minutes are generated based on the text data and contents referred to by the participants. In this embodiment, a plurality of types of minutes are generated corresponding to persons who are expected to use the minutes. Participants and non-participants are an example of persons who are expected to use the minutes.

First, considering minutes that are easy for participants to use, what participants need is a summary of the meeting. In other words, the participants of the conference know the details of the discussions held at the conference and do not need the detailed information of the conference. For example, it is sufficient to know what action items were decided for what purpose and agenda.

On the other hand, considering minutes that are easy for non-participants to use, non-participants would like to obtain knowledge from the minutes of meetings in which they did not participate and share knowledge and information. On the other hand, I don't want to share the minutes of meetings that I can't use even if I share knowledge and information. In other words, people other than the participants of the conference want to acquire only the information they need without excess or deficiency.

As described above, the conference server 30 generates two minutes at the end of the conference.
First Minutes 9: Meeting Summary (Example of First Minutes Data)
Second minutes 8: Detailed proceedings that enable sharing and utilization of knowledge (an example of second minutes data)
The first minutes 9 are mainly used to clarify the gist of the meeting and follow-up matters generated by the results of the meeting and follow-up persons, and to notify relevant parties. In FIG. 2 the purpose, agenda, action items and participants are recorded in the first minutes 9 . The second minutes 8 record all of the contents of the meeting, and enable people other than the participants to search for the knowledge created at the meeting to enable sharing and effective use of knowledge. In FIG. 2, the second minutes 8 contains the entire information displayed in the large text display area 502 and the content display area 503, including the first minutes.

The second minutes 8 are stored in a search system 60 equipped with a search function for efficiently obtaining information. Artificial intelligence is a technology that enables computers to perform intelligent processing similar to humans. By storing the second minutes 8 in which the entire information of the conference is recorded in the search system 60, it becomes possible for persons other than the participants to efficiently search for necessary information. As the number of second minutes 8 stored in the search system 60 increases, the resources that the search system 60 can use for answering also increase, and it becomes possible to provide more appropriate answers (search results) to inquiries.

As described above, the conference system 100 of the present embodiment generates a plurality of minutes in a format that is easy for users to use, so that various users can effectively use the minutes. For example, the participants of the conference can confirm follow-up matters and followers determined at the conference, and persons other than the participants can obtain only the necessary information by inquiring the search system 60 .

<Terms>
A participant is a person who participates in a conference, and more specifically, a person who can log in to the conference server 30 and make a statement. Even if you don't attend the conference in the end, you just need to be qualified to participate. The organizer is an administrator who registers the conference with the conference server 30 . The organizer is often one of the participants.

A user is a person who obtains knowledge from the minutes, and a user may include a person who is a participant or an organizer.

Data shared by a plurality of terminal devices in the claims refers to data displayed by a plurality of terminal devices 10 . In this embodiment, a conference is assumed as a scene in which data is shared in this way, but this scene may be called a meeting, discussion, meeting, review meeting, presentation meeting, presentation meeting, or the like.

Also, the scene in which the data is shared may be a lecture. In this case, the speech of the sponsor is shared by the terminal devices 10 of the audience. Questions and answers from the audience and supporters will also be shared.

Also, data summaries or summary information refers to a summary of the shared data rather than all of the data. In this embodiment, the term "first minutes" will be used. Further, the information to be searched means a collection of various data shared by a plurality of terminal devices 10 . In this embodiment, the term "second minutes" will be used. Minutes may also be referred to as meeting minutes, meeting records, overall summaries, conversation histories, and the like.

<System configuration example>
FIG. 3 is an example of a schematic configuration diagram of the conference system 100 of this embodiment. As shown in FIG. 3(a), the conference system 100 has one or more terminal devices 10, a conference server 30, and a voice recognition server 50 that can communicate via a network N. As shown in FIG. Since the search system 60 can be operated independently of the conference system 100 , it may or may not be included in the conference system 100 .

In the conference system 100 of FIG. 3(a), terminal devices 10A and 10B (hereinafter, any terminal device is denoted by 10) in the conference room 7 are connected to the in-house network N1. 50 is connected to the external network N2. Further, there are cases where a conference is held including terminal devices 10X at other bases connected to the network N2 outside the company. The terminal device 10X at another base may exist anywhere other than the internal network N1, but may be, for example, a branch of the same company as the company where the conference room 7 is located, or a building at a business trip destination.

In FIG. 3A, devices (terminal device 10, conference server 30, speech recognition server 50, search system 60) are connected one-to-one for explanation of the system configuration, but each device is connected to a network. there is In addition, the network is a LAN constructed in a facility where the conference room 7 exists, a wide area Ethernet (registered trademark), a WAN in which LANs are connected, a telecommunications carrier's telephone network, and a part or the whole of the Internet. is constructed from

The terminal device 10 is an information processing device that operates as a client terminal of the conference server 30 in this embodiment. In the terminal device 10, browser software or application software having equivalent functions is running, and the web page requested by the terminal device 10 is received from the conference server 30 and displayed on a display (LCD: Liquid Crystal Display, which will be described later). Let

The terminal device 10 is, for example, a PC (Personal Computer), a tablet device, a smart phone, a PDA (Personal Digital Assistant), a game machine, a navigation terminal, a wearable PC, or the like. I wish I could. In this sense, the terminal device 10 can be a device having the function of an information processing device. For example, the terminal device 10 may be an electronic blackboard, a digital signage, a multifunction device, a teleconference terminal, a projector, or the like.

Note that, in the present embodiment, the term "web page" includes a web page appropriately constructed by a web application. A web application is software or its mechanism that operates through cooperation between a script language (such as JavaScript (registered trademark)) program that runs on a web browser and a program on the web server side, and that is used on the web browser. .

The conference server 30 is an information processing device that provides information and functions to the terminal device 10 through the network. The conference server 30 provides the terminal devices 10 with Web pages written in HTML, script language, and CSS (Cascade Style Sheet). The conference server 30 generates a web page for each conference and transmits the same web page (or only some elements of HTML data) to the terminal devices 10 participating in the same conference. Therefore, the terminal devices 10 of the participants participating in the same conference display the conference screen 501 containing the same information (it is possible to color-code the remarks of the participant and the other participants, and the appearance may be different). ).

In order to synchronize the conference screens 501 of a plurality of terminal devices 10, the conference server 30 and the terminal devices 10 communicate using a two-way communication protocol such as WebSocket. As a result, the participant's remarks and the participant's operation on the terminal device 10 are transmitted to the conference server 30, and the conference screens 501 of the other terminal devices 10 can be synchronized in real time.

Note that the conference server 30 of the external network N2 may be compatible with cloud computing. Cloud computing is a form of usage in which resources on a network are used without being conscious of specific hardware resources. When cloud computing is supported, the physical configuration of the conference server 30 of this embodiment does not have to be fixed, and hardware resources can be dynamically connected/disconnected according to the load and the like. may consist of

The voice recognition server 50 is an information processing device that provides information and functions to the conference server 30 through the network. In this embodiment, speech recognition is applied to voice data transmitted from the conference server 30 to convert it into text data, and the text data is transmitted to the conference server 30 . The conference server 30 generates a web page containing this text data. The speech recognition server 50 publishes a WebAPI, and the conference server 30 requests the WebAPI of the speech recognition server 50 for processing. The WebAPI is a mechanism for making agreements (such as a method of requesting processing and specifying data) when information processing apparatuses perform processing via a network, or a mechanism for connecting information processing apparatuses through communication. It may simply be called an “IF (interface)”. If the speech recognition server 50 also exists in the network N2 outside the company, it may be compatible with cloud computing.

3B and 3C show another system configuration example of the conference system 100. FIG. In FIG. 3B, the conference server 30 is connected to the in-house network N1, and such a configuration of the conference server 30 is called on-premise. The conference server 30 may thus exist in the internal network N1 or may exist in the external network N2.

In FIG. 3(c), the conference server 30 and the voice recognition server 50 are connected to the company network N1. The speech recognition server 50 may thus exist in the company network N1 or may exist in the external network N2. However, in cases such as when the bandwidth of the network N2 outside the company is low, it may be preferable for the voice recognition server 50 to exist on-premises in order to improve real-time performance of voice recognition.

3B and 3C, the terminal device 10 can communicate with the conference server 30 from the external network N2. In this case, as an example, there is a method in which the terminal device 10 communicates with the conference server 30 of the in-house network N1 through a firewall using a PVN (Private Network). Further, the search system 60 may be arranged in either the in-house network N1 or the external network N2.

<Hardware configuration example>
FIG. 4 is an example of a hardware configuration diagram of the terminal device 10, the conference server 30, the speech recognition server 50, and the search system 60. As shown in FIG. In the description of FIG. 4, the hardware configuration will be described with the terminal device 10 in mind. Terminal 10 may generally be implemented as a personal computer or workstation. The terminal device 10 includes a CPU 201 and a memory 202 that enables high-speed access to data used by the CPU 201 . CPU 201 and memory 202 are connected via system bus 203 to other devices or drivers of terminal device 10 , such as graphics driver 204 and network device (NIC) 205 .

A graphics driver 204 is connected to an LCD (display device) 206 via a bus to monitor processing results by the CPU 201 . A touch panel may be integrated with the LCD 206 . In this case, the participant can operate the terminal device 10 using a finger as an operation means.

Also, the network driver 205 connects the terminal device 10 to the network N at the transport layer level and the physical layer level to establish a communication session with the conference server 30 and the like.

Also connected to system bus 203 is I/O bus bridge 207 . On the downstream side of the I/O bus bridge 207, via an I/O bus 208 such as PCI, IDE, ATA, ATAPI, serial ATA, SCSI, USB (Universal Serial Bus), etc., a storage device such as an HDD 209 is connected. An SSD (Solid State Drive) may be provided instead of or together with the HDD 209 .

4 is the terminal device 10, the HDD 209 stores a program 209p for controlling the terminal device 10 as a whole. The program 209p includes web pages transmitted by the conference server 30 in addition to browser software. 4 is the conference server 30, the HDD 209 stores a program 209p for controlling the conference server 30 as a whole. The program 209p includes a function of performing communication processing such as HTTP or HTTPs as a web server and a function of a web application for constructing the conference screen 501 . 4 is the voice recognition server 50, the HDD 209 stores a program 209p for controlling the voice recognition server 50 as a whole. 4 is the retrieval system 60, the HDD 209 stores a program 209p for controlling the retrieval system 60 as a whole.

An input device 210 such as a keyboard and a mouse (referred to as a pointing device) is connected to the I/O bus 208 via a bus such as USB, and receives inputs and commands from operators such as participants.

When the configuration shown in FIG. 4 is for the conference server 30 and the voice recognition server 50, the LCD 206 and the microphone 211 may be omitted. The microphone 211 may be built in the terminal device 10 or may be externally attached via a USB or the like.

<About functions>
FIG. 5 is an example of a functional block diagram showing functions of the terminal device 10, the conference server 30, and the voice recognition server 50 included in the conference system 100 in block form. Although only one terminal device 10 is shown in FIG. 5, a plurality of terminal devices 10 may exist.

<<terminal device>>
The terminal device 10 includes a web page analysis unit 11, a display control unit 12, an operation reception unit 13, a voice acquisition unit 14, a voice data transmission unit 15, a web page acquisition unit 16, an operation content transmission unit 17, and a login request unit 18. have Each of these functions possessed by the terminal device 10 is a function or means implemented by the CPU 201 executing the program 209p developed in the memory 202 from the HDD 209 .

The login request unit 18 requests login to the conference server 30 by connecting to a URL (Universal Resource Locator) predetermined for each conference and transmitting authentication information. As the authentication information, the conference password accepted by the operation accepting unit 13 is transmitted to the conference server 30 . A password is determined at least for each conference, and if the combination of the URL and password is correct, the participant can log in to the conference server 30 and participate in the conference. Further, the login requesting unit 18 transmits identification information (for example, user name) of the participant and a password unique to the participant to the conference server 30 as authentication information so that the participant can be identified. This allows the conference server 30 to identify each individual participant.

The voice acquisition unit 14 converts the voice collected by the microphone 211 into an electric signal and acquires it as voice data. When audio data is input from the microphone 211, the audio data is called stream data. Audio data may be stored in a file by the microphone 211, and the audio data may be read from the file.

It is assumed that one terminal device 10 is held by each participant, but the terminal device 10 may acquire voices of participants other than the participant in front. There are several methods to prevent this, for example, there is a method in which the participant presses a predetermined button when speaking. Alternatively, there is a method in which participants register their own voices in the terminal device 10 and do not acquire voices other than this voice. For voice registration, for example, Fourier transform is used. There is also a method of not acquiring the voices of participants other than the participants in front of the terminal device 10, on the premise that the terminal device 10 uses a highly preferred microphone.

The voice data transmission unit 15 repeatedly transmits the voice data for a predetermined time (for example, 20 milliseconds) acquired by the voice acquisition unit 14 to the conference server 30 . The predetermined time is set as a time advantageous for voice recognition. The audio data transmission unit 15 transmits to the conference server 30 using communication protocols such as WebSocket, AJAX (Asynchronous JavaScript + XML), COMET, HTTP, and HTTPs. Assume that the IP address of the conference server 30 is known in advance.

The web page acquisition unit 16 acquires the web page generated by the conference server 30 . When two-way communication is possible like WebSocket, the web page is transmitted from the conference server 30 by push communication without requesting the web page from the terminal device 10 . The conference server 30 transmits the web page when new text data is acquired. A trigger for requesting a web page from the terminal device 10 is when the operation content transmission unit 17 transmits an operation accompanied by a change in the conference screen 501 to the conference server 30 .

The web page analysis unit 11 reads HTML included in the web page in order from the beginning and analyzes the structure of sentences and image data included in the HTML. In addition, the style of the text and image data included in the HTML is determined by detecting the linkage with HTML text and image data described in the CSS. Also, the web page analysis unit 11 detects a script tag from HTML and extracts a script written in a script language. As a result, the script is executed in the terminal device 10, and when an operation event by the participant is detected, the processing described in the script is performed. The script accepts, for example, correspondence between action items and participants.

The display control unit 12 displays the conference screen 501 on the LCD 206 in the order in which analysis is completed from the top of the HTML. Also, the conference screen 501 is updated according to the operation on the conference screen 501 . The display control unit 12 has an extraction unit 12a, and the extraction unit 12a extracts part of the text data according to user's operation.

The operation content transmission unit 17 transmits the operation content to the conference server 30 . Since the operation content that accompanies the change of the conference screen 501 is transmitted to the conference server 30 , the conference screen 501 can be synchronized among the plurality of terminal devices 10 . The operation contents to be transmitted include correspondence between action items and participants, correspondence between agendas and conclusions, text data manually input by participants, and the like. However, operations that do not change the displayed information itself, such as operations related to the layout of the conference screen 501 (color scheme, character size, size of the text display section 502 and content display section 503, etc.), are not sent to the conference server 30. you can This allows each participant to display the conference screen 501 in an individual layout.

<<Meeting Server>>
The conference server 30 includes a text conversion request unit 31, a voice data reception unit 32, a web page transmission unit 33, an operation content reception unit 34, a text reception unit 35, a web page generation unit 36, a minutes generation unit 37, and an email transmission unit 38. , an authentication processing unit 39 , and a minutes transmission unit 43 . Each of these functions possessed by the conference server 30 is a function or means implemented by the CPU 201 executing the program 209p developed in the memory 202 from the HDD 209 .

The conference server 30 also has a storage unit 49 . The storage unit 49 is storage means realized by at least one of the HDD 209 and the memory 202 shown in FIG. A content DB (Data Base) 491 , a meeting information DB 492 , and a user information DB 493 are constructed in the storage unit 49 . First, these databases are explained.

表１はコンテンツＤＢ４９１に登録されている情報を模式的に示す。コンテンツＤＢ４９１は会議で使用される会議の資料が登録されたデータベースである。本実施形態ではコンテンツ表示欄５２１に表示される情報をコンテンツと称するため資料がコンテンツと呼ばれる場合がある。コンテンツＤＢ４９１には、会議ＩＤに対応付けてコンテンツが登録されている。会議ＩＤは会議を一意に識別又は特定する会議の識別情報である。なお、ＩＤはIdentificationの略であり識別子や識別情報という意味である。ＩＤは複数の対象から、ある特定の対象を一意的に区別するために用いられる名称、符号、文字列、数値又はこれらのうち１つ以上の組み合わせをいう。以下のＩＤについても同様である。表１では２つのコンテンツが登録されているが、コンテンツの数は０でもよいし３つ以上でもよい。コンテンツは、例えば、文書ファイル、プレゼンテーションファイル、ＰＤＦファイル、及びＨＴＭＬデータ等である。会議サーバ３０が開くことができればどのようなフォーマットのコンテンツでもよい。

Table 1 schematically shows information registered in the content DB 491 . The content DB 491 is a database in which conference materials used in the conference are registered. In this embodiment, the information displayed in the content display column 521 is called content, so the material may be called content. Content is registered in the content DB 491 in association with the conference ID. The meeting ID is meeting identification information that uniquely identifies or specifies the meeting. Note that ID is an abbreviation for Identification and means an identifier or identification information. An ID is a name, a code, a character string, a numerical value, or a combination of one or more of these used to uniquely distinguish a specific object from a plurality of objects. The same applies to the following IDs. Although two contents are registered in Table 1, the number of contents may be zero or three or more. Contents are, for example, document files, presentation files, PDF files, and HTML data. The content can be in any format as long as the conference server 30 can open it.

表２は会議情報ＤＢ４９２に登録されている情報を模式的に示す。会議情報ＤＢ４９２には、会議に関する各種の情報が登録されている。主に、会議画面５０１に表示される情報が登録されている。会議ＩＤはコンテンツＤＢ４９１と同じものである。目的は会議が開催された目的であり、会議の主催者等が入力する。議題は会議の中で討議される問題であり、会議の主催者等が入力する。アクションアイテムは成すべきことのリストであり、会議の進行中に参加者等が対応付けることで入力される。参加者は会議に参加する者である。会議の事前に登録されている場合と、会議中に随時、増大される場合がある。会議内容は、参加者の発言内容を時系列に記録したファイルである。表２ではテキストファイルに格納されているがＸＭＬ形式、Ｊｓｏｎ形式などでもよい。ＵＲＬは会議ごとに生成され当該会議に参加する際の各端末装置１０の通信先を示す。このＵＲＬは事前に各参加者に電子メールなどで送信される。パスワードは当該会議にログインするための認証情報となる。表２に記載した項目の他、会議の時刻（開始時刻、終了時刻）、参考にすべき他の会議の議事録等が登録されていてもよい。

Table 2 schematically shows information registered in the meeting information DB 492 . Various types of information about meetings are registered in the meeting information DB 492 . Mainly, information displayed on the conference screen 501 is registered. The meeting ID is the same as that of the contents DB 491 . The purpose is the purpose for which the conference was held, and is input by the organizer of the conference or the like. The agenda is an issue to be discussed in the meeting, and is entered by the organizer of the meeting. An action item is a list of things to be done, and is input by a participant or the like while the conference is in progress. A participant is a person who participates in a conference. It may be registered in advance of the meeting or may be augmented at any time during the meeting. The content of the conference is a file in which the content of remarks of the participants is recorded in chronological order. Although it is stored in a text file in Table 2, it may be in XML format, Json format, or the like. A URL is generated for each conference and indicates a communication destination of each terminal device 10 when participating in the conference. This URL is sent to each participant in advance by e-mail or the like. The password serves as authentication information for logging into the conference. In addition to the items listed in Table 2, meeting times (start time, end time), minutes of other meetings to be referred to, etc. may be registered.

表３はユーザ情報ＤＢ４９３に登録されている情報を模式的に示す。ユーザ情報ＤＢ４９３は会議に参加しうるユーザに関する各種の情報が登録されたデータベースである。ユーザ情報ＤＢ４９３にはユーザ名、パスワード、所属、メールアドレス、及び、情報階級の各項目が登録されている。ユーザ名は一意性が保証されたユーザの名称である。ユーザ名の代わりに又はユーザ名と共にユーザＩＤが対応付けられていてもよい。このユーザ名は参加者の識別に使用される。パスワードは各ユーザ別に登録されている認証情報である。社員証などのＩＣカードに関する情報が登録されていてもよい。所属はユーザが所属する部署である。メールアドレスはユーザに電子メールを送信するためのメールアドレスである。情報階級は情報へのアクセス権の大きさを示す。数字が大きいほど重要な情報にアクセスできる。

Table 3 schematically shows information registered in the user information DB 493 . The user information DB 493 is a database in which various types of information regarding users who can participate in the conference are registered. User name, password, affiliation, mail address, and information class are registered in the user information DB 493 . A user name is a name of a user whose uniqueness is guaranteed. A user ID may be associated with or in place of the user name. This username will be used to identify the participant. The password is authentication information registered for each user. Information about an IC card such as an employee ID card may be registered. Affiliation is the department to which the user belongs. A mail address is a mail address for sending an e-mail to the user. The information class indicates the degree of access rights to information. The higher the number, the more important information you can access.

The affiliation and information rank are used by the search system 60 to give an appropriate answer when a user makes an inquiry to the search system 60 . Also, the affiliation and information rank are used to control whether the conference server 30 stores the second minutes 8 in the retrieval system 60 .

(About functions)
The authentication processing unit 39 determines whether or not the participant has the authority to log in to the conference based on the password of the conference and the user name and password transmitted from the terminal device 10 . The conference server 30 can determine whether or not the conference password is qualified to participate in the conference, and the user name and password can identify the participants. A user name and password are not required if the participant is not specified. It should be noted that transmission of the conference password may be omitted. In this case, if the participant specified based on the user name and password is registered in the conference information DB 492, it is determined that the participant is qualified to participate in the conference. It should be noted that the conference server 30 may use the result of authentication by the authentication server instead of the conference server 30 for the authentication relating to the identification of the participants.

The voice data receiving section 32 receives voice data from the terminal device 10 and sends it to the text conversion requesting section 31 . In addition, in order to specify the speaker of the text data returned from the voice recognition server 50, the user name of the participant who transmitted the voice data is obtained together with the voice data. You may acquire the identification information of the terminal device 10 instead of a user name.

The text conversion request unit 31 transmits to the speech recognition server 50 a conversion request to text data together with the speech data and the user name in accordance with the WebAPI of the speech recognition server 50 . The text receiver 35 receives text data and a user name from the speech recognition server 50 and sends them to the web page generator 36 . The text data is appropriately stored in the storage unit 49,
The operation content reception unit 34 receives the operation content for the conference screen 501 from the terminal device 10 and sends it to the web page generation unit 36 .

A web page generator 36 generates a web page for the conference screen 501 . Since the conference screens 501 of the terminal devices 10 participating in the same conference may be the same, the web page generator 36 may create one web page regardless of the number of participants. The web page generator 36 has a text processor 41 and a content processor 42 . The text processing unit 41 has a function of generating a text display unit 502 in the conference screen 501, and is realized by a web application called a text application. The content processing unit 42 has a function of generating the content display unit 503 of the conference screen 501, and is implemented by a web application called a content application. The text processing unit 41 causes the text data received by the text receiving unit 35 to be reflected on the text display unit 502 , and causes the operation content received by the operation content receiving unit 34 to be reflected on the text display unit 502 . The content processing unit 42 opens the content used in the conference, converts it into image data, and arranges this image data on the content display unit 503 . The content displayed on the initial conference screen 501 may be only the list of content displayed on the content list 522 (a list of thumbnails, file names, creators, time stamps, etc. on the first page of content). When an arbitrary participant selects content and opens an arbitrary page, the content processing unit 42 acquires the page number and transmits image data of the corresponding page of the content to each terminal device 10 .

The web page transmission unit 33 transmits image data to each terminal device 10 as elements of the web page generated by the web page generation unit 36 and HTML data. By using WebSockets as described above, the Web page transmission unit 33 can transmit a Web page, image data, or the like without a request from the terminal device 10 .

Minutes generating unit 37 generates minutes. The minutes generator 37 has a first generator 37a and a second generator 37b. When the conference ends, the first generation unit 37a of the minutes generation unit 37 generates the first minutes 9 from the information about the conference registered in the conference information DB 492, and the second generation unit 37b generates the second minutes 8. Generate. The first minutes 9 are sent to the mail transmission section 38 and the second minutes 8 are sent to the minutes transmission section 43 .

The mail transmission unit 38 transmits the first minutes 9 to the participants of the conference. The participants of the conference are specified from the conference information DB 492, the e-mail addresses of the participants are obtained from the user information DB 493, and the first minutes 9 are sent as e-mail.

The minutes transmission unit 43 transmits the second minutes 8 to the search system 60 . It is assumed that the IP address and the like of the search system 60 are known in advance.

<<Voice Recognition Server>>
The speech recognition server 50 has a text conversion reception section 52 , a text conversion section 51 and a text response section 53 . Each of these functions possessed by the speech recognition server 50 is a function or means implemented by the CPU 201 executing the program 209p developed in the memory 202 from the HDD 209 .

The text conversion reception unit 52 receives from the conference server 30 a request for conversion into text data together with voice data and a user name. The text conversion unit 51 converts voice data into text data using the learning result. Details will be described with reference to FIG. The text response unit 53 transmits the text data converted from the voice data and the user name to the conference server 30 .

<<Search System>>
The search system 60 has a communication section 61 and a search section 62 . Each of these functions possessed by the search system 60 is a function or means implemented by the CPU 201 executing the program 209p developed in the memory 202 from the HDD 209 .

The communication unit 61 communicates various information with the conference server 30 . In this embodiment, the second minutes 8 are received. It also receives search words from the terminal device 10 and transmits search results (answers) to the terminal device 10 .

The search unit 62 searches the information DB 63 using search words received from the terminal device 10 . The information DB 63 stores information including the second minutes 8 divided into various cabinets (to be described later). The search unit 62 can search the information DB 63 in consideration of user attributes.

<About voice recognition>
FIG. 6 is an example of a functional block diagram showing functions of the text conversion unit 51 in block form. The text conversion unit 51 has a preprocessing unit 54, a feature extraction unit 12a55, and an identification calculation unit 56. The identification calculation unit 56 further uses an acoustic model 57 and a language model 58. FIG.

A preprocessing unit 54 performs processing on the audio data to facilitate feature extraction. For example, the audio data is quantized into fixed steps. Also, the preprocessing unit 54 performs processing such as converting the voice data into a frequency spectrum and removing the frequency portion corresponding to noise. The feature extraction unit 12a55 extracts information useful for pattern recognition from the voice data. Since the audio data is continuous data, the audio data is acquired at predetermined time intervals such as 20 milliseconds, and features are extracted at each predetermined time interval. In the case of voice data, for example, it is converted into a frequency spectrum and formants or MFCC (Mel Frequency Cepstrum Coefficient) are extracted. Since the method of obtaining the formant and MFCC is well known, the explanation is omitted. As a result, several dozen-dimensional feature vectors are obtained from the speech data.

The identification calculation unit 56 uses the acoustic model 57 and the language model 58 to convert the speech data into text data. The acoustic model 57 is obtained by associating the voice waveform with text data of the voice at that time. A speech waveform is represented by, for example, a set of phonemes called a triphone, which is a combination of three phonemes. The language model 58 is data that expresses the connection between words created from text data in terms of probability. For example, it has data on the probability that words such as "take", "acquire", and "give up" appear following "patent". The identification calculation unit 56 compares the feature vector of the input speech data and the acoustic data to estimate the triphone. This is continuously performed, and the word with the highest appearance probability is estimated while referring to the language model 58 . Since words cannot be uniquely determined depending on how similar the speech data is to the acoustic data and the appearance probability of the language data, multiple words are often estimated for the words of the input speech data. Also, the reliability is calculated based on the appearance probability.

Note that the speech recognition method described here is merely an example, and various modifications may be made, and speech recognition may be performed by a method unique to the speech recognition server 50 .

<Operation procedure>
In the following, operations performed by a participant or organizer regarding the conference system 100 and operations of the conference system 100 will be described for each phase.

<<Meeting registration phase>>
FIG. 7 is an example of a sequence diagram illustrating processing in the phase in which the organizer registers the conference in the conference server 30. As shown in FIG.

S101: The organizer operates the terminal device 10 to connect the terminal device 10 to the conference server 30, and logs in to the conference server 30 using the user name and password. This is to confirm that the user is authorized to use the conference system 100 . The terminal device 10 transmits a conference registration request to the conference server 30 . It is assumed that the URL for registering the conference is known in advance.

S102: The conference server 30 receives the conference registration request, numbers the conference ID, generates a URL for access by the terminal devices 10 of the participants during the conference, generates a password for the conference, and creates a web page for the conference screen 501. Create, etc.

S<b>103 : The web page transmission unit 33 of the conference server 30 transmits the web page of the conference screen 501 to the terminal device 10 .

S<b>104 : The web page acquisition unit 16 of the terminal device 10 receives the web page of the conference screen 501 , and the display control unit 12 displays the conference screen 501 on the LCD 206 . The conference screen 501 displayed by the terminal device 10 at the time of registration is the conference screen 501 in the initial state, and does not include text data converted from voice data. FIG. 8 shows an example of the conference screen 501 in the initial state.

S<b>105 : The host inputs text data to the conference screen 501 . Namely, the purpose of the meeting, the agenda, action items, participants, etc. are entered. Text data can be entered after the conference is held, and can be entered only during the conference registration phase. The operation reception unit 13 of the terminal device 10 receives input of text data.

S106: Also, the host associates the input text data with the items on the conference screen 501. FIG. The item is the purpose or agenda of the meeting. Details will be described with reference to FIG. 9, but the organizer drags and drops the text data onto the items on the conference screen 501 . The operation accepting unit 13 of the terminal device 10 accepts the association of the text data with the items of the conference screen 501 , and the extracting unit 12 a extracts a part of the text data according to the association and copies it to the items of the conference screen 501 . A first minutes is created from this extracted text data.

S<b>107 : The operation content transmission unit 17 of the terminal device 10 transmits the operation content for the conference screen 501 to the conference server 30 . The contents of this operation are the input text data and the correspondence with the item. Since the communication is asynchronous communication, the communication between the conference server 30 and the terminal device 10 is not limited to the illustrated timing, and communication may occur at any timing when information to be transmitted occurs.

S<b>108 : The operation content reception unit 34 of the conference server 30 receives the operation content and sends it to the web page generation unit 36 . Thereby, the text processing unit 41 registers the text data in association with the items of the meeting information DB 492 .

S109: Also, the organizer registers the contents in the conference server 30 as necessary. You don't have to register if you don't plan to use the content in the meeting. The operation accepting unit 13 of the terminal device 10 accepts content registration.

S<b>110 : The operation content transmission unit 17 of the terminal device 10 transmits the operation content for the conference screen 501 to the conference server 30 . This operation content is an operation related to content registration, and includes the content itself.

S<b>111 : The operation content reception unit 34 of the conference server 30 receives the operation content and sends it to the web page generation unit 36 . Thereby, the content processing unit 42 registers the content in the content DB 491 . It should be noted that content registration is possible even during a meeting.

FIG. 8 is a diagram showing an example of a conference screen 501 as a display screen displayed by the terminal device 10. As shown in FIG. A conference screen 501 is divided into a text display portion 502 and a content display portion 503 . First, the text display section 502 has a purpose column 511 , an agenda column 512 , an action item column 513 , a participant column 514 , a meeting content column 515 and a text input column 516 . A purpose field 511 is a field for displaying the purpose of the conference, an agenda field 512 is a field for displaying the agenda of the conference, an action item field 513 is a field for displaying action items, and a participant field 514 is a field for displaying the participants. This is the column to display. Note that the action item means homework, things to do, To Do, or the like. A conference content column 515 is a column for displaying the statements of each participant in chronological order. A text input field 516 is a field for the participant to input text data.

The following four pieces of information are displayed in the conference contents column 515 of the text display portion 502 .
・Audio picked up by the microphone (after conversion to text data)
・Text data entered in the text input field 516 ・Status change of the content displayed in the content display section 503 ・Notice from the conference server 30 The size, order, etc. are designed as appropriate. In addition, each item does not have to be displayed all the time, and the participant can close any item and display other items in a large size. There may also be items such as the time of the meeting and the location of the meeting room 7 .

The content display section 503 has a content display column 521 and a content list 522 . The content display column 521 displays the content (image data) of one page of the content currently shared by each terminal device 10 . A content list 522 displays a list of content that can be displayed at the conference (content registered in the conference information DB 492).

Also, the content display field 521 has a function of browser software, and the participant can display any web page. The web page displayed in the content display field 521 is shared among the terminal devices 10 . In addition, the content display column 521 may display files held by the terminal device 10 as image data. The displayed image data is shared among the terminal devices 10 .

The size and position of the text display section 502 and the size and position of the content display section 503 are not fixed, and the participant can display only one of them or change the size of the other as desired. Also, when the size and position of one is changed, the display control unit 12 automatically determines the size and position of the other. In other words, the size and position of the other are always interlocked with the size and position of the other, and the size and position of the other are determined so as to achieve the maximum size.

FIG. 9 is a diagram showing an example of a conference screen 501 on which the host has entered information about the conference. First, the text processing unit 41 of the conference server 30 displays the messages M1 and M2 in the conference content column 515. FIG. The message M1 is "Arrange a new meeting", and the message M2 is "Register purpose, agenda, action items, participants, content registration as required".

When the organizer enters text data into the text input column 516 for the messages M1 and M2, the text data T1 to T6 are displayed in the conference content column 515. FIG. Next, the organizer drags and drops the text data T1 to T6 onto each item using a pointing device such as a mouse or a touch panel. Since the text data T1 to T6 can be input in any order, the organizer drags and drops the text data T1 to T6 onto appropriate items while reading the content of the text data T1 to T6. As a result, the items are associated with the text data T1 to T6, and the extraction unit 12a causes the purpose column 511, the agenda column 512, the action item column 513, and the participant column 514 to display the information, respectively. Also, the text data T1 to T6 in the meeting content column 515 may remain as they are or may be deleted. In this embodiment, it is assumed that the text data T1 to T6 remain in the conference content column 515. FIG.

Since the script running on the terminal device 10 takes charge of moving the text data T1 to T6 by dragging and dropping, the processing is completed within the terminal device 10. FIG. However, since the operation to the conference screen 501 is notified to the conference server 30, the correspondence between the items and the text data T1 to T6 is held by the conference server 30 and shared with the other terminal devices 10. FIG.

An address book or the like may be used to input the participants. User names, e-mail addresses, etc. are registered in the address book, and the organizer can input participants by simply dragging and dropping (or copying and pasting) the user names in the address book.

Also, the organizer registers contents to be used in the conference in the contents list 522 . Drag and drop, for example, is also used for this registration. The host drags and drops (or copies and pastes) the content displayed by the file management software or the content obtained from the Web to the content list 522 of the conference screen 501 . As a result, the content 505 is registered in the content list 522 as shown.

Registration of content by drag-and-drop is performed by an OS (Operating System) and a script running on the terminal device 10 . Since the operation content transmission unit 17 of the terminal device 10 transmits the content to the conference server 30 , the conference server 30 converts the first page (for example, cover page) of the content into a thumbnail and transmits it to the terminal device 10 . As a result, thumbnails of the content 505 are displayed in the content list 522 . Note that the terminal device 10 may perform the conversion to thumbnails.

<<Email sending phase>>
FIG. 10 is an example of a sequence diagram illustrating processing in a phase in which the conference server 30 sends conference e-mails to participants. The processing in FIG. 10 follows the processing in FIG.

S201: The organizer operates the terminal device 10 to send an e-mail. This e-mail is for informing the participants of the holding of the conference. The operation reception unit 13 of the terminal device 10 receives this operation. In order to send an e-mail, the conference screen 501 is provided with soft buttons or the like that allow the host to select sending to all participants or sending to individual participants. Press the soft button to send to.

S<b>202 : The operation content transmission unit 17 of the terminal device 10 transmits an email transmission request to the conference server 30 .

S203: The operation content reception unit 34 of the conference server 30 receives the mail transmission request and sends it to the web page generation unit 36. FIG. The text processing unit 41 prepares for transmission of electronic mail. Specifically, it generates a message to the effect that an e-mail will be sent, and generates an outline of the e-mail based on the information about the meeting read from the meeting information DB 492 . Also, the user information DB 493 is searched for the participant and it is confirmed that the participant is registered. Generate a message that emails cannot be sent for participants that cannot be verified.

S204: The Web page transmission unit 33 of the conference server 30 transmits the message and the mail summary of the conference server 30 to the terminal device 10. FIG.

S205: The Web page acquisition unit 16 of the terminal device 10 acquires the message and mail summary of the conference server 30, and the display control unit 12 displays them on the conference screen 501. FIG. FIG. 11 shows the conference screen 501 at this time.

S206: The host confirms the message and mail summary of the conference server 30 and inputs an operation to the terminal device 10 to permit transmission. For example, double-click (or double-tap) the message on the conference server 30 . The operation reception unit 13 of the terminal device 10 receives this operation.

S 207 : The operation content transmission unit 17 of the terminal device 10 transmits transmission permission to the conference server 30 .

S208: The operation content receiving unit 34 of the conference server 30 receives the transmission permission and sends it to the mail transmitting unit 38. The mail sending unit 38 reads information about the meeting from the meeting information DB 492 to generate an e-mail, reads the e-mail addresses of the participants from the user information DB 493, and sends the e-mail to each participant. In this way, information about the conference registered by the organizer is sent to each participant. This is convenient because the organizer can send information about the conference from the conference screen 501 . There is no need to separately activate mail software and send information about the conference by e-mail.

FIG. 11 is a diagram for explaining the conference screen 501 when sending an e-mail. In FIG. 11, the message M3 of the conference server 30 is newly displayed. The message M3 is "Inform the participants of the meeting outline".

In addition, an e-mail summary 531 is displayed on the conference screen 501 of FIG. The e-mail summary 531 may be displayed by the terminal device 10 when it is received from the conference server 30, or may be displayed as a pop-up by the terminal device 10 when the host performs a mouse-over operation. For example, the subject is "Notice of meeting 'Concerning implementation plan of user hearing'". The text data of the purpose column 511 is used for the subject. In addition, the text reads, "Mr. Suzuki has arranged a meeting. Details are here: https://k001/index.html Please check the contents and take action." The body text uses the user name of the organizer and the URL of the meeting.

In addition, it is preferable that the organizer can edit the subject and text of the outline of the e-mail. As a result, the organizer can add notes and the like regarding the conference.

Also, the conference server 30 may reserve the conference room 7 for the conference. In this case, the conference server 30 reserves the conference room 7 based on the number of participants and the conference time designated by the organizer.

<<During the meeting Phase of text conversion>>
FIG. 12 is an example of a sequence diagram illustrating processing in a phase in which the terminal device 10 transmits voice data to the conference server 30 and displays text data as needed. It is assumed that the process of FIG. 12 is started when the conference start time has come and the participants have gathered in the conference room 7 . The participant may be outside the company and the terminal device 10 may be connected to the outside network N2.

S301, S302: Each participant clicks or taps the URL described in the e-mail to connect the terminal device 10 to the conference server 30. FIG. A participant performs a login operation. The operation accepting unit 13 of each of the terminal devices 10A to 10E accepts the login operation, and the login requesting unit 18 transmits the login request to the conference server 30. FIG. As a result, the password of the conference, the user name, and the password of the participant are sent to the conference server 30 . By accessing the URL described in the e-mail, the terminal devices 10 that access the URL of the same conference establish a communication session with each other via the conference server 30 . Also, the terminal devices 10 participating in the same conference (communication session) can share data by transmitting or receiving voice data, text data, messages, etc. via the established communication session.

S303, S304: The authentication processing unit 39 of the conference server 30 receives the login request, specifies the conference by the accessed URL, and determines whether or not the password of the conference matches the password of the conference information DB 492. Also, it is determined whether or not the user name and password match the user name and password in the user information DB 493 . In the present embodiment, it is assumed that login has been successful (authentication has been established). Next, the text processing unit 41 of the web page generating unit 36 identifies information about the conference based on the URL from the conference information DB 492, reads out the conference ID, purpose, agenda, action items, and participants, and reads out the information, which corresponds to the text display unit 502. Generate a web page to Also, the content processing unit 42 reads content from the content DB 491 based on the conference ID, creates thumbnails, arranges them in the content list 522 of the content display unit 503 , and generates a Web page corresponding to the content display unit 503 . The web page transmission unit 33 transmits the web page of the conference screen 501 to the terminal device 10 .

S304-1, S304-2: The web page acquisition unit 16 of the terminal device 10A-E acquires a web page and sends it to the web page analysis unit 11. FIG. The web page analysis unit 11 analyzes the web page and sends HTML data, scripts, CSS, etc. to the display control unit 12 . The display control unit 12 displays the conference screen 501 based on HTML data or the like.

S305: When the participant speaks, the voice acquisition unit 14 of the terminal device 10 acquires voice data.

S306: The voice data transmission section 15 of the terminal device 10 transmits the voice data and the user name to the conference server 30. FIG. The username is used to identify the speaker. Identification information of the terminal device 10 may be used instead of the user name, as long as the speaker can be specified.

S307: The voice data receiving unit 32 of the conference server 30 receives the voice data and sends it to the text conversion requesting unit 31. Text conversion request unit 31 transmits the voice data and the user name to voice recognition server 50 . The user name is only sent to identify the text data, and a unique serial number or the like may be sent along with the voice data. The conference server 30 records the correspondence between the serial number and the user name. Moreover, not only the voice data but also the text data entered in the text input field 516 by the participant is similarly transmitted.

S308: The text conversion reception unit 52 of the speech recognition server 50 receives the speech data, and the text conversion unit 51 converts the speech data into text data. The text response unit 53 then transmits the text data and the user name to the conference server 30 . Any data format may be used at the time of transmission, and the data may be transmitted in HTML, XML, SOAP, or the like.

S309: The text receiving section 35 of the conference server 30 receives the text data and the user name, and sends them to the web page generating section . The text processing unit 41 adds the user name to the newly acquired text data and sends the text data to the Web page transmission unit 33 . That is, the text data for the utterance is additionally transmitted. The text data to be transmitted may be created in common for each terminal device 10, or may be transmitted by changing the color of the text data of the participant's speech and that of the other participants' speeches. Alternatively, the user name and text data may be transmitted and displayed in different colors on the terminal device 10 side. In this embodiment, a mode in which text data and a user name are transmitted will be described as an example. However, even when the user name is transmitted, color coding may be performed on the conference server 30 side.

S310, S311: The Web page transmission unit 33 of the conference server 30 transmits the text data and the user name to the terminal devices 10A to 10E including the terminal device 10 that transmitted the voice data. In this way, the voice data is once transmitted to the conference server 30 and then transmitted to each of the terminal devices 10A-E. However, the terminal device 10A that has acquired the voice data may convert it into text data and display it, transmit the converted text data to the conference server 30, and the conference server 30 may transmit it to the terminal devices 10B-E.

S312, S313: The web page acquisition unit 16 of the terminal device 10A-E acquires the text data and user name, and sends them to the web page analysis unit 11. FIG. A technology for dynamically changing the content of HTML without the terminal device 10 updating the conference screen 501 is called AJAX. While asynchronous communication is performed by XMLHttpRequest and the latest data is acquired from the outside, the contents of arbitrary HTML elements can be updated in real time by dynamic HTML.

The Web page analysis unit 11 determines from the user name that it is the text data of the participant, and sends the text data to the display control unit 12 . Since the additionally transmitted text data belongs to the participant, the display control unit 12 adds and displays the text data closer to the right end of the conference content column 515 of the conference screen 501 . The conference contents column 515 is scrolled when the text data cannot be accommodated. It should be noted that the user may be determined based on a flag, identification information, or the like instead of the user name.

S314: Next, it is assumed that the participant inputs an operation to associate the participant with the action item. The operation reception unit 13 of the terminal device 10 receives this operation and sends it to the operation content transmission unit 17 .

S315: The operation content transmission unit 17 transmits the operation content to the conference server 30 together with the user name. This operation content is the user name of the participant associated with the content of the action item. Note that the user name is sent to identify the operator, but it does not have to be sent if it is not necessary to identify the operator.

S<b>316 : The operation content reception unit 34 of the conference server 30 receives the operation content and sends it to the web page generation unit 36 . The text processing unit 41 registers the correspondence between action items and participants in the conference information DB 492 .

S317: The text processing unit 41 of the conference server 30 generates a display control command for associating the participant with the action item and a message of the conference server 30 displayed in the conference content field 515. FIG. This message is for notifying each participant of the correspondence between the action item and the participant. In addition, information that allows the terminal device 10 to determine that the message (utterance) is from the conference server 30 is added to the message. For example, the speaker's user name (conference server 30 in this case) is added in the same manner as the text data.

S318, S319: The Web page transmission unit 33 of the conference server 30 transmits the display control command for associating the action item with the participant, the message, and the user name (conference server 30) to the terminal devices 10A to 10E. Since the terminal device 10A has completed the correspondence, it is not necessary to send the information to the terminal device 10A. Instead of sending only the difference between the operation content and the message, the whole or part of the web page may be sent.

S320, S321: The web page acquisition unit 16 of the terminal device 10A-E acquires the display control command for associating the action item with the participant, the message, and the user name (conference server 30), and sends them to the web page analysis unit 11. . The web page analysis unit 11 detects the display control command and the message of the conference server 30 and sends them to the display control unit 12 . The display control unit 12 displays the user name associated with the action item in the action item column 513 . Also, the message of the conference server 30 is added to the conference content column 515 and displayed. When the message is from the conference server 30 , the display control unit 12 of the terminal device 10 displays the message in such a manner that the message is from the conference server 30 . The correspondence between action items and participants will be described with reference to FIG. When the conference server 30 on the system side speaks, the conference server 30 may be called a "bot".

Since the text data of the participants and the message of the conference server 30 are displayed in different manners, the participants can easily distinguish between the text data and the message of the conference server 30 .

FIG. 13 is a diagram showing an example of a conference screen 501 for explaining correspondence between action items and participants. For example, the participant drags and drops Mr. Suzuki to the action item "Confirm availability date for customer". Since the operation reception unit 13 receives this operation and sends it to the display control unit 12 , the display control unit 12 indents the action item in the action item column 513 and displays the user name of the participant. Since the Web page is transmitted from the conference server 30 to the terminal device 10 of the participant who is not the operator, it is possible to similarly display the action item in which the user name of the participant is indented.

Note that the action items may be updated in the operator's terminal device 10 as well, based on the web page transmitted from the conference server 30 .

In addition, in the conference content column 515, as a message M4 for notifying each participant of the correspondence between the action item and the participant, "Ms. is displayed. As a result, all the terminal devices 10 including the terminal device 10 of the operator can display that the action item and the participant are newly associated with each other.

This message M4 is displayed so as to be in contact with the left side of the conference content column 515 unlike text data uttered by a participant. The display control unit 12 of the terminal device 10 displays the utterance of the conference server 30 in such a manner that it can be recognized as the utterance of the conference server 30 . In addition to being in contact with the left side of the conference contents column 515, the color may be changed or the brightness may be lowered. Also, a column for remarks of the conference server 30 may be provided independently.

<<In-meeting content switching phase>>
FIG. 14 is an example of a sequence diagram illustrating processing in a phase in which the conference screen 501 is updated when a participant switches content. The process of FIG. 14 is executed subsequent to the process of FIG.

S401: Assume that the participant has input an operation to switch contents. The content switching may be either page turning of the same content or switching of the content itself. The operation reception unit 13 of the terminal device 10 receives this operation and sends it to the operation content transmission unit 17 .

S<b>402 : The operation content transmission unit 17 transmits the operation content to the conference server 30 . The contents of this operation are content identification information and a page number.

S<b>403 : The operation content reception unit 34 of the conference server 30 receives the operation content and sends it to the web page generation unit 36 . The content processing unit 42 converts the content page specified by the content identification information and the page number into image data.

S404, S405: The Web page transmission unit 33 of the conference server 30 transmits image data to the terminal devices 10A to 10E together with the content. Since the corresponding page is not displayed on the operated terminal device 10A, the image data is also transmitted to the terminal device 10A. In addition, the conference server 30 holds the page numbers that have already been delivered to the terminal device 10, and the communication load is reduced by notifying only the page number of the image data of the page once transmitted.

S406, S407: The web page acquisition unit 16 of each of the terminal devices 10A to 10E acquires image data and sends it to the web page analysis unit 11. FIG. The web page analysis unit 11 determines that the image data is content based on the content, and the display control unit 12 displays the image data in the content display field 521 .

S408: Next, the text processing unit 41 of the conference server 30 generates a message for the conference server 30 displayed in the conference content field 515. FIG. This message is for notifying each participant that the content has been switched. In addition, information that allows the terminal device 10 to determine that the message is a message (speech) from the conference server 30 is added to the message.

S409, S410: The Web page transmission unit 33 of the conference server 30 transmits the message and the user name (conference server) to the terminal devices 10A-E.

S411, S412: The web page acquisition unit 16 of the terminal devices 10A to 10E acquires the message and user name (conference server) and sends them to the web page analysis unit 11. FIG. Web page analysis unit 11 determines that the message is from conference server 30 based on the user name, and sends the message and the user name (conference server) to display control unit 12 . The display control unit 12 adds and displays a message indicating that the content has been switched to the conference contents column 515 of the conference screen 501 in a manner in which it can be seen that the message was made by the conference server 30 . The conference screen 501 at the time of content switching will be described with reference to FIG. Even if the participants are not looking at the content, they can notice that the content has changed (that the explanation is progressing) due to the message from the conference server 30 .

FIG. 15 is a diagram showing an example of a conference screen 501 for explaining content switching. In FIG. 15, the message M5 of the conference server 30 is displayed in contact with the left end of the conference content column 515. In FIG. The message M5 for notifying the switching of the content is, for example, "The content has been switched." As a result, all the terminal devices 10 including the terminal device 10 of the operator can notify that the content has been switched.

Image data (thumbnail) 532 of the content is displayed in this message M5. By displaying the image data 532 together with the message in this way, it becomes easier for the participants to notice the switching of the contents. Since the image data 532 is the same as the image data displayed in the content display column 521, the text processing unit 41 can acquire the image data from the content processing unit 42 and create a Web page to display the image data together with the message M5.

<<Phase when generating minutes>>
When the conference ends, the conference server 30 creates minutes. As described above, the first minutes 9 and the second minutes 8 are generated in this embodiment.

FIG. 16 is an example of a sequence diagram illustrating the operation of the conference server 30 to generate minutes.

S501: Any participant performs an operation to end the conference. For example, there is a method of pressing an end button on the conference screen 501 . Alternatively, the conference server 30 may detect that the terminal device 10 and the conference server 30 have stopped communicating. The operation accepting unit 13 of the terminal device 10 accepts an operation to end the conference.

S502: The operation content transmission unit 17 of the terminal device 10 transmits the end of the conference to the conference server 30. FIG.

S503: The operation content reception unit 34 of the conference server 30 receives the end of the conference and sends it to the web page generation unit 36. FIG. The text processing unit 41 of the conference server 30 generates a conference ending message. Information that allows the terminal device 10 to determine that the message is a message (speech) from the conference server 30 is added to the message. An example of the message is shown in FIG.

S504, S505: The Web page transmission unit 33 of the conference server 30 transmits the message and the user name (conference server 30) to the terminal devices 10A-E.

S506, S507: The web page acquisition unit 16 of each of the terminal devices 10A to 10E acquires the message and user name (conference server 30) and sends them to the web page analysis unit 11. FIG. Web page analysis unit 11 determines that the message is from conference server 30 based on the user name, and sends the message and the user name (conference server) to display control unit 12 . The display control unit 12 additionally displays the message at the end of the conference in the conference content column 515 of the conference screen 501 in a manner that makes it clear that the message is from the conference server 30 .

S508: Next, the minutes generating unit 37 of the conference server 30 generates first minutes and second minutes. Details will be described with reference to FIG.

S509, S510: The mail sending unit 38 of the conference server 30 sends an e-mail containing the first minutes 9 to the terminal devices 10A to 10E. Since it is an e-mail, it is not sent directly to the terminal devices 10A-10E, but the terminal devices 10A-10E can acquire the e-mail after this timing. Each participant can confirm follow-up matters and the like by reading the first minutes 9 of the e-mail. The first minutes 9 contains the main points, so there is no need to read the entire text.

S<b>511 : Also, the minutes transmission unit 43 of the conference server 30 transmits the second minutes 8 to the search system 60 . In this way, the minutes are automatically generated at the end of the meeting, so that one of the participants does not have to generate the minutes.

FIG. 17 is an example of a conference screen 501 for explaining generation of minutes. First, on the conference screen 501 of FIG. 17, a message M6 is displayed that reads, "Thank you for your hard work. A summary of the proceedings of the conference will be sent to you by e-mail separately. Details of the proceedings of the conference will be archived." The participants can read the message M6 and understand that there is no need to create minutes.

Also, in FIG. 17, the first minutes 9 are displayed. This first minutes 9 is sent by e-mail. The first minutes 9 has a subject 541 and a text 542. The subject 541 describes the purpose of the meeting, and the text 542 describes the purpose of the meeting, agenda 1 (code 543), agenda 2 (code 544). ), action item 1 (code 545), action item 2 (code 546), and URL (code 547) are described. Agendas 1 and 2 are associated with agendas and conclusions, and action items 1 and 2 are associated with contents and participants. All of them are information that can be extracted from the items of the meeting information DB 492 .

Also, the second minutes 8 includes the following four pieces of information.
(i) meeting summary (same information as the first meeting minutes);
(ii) information in the meeting description section;
(iii) content
(iv) Shared Screen Supplementally, the shared screen is image data displayed in the content display field 521 . This image data includes the image data of the displayed page of the content, and any other Web page screen becomes the shared screen. The search system 60 uses the results of learning these four pieces of information to generate answers to queries. Since the shared screen is image data, the search system 60 extracts characters by OCR (Optical Character Reader). Extracting characters makes searching easier. Pattern recognition may be performed on the image data as it is. Also, OCR may be performed by the conference server 30 .

Note that the message of the conference server 30 may be excluded from the information in the conference content column. This is because the message of the conference server 30 may not contribute to knowledge sharing.

Note that the message of the conference server 30 may be excluded from the information in the conference content column. This is because the messages of the conference server 30 hardly contribute to knowledge sharing.

FIG. 18 is an example of a flow chart diagram showing a procedure for the minutes generation unit 37 to generate the first minutes 9 and the second minutes 8 .

The first generation unit 37a of the minutes generation unit 37 reads out the information about the meeting from the meeting information DB 492, sets the purpose in the subject, and sets the purpose, agenda, action item, and URL in the text to create the first information. A minutes 9 is generated (S10).

Next, the second generation unit 37b of the minutes generation unit 37 determines whether or not to generate the second minutes 8 (S20). This is because the content contained in the second minutes 8 may be provided by the search system 60, and the second minutes 8 should not be generated for information that should not be disclosed even within the company. For example, meetings on new technology, personnel, payroll, restructuring proposals, etc. before publication should not generate the second minutes 8 . For example, the participant who finished the conference instructs the conference server 30 whether or not to generate the second minutes 8 . Alternatively, the second generation unit 37b of the minutes generation unit 37 makes a determination based on the participant's affiliation or information class. For example, if the ratio of participants in the personnel department is 50% or more, the second minutes 8 are not generated, and if the ratio of participants whose information class is above the threshold is 50% or more, the second minutes 8 are generated. is not generated, or the second minutes 8 are not generated if the sum of the information ranks of the participants is equal to or greater than the threshold.

When generating the second minutes 8 (Yes in S20), the second generation unit 37b of the minutes generation unit 37 generates (i) the summary of the proceedings (first minutes 9), (ii) the content of the meeting, ( iii) content and (iv) the shared screen are stored in one file to generate the second minutes 8 (S30).

<Search system configuration example>
FIG. 19 is an example of a schematic configuration diagram of the search system 60. As shown in FIG. In the search system 60 , a plurality of child servers 72 are connected to an integrated server 71 , and the child servers 72 manage information for each cabinet 73 . As an example, cabinets 73 are divided into conference cabinets 73a, shared cabinets 73b, and personal cabinets 73c. The second minutes 8 of this embodiment are stored in the conference cabinet 73a. Information that can be shared by arbitrary users is stored in the shared cabinet 73b. The personal information of the user is stored in the personal cabinet 73c.

FIG. 20 is an example of a diagram for explaining responses to inquiries from the search system 60. As shown in FIG. First, as shown in FIG. 20(a), words and materials are associated with probabilities. For example, in "a-1" in FIG. 20(a), the word "product A" is associated with the materials "catalog", "service manual", and "failure point list" with probabilities of 0.6, 0.3, and 0.1, respectively. It is In "a-2" in Fig. 20 (a), the word "policy briefing" is 0.6, 0.3, and 0.1 for the materials "2016 financial report, 2015 financial report, and future vision," respectively. are associated with the probability of Probabilities represent the strength of associations between words and documents and are updated with search results.

Also, as shown in FIG. 20(b), affiliations and materials are associated with probabilities. For example, in "b-1" in FIG. 20(b), the service is associated with materials such as a service manual and a failure point list with probabilities of 0.6 and 0.4, respectively. In "b-2" in FIG. 20(b), materials such as a catalog and a price list are associated with the sales department with probabilities of 0.5 and 0.5, respectively.

In addition, as shown in FIG. 20(c), information classes of employees and materials are associated with probabilities. Officers and general employees are converted into information ranks by a table that associates positions and information ranks held in advance. In “c-1” in FIG. 20(c), the officer is associated with the materials “FY 2016 settlement report” and “FY 2015 settlement report” with probabilities of 0.7 and 0.3, respectively. In "c-2" of FIG. 20(c), materials such as catalogs and price lists are associated with general employees at probabilities of 0.5 and 0.5, respectively.

A case where an employee makes an inquiry while the search system 60 holds such information will be described. First, if an employee who belongs to the service searches for "product A", the catalog, service manual, and failure point list shown in FIG. Next, when the search system 60 searches for materials associated with the service shown in FIG. 20(b), a service manual and a failure point list are hit. The probability that a service employee needs a catalog, a service manual, and a failure point list is the product of the probability in FIG. 20(a) and the probability in FIG. 20(b). The probability is 0.18%, and the failure location list is 0.04%. The search system 60 returns the service description with the highest probability.

When a service employee downloads this material or returns useful feedback to the search system 60, the search system 60 increases the probability of the service manual for product A by a predetermined amount, is increased by a predetermined amount. Decrease the probabilities of other materials so that they become 1 as a whole.

Similarly, if an employee belonging to the sales department searches for "merchandise A," the catalog, service manual, and failure point list shown in FIG. 20(a) will be hits. Next, when the search system 60 searches for materials associated with the sales department in FIG. 20(b), catalogs and price lists are hit. In this case, the probability of the catalog is 0.3%, the probability of the service manual is 0%, and the probability of the failure point list is 0%, so the retrieval system 60 returns the catalog with the highest probability. In addition, based on the feedback from the sales department staff, the catalog probability for product A is increased by a predetermined amount, and the catalog probability for the sales department is increased by a predetermined amount. Decrease the probabilities of other materials so that they become 1 as a whole.

In this way, it is possible to provide an appropriate answer to the search system 60 depending on the affiliation of the employee making the inquiry.

Similarly, if an officer searches for "policy briefing", the 2016 financial report, the 2015 financial report, and the future vision in FIG. 20(a) will be hits. Next, when the search system 60 searches for the materials associated with the officers in FIG. 20(c), the 2016 financial statements and the 2015 financial statements are hits. In this case, the probability of the 2016 financial report is 0.42%, the probability of the 2015 financial report is 0.09%, and the probability of the future vision is 0%. to answer.

In addition, based on the feedback of the officers, the probability of the fiscal 2016 financial results report for the policy briefing is increased by a predetermined amount, and the probability of the financial statements for the fiscal year 2016 is increased by a predetermined amount for the executives. Decrease the probabilities of other materials so that they become 1 as a whole.

If a general employee searches for "policy briefing", the 2016 financial report, the 2015 financial report, and the future vision in FIG. 20(a) will be hits. Next, when the search system 60 searches for materials associated with general employees in FIG. In this case, the probability of the 2016 financial report is 0.06%, the probability of the 2015 financial report is 0%, and the probability of the future vision is 0.09%. to answer. Further, based on the feedback from general employees, the probability of the future vision for the policy briefing is increased by a predetermined amount, and the probability of the future vision for general employees is increased by a predetermined amount. Decrease the probabilities of other materials so that they become 1 as a whole.

In this manner, appropriate answers can be provided to the search system 60 according to the rank of the employee inquiring.

Although the information is simplified in FIG. 20 for convenience of explanation, in reality, one word may be associated with dozens of materials, and an enormous amount of materials may be associated with affiliations and ranks. Also, there are as many relationships between words and materials as there are words. The search system 60 can return the appropriate material among them by calculating the probabilities.

In addition, although affiliations and ranks have been described separately in the explanation of FIG. 20, affiliations and ranks may be considered simultaneously. In this case, for each material associated with a word, the probability of belonging and the probability of class are multiplied to provide the material with the highest probability.

It should be noted that the probability does not have to be a strict probability that the total is 1, and it is sufficient if each material is weighted. In this case, the weighting may be normalized so that the highest value is one.

A supplementary explanation is provided for the case where the second minutes 8 of the meeting are retrieved. As shown in FIG. 20(d), for example, the affiliation of an employee is associated in advance with minutes of various meetings and their probabilities. Various types of meetings include development meetings, planning meetings, sales meetings, sales briefings, and the like. In "d-1" and "d-2" in FIG. 20(d), the development department and the sales department are associated with the development, planning meeting, sales meeting, and sales briefing, respectively, and their probabilities.

In addition, since the second minutes 8 originally contain words based on the content of the statement, it includes correspondences between words and materials as shown in FIG. 20(a). Therefore, when the second minutes 8 are used for retrieval, the second minutes 8 need only be associated with probabilities.

For example, when an employee makes an inquiry to the search system 60 with the search sentence "printer silent technology", the second minutes 8 including the words "printer, silent technology" are hit. Two or more words are entered separated by spaces, and AND search or OR search is specified. When two or more words are input as sentences, the terminal device 10 performs a morphological analysis and breaks them down into independent words. In this case, an AND search is performed.

For the sake of simplicity, assuming that the probability of multiple hit second minutes 8 is constant, if the employee belongs to the development department, the highest probability is the development meeting, so even if this employee belongs to the development department For example, the second minutes 8 of the meeting of the development department is answered. If the employee belongs to the sales department, the business meeting is most likely to be the business meeting.

The probability of being associated with the second minutes 8 will be explained. First, the second minutes 8 having the largest product of the probability of each of the plurality of hit second minutes 8 and the probability of the type of conference associated with the affiliation is answered. The probability of the second minutes 8 is calculated, for example, from the frequency with which the second minutes 8 are referred to. Divide the number of times each second minutes is referenced by the sum of the number of second minutes as a parameter. However, the initial value of the number of references is set to 1 so that the probability does not become zero.

When weights are associated, for example, based on the idea that minutes of important meetings are worth referring to, weights can be given according to the average information class of the participants. Also, based on the idea that the longer the duration of the meeting is, the more important information is included, weighting can be given according to the duration of the meeting. Also, based on the idea that the larger the volume of the minutes, the more information is included, it is possible to give weighting according to the volume of the second minutes. In addition, based on the idea that minutes related to a product whose number of shipments is increasing have a high reference value, it is possible to weight the second minutes containing the name of the product based on the year-on-year change in sales of the product. These considerations may be combined and weighted.

Since the employee's feedback changes the probability or weight associated with the affiliation, the probability or weight is adjusted each time a search is performed, and a more appropriate second minutes 8 is answered.

Further, in addition to the affiliation being associated with the minutes of various meetings and their probabilities, individual users may be associated with the minutes of various meetings and their probabilities. By doing so, it is possible to determine the probability of the minutes according to the user's interest, so that more appropriate minutes can be provided.

In this way, the conference system 100 includes detailed content such as backgrounds and reasons for topics related to agendas other than the agenda items and decisions that are not included in the minutes so that employees who are not participating in the conference can search and share knowledge. Generates a second minutes 8 that is Since many employees can search the second minutes 8, it becomes easier to share knowledge and reduce unnecessary meetings. Also, since the second minutes 8 are registered in the search system 60, appropriate minutes can be provided according to the attributes of the employees.

<Operation procedure>
FIG. 21 is an example of a flowchart for explaining the procedure for the search system 60 to reply to an inquiry. FIG. 21 starts with a word being sent to the search system 60, for example.

First, the search system 60 accepts search words (S11). It doesn't matter if it contains multiple words. The employee can select AND search or OR search. You can also search by text. In this case, the sentence is broken down into words and AND-searched.

Next, the search system 60 acquires the employee's affiliation and information class (S12). The employee's affiliation and information class may be input by the employee or obtained from the conference server 30 .

The search system 60 searches for cabinets (S13). The cabinets to be searched may be public, personal, or conference. You may also specify which cabinets employees will search. Here, it is assumed that at least the conference cabinet has been searched.

The search system 60 determines whether or not only one hit is found for the second minutes 8 (S14). If there is no hit, the process ends.

If the determination in step S14 is YES, the process proceeds to step S16, and the hit second minutes 8 are provided to the employee (S16).

If the determination in step S14 is No, the process proceeds to step S15, and the search system 60 performs the probability calculation described with reference to FIG. 20 based on the affiliation and information class (S15).

Then, the search system 60 replies to the employee with the second minutes 8 with the highest probability (S16). It should be noted that the second minutes 8 with the highest N probabilities may be answered to the employee, or all the second minutes 8 with the probability equal to or higher than the threshold may be answered to the employee.

<Screen example>
FIG. 22 shows an example of the search screen 601. As shown in FIG. The search screen 601 is displayed when the terminal device 10 communicates with the search system 60 and logs in. FIG. The search screen 601 has a search word column 602 and a recommendation search check box 603 . A search word column 602 is a column for an employee to enter a search word. When a check mark is set in the recommendation search check box 603, materials are extracted in consideration of user attributes such as affiliation and information class. If no check mark is set in the recommendation search check box 603, the search system 60 simply performs a full-text search.

Instead of using a display such as the search screen 601 and a user interface based on text input, it may be possible to search by voice input. In this case, the conference system 100 performs speech recognition on the voice data input from the microphone and converts it into text data.

<Summary>
As described above, the conference system 100 of the present embodiment provides a plurality of types of minutes corresponding to users who are expected to use the minutes, and a plurality of minutes in formats that are easy for users to use. is generated, various users can effectively utilize the minutes. For example, the participants of the conference can confirm the follow-up items decided at the conference and the follower, etc. by the first minutes 9, and the person other than the participants can inquire the second minutes 8 to the search system 60 to obtain necessary information. You can get just enough.

<Other application examples>
Although the best mode for carrying out the present invention has been described above using examples, the present invention is by no means limited to such examples, and various modifications can be made without departing from the scope of the present invention. and substitutions can be added.

For example, FIG. 23 is a diagram showing a configuration example of the conference system 100. As shown in FIG. As shown in FIG. 23(a), the conference server 30 and the voice recognition server 50 may be integrated. By doing so, it is possible to reduce the time lag of speech recognition and shorten the time until the terminal device 10 displays the text data. Further, in the configuration of FIG. 23( a ), the search system 60 may be integrated with the conference server 30 .

Also, as shown in FIG. 23B, one terminal device 10 in the conference room 7 may have the server function of the conference server 30 . Furthermore, the terminal device 10 may have the function of the voice recognition server 50, and the voice recognition server 50 may be independent or may exist in the external network N2. Since the terminal device 10 has the function of the conference server 30, the conference server 30 is not required separately, and the conference can be started by gathering the participants.

In addition, minutes may be generated in two categories of participants and non-participants, and three or more types of minutes may be generated. For example, minutes other than the participants are divided into minutes of the same department and minutes of other departments. The minutes of the same department are the same as the second minutes of this embodiment, but the minutes of the other departments are mechanically summarizing the contents column 515 of the meeting. Alternatively, minutes without content may be generated as minutes of other departments.

Also, in the present embodiment, an e-mail in which the first minutes 9 are described in the text is sent, but the first minutes 9 may be attached to the email. Also, the first minutes 9 may be delivered by a method other than e-mail (for example, delivered to a personal folder).

Also, one terminal device 10 does not have to be provided for each participant, and may be provided for a plurality of participants.

In addition, the microphone 211 may be built in or externally attached to the terminal device 10, or may be prepared commonly for a plurality of participants. In this case, the position of each participant is specified by face recognition or the like, and the voice of each participant is specified based on the direction in which the voice is collected.

Alternatively, instead of the conference server 30 sending the speech data to the speech recognition server 50 , the terminal device 10 may send the speech data to the speech recognition server 50 and the acquired text data may be sent to the conference server 30 .

5 and 6 are divided according to main functions in order to facilitate understanding of the processing of the terminal device 10, the conference server 30, the speech recognition server 50, and the search system 60. is. The present invention is not limited by the division method or name of the unit of processing. The processing of the terminal device 10, the conference server 30, the speech recognition server 50, and the search system 60 can also be divided into more processing units according to the processing content. Also, one processing unit can be divided to include more processing.

Also, the functions of the conference server 30, the speech recognition server 50, and the search system 60 may be distributed over a plurality of server devices, and a plurality of speech recognition servers 50 and search systems 60 may exist.

The text reception unit 35 is an example of text data acquisition means, the web page transmission unit 33 is an example of data transmission means, the web page acquisition unit 16 is an example of data reception means, and the display control unit 12 is an example of a display unit. An example of the control means, the first generation unit 37a of the minutes generation unit 37 is an example of the first minutes generation means, the second generation unit 37b is an example of the second minutes generation means, The minutes transmitting unit 43 is an example of second providing means, the mail transmitting unit 38 is an example of first providing means, the extracting unit 12a is an example of extracting means, the audio data receiving unit 32 is an example of audio data The text conversion requesting unit 31 and the text conversion unit 51 are examples of an acquisition unit, the operation reception unit 13 is an example of reception unit, and the operation content transmission unit 17 is an example of an operation content transmission unit. Yes, the conference system 100 is an example of an information generation system, and the speech recognition server 50 is an example of a speech recognition device.

8 second minutes 9 first minutes 10 terminal device 30 conference server 50 speech recognition server 60 search system 100 conference system

Japanese Patent Application Laid-Open No. 2006-268800

Claims (14)

An information generating system for generating information about data transmitted or received over a conference communication session established between a plurality of terminal devices,
text data acquisition means for acquiring text data by recognizing voice data acquired by one of the plurality of terminal devices participating in the communication session;
storage means for storing a plurality of pieces of text data acquired by the text data acquisition means during the period from the start of the communication session to the end thereof;
data transmission means for transmitting the text data acquired by the text data acquisition means to any one of the plurality of terminal devices participating in the same communication session via a network;
any one of the plurality of terminal devices,
data receiving means for receiving the text data from a device that generated the text data via a network;
display control means for displaying the text data received by the data receiving means on a display screen;
extracting means for extracting part of the displayed text data according to the input received from the user on the display screen;
The information generation system further includes:
a first minutes generation means for generating first minutes data to be transmitted to participants who have participated in the communication session based on the partial text data extracted by the extraction means;
a first providing means for providing information on the generated first minutes data to users of terminal devices participating in the communication session;
The communication session for searching second minutes data generated by combining a plurality of text data stored in the storage means during the period from the start to the end of the communication session. a second providing means for transmitting to an in-house search system in which a person other than the participants who participated in the meeting can search the second minutes data;
It is determined whether or not to generate the second minutes data based on the instruction of the user of the terminal device participating in the communication session, and if it is determined to generate it, it is generated, and it is determined not to generate it. and a second minutes generation means that does not generate when
The first minutes generation means generates the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means,
The second minutes generation means generates the second minutes data that enables utilization of the text data displayed by the display control means,
The first providing means transmits the first minutes data to users of terminal devices participating in the communication session,
The second providing means registers the second minutes data in the search system,
The second minutes generating means determines whether or not to generate the second minutes data based on attributes of the users of the plurality of terminal devices, and does not generate the second minutes data. The information generating system wherein the second minutes generating means does not generate the second minutes data when it is determined that voice data acquisition means for acquiring voice data of the user of the terminal device;
conversion means for converting the voice data into the text data or transmitting the voice data to a voice recognition device;
2. The information generating system according to claim 1 , wherein said display control means displays said text data received by said data receiving means on a display screen in chronological order. having a server function for managing the information generation system,
The data receiving means acquires from the server function a message regarding the information generation system generated by the server function,
3. The information generating system according to claim 2 , wherein said display control means displays said message obtained by said data receiving means from said server function on said display screen in a manner different from said text data. wherein the plurality of terminal devices share content displayed separately from the text data by the display control means with other terminal devices via a network;
an operation content transmission unit configured to transmit operation content for the content to the server function when the display control unit accepts switching of the content to be displayed;
The data receiving means acquires from the server function a message indicating that the content generated by the server function has been switched,
4. The information generating system according to claim 3 , wherein said display control means displays on a display screen that said contents have been switched together with said text data. 5. The information generation system according to claim 4 , wherein said display control means displays image data of said content after switching together with a message indicating that said content has been switched. 6. Information according to claim 4 , wherein said second minutes generating means includes said text data, said content, and image data of said content displayed by said display control means in said second minutes data. generation system. When receiving a search for the search system, acquire attributes of the user to be searched,
7. The information generation system according to any one of claims 1 to 6 , wherein said search system provides different search results of said second minutes data according to said user's attributes. The search system holds information in which types of meetings and probabilities for each type of meetings are associated with the attributes of the user, and
The search system selects the second minutes data of the type of the meeting associated with the attribute of the searched user with the highest probability among the second minutes data that hit the searched word. 8. The information generation system of claim 7 , providing an information generation system. the communication session is established via an internal network;
9. The information generation system according to any one of claims 1 to 8 , wherein said search system is connected to said company network. The communication session is a conference communication session established via a conference server,
The first minutes data is a summary of the meeting, and the second minutes data includes text-displayed information and content-displayed information in the meeting in addition to the meeting summary. The information generation system according to any one of claims 1-9 . An information generation method performed by an information generation system for generating information about data transmitted or received via a conference communication session established between a plurality of terminal devices, comprising:
a step in which a text data acquisition means acquires text data by recognizing voice data acquired by one of the plurality of terminal devices participating in the communication session;
a step of storing a plurality of pieces of text data acquired by said text data acquisition means between the start and end of said communication session in a storage means;
a step in which the data transmission means transmits the text data acquired by the text data acquisition means to one of the plurality of terminal devices participating in the same communication session via a network;
A step performed by any one of the plurality of terminal devices,
a data receiving means receiving the text data from a device that generated the text data via a network;
a step in which the display control means displays the text data received by the data reception means on a display screen;
an extracting means extracting a part of the displayed text data according to an input received from a user on the display screen;
The following steps performed by the information generating system, comprising:
a first minutes generation means generating first minutes data to be transmitted to participants who have participated in the communication session based on the partial text data extracted by the extraction means;
a first providing means providing information on the generated first minutes data to users of terminal devices participating in the communication session;
A search target is second minutes data generated by a second providing means by combining a plurality of text data stored in the storage means during the period from the start of the communication session to the end of the communication session. providing the second minutes data to an in-house search system that allows a person other than the participants who participated in the communication session to search for the second minutes data in order to
A second minutes generation means determines whether or not to generate the second minutes data based on an instruction from a user of a terminal device participating in the communication session, and if it is determined to generate the second minutes data is generated, and if determined not to be generated, not generated ,
The first minutes generation means generates the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means,
The second minutes generation means generates the second minutes data that enables utilization of the text data displayed by the display control means,
The first providing means transmits the first minutes data to users of terminal devices participating in the communication session,
The second providing means registers the second minutes data in the search system,
The second minutes generating means determines whether or not to generate the second minutes data based on attributes of the users of the plurality of terminal devices, and does not generate the second minutes data. The information generating method , wherein the second minutes generating means does not generate the second minutes data when it is determined that data receiving means for receiving text data generated by an information processing device from the information processing device that generated the text data via a network;
display control means for displaying the text data received by the data receiving means on a display screen;
extracting means for extracting part of the displayed text data in accordance with the input received from the user on the display screen, via a communication session of a conference established between a plurality of terminal devices; An information processing device that generates information about data transmitted or received by
text data acquisition means for acquiring text data by recognizing voice data acquired by one of the plurality of terminal devices participating in the communication session;
storage means for storing a plurality of pieces of text data acquired by the text data acquisition means during the period from the start of the communication session to the end thereof;
data transmission means for transmitting the text data acquired by the text data acquisition means to one of the plurality of terminal devices participating in the same communication session via a network;
a first minutes generation means for generating first minutes data to be transmitted to participants who have participated in the communication session based on the partial text data extracted by the extraction means;
a first providing means for providing information on the generated first minutes data to users of terminal devices participating in the communication session;
The communication session for searching second minutes data generated by combining a plurality of text data stored in the storage means during the period from the start to the end of the communication session. a second providing means for providing the second meeting minutes data to an in-house search system in which a person other than the participants who participated in the meeting can search the second meeting minutes data;
It is determined whether or not to generate the second minutes data based on the instruction of the user of the terminal device participating in the communication session, and if it is determined to generate it, it is generated, and it is determined not to generate it. and a second minutes generation means that does not generate when
The first minutes generation means generates the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means,
The second minutes generation means generates the second minutes data that enables utilization of the text data displayed by the display control means,
The first providing means transmits the first minutes data to users of terminal devices participating in the communication session,
The second providing means registers the second minutes data in the search system,
The second minutes generating means determines whether or not to generate the second minutes data based on attributes of the users of the plurality of terminal devices, and does not generate the second minutes data. The information processing apparatus in which the second minutes generating means does not generate the second minutes data when it is determined that data receiving means for receiving text data generated by an information processing device from the information processing device that generated the text data via a network;
display control means for displaying the text data received by the data receiving means on a display screen;
extracting means for extracting part of the displayed text data in accordance with the input received from the user on the display screen, via a communication session of a conference established between a plurality of terminal devices; an information processing device that generates information about data transmitted or received by
text data acquisition means for acquiring text data by recognizing voice data acquired by one of the plurality of terminal devices participating in the communication session;
storage means for storing a plurality of pieces of text data acquired by the text data acquisition means during the period from the start of the communication session to the end thereof;
data transmission means for transmitting the text data acquired by the text data acquisition means to one of the plurality of terminal devices participating in the same communication session via a network;
a first minutes generation means for generating first minutes data to be transmitted to participants who have participated in the communication session based on the partial text data extracted by the extraction means;
a first providing means for providing information on the generated first minutes data to users of terminal devices participating in the communication session;
The communication session for searching second minutes data generated by combining a plurality of text data stored in the storage means during the period from the start to the end of the communication session. a second providing means for providing the second meeting minutes data to an in-house search system in which a person other than the participants who participated in the meeting can search the second meeting minutes data;
It is determined whether or not to generate the second minutes data based on the instruction of the user of the terminal device participating in the communication session, and if it is determined to generate it, it is generated, and it is determined not to generate it. Function as a second minutes generation means that does not generate when
The first minutes generation means generates the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means,
The second minutes generation means generates the second minutes data that enables utilization of the text data displayed by the display control means,
The first providing means transmits the first minutes data to users of terminal devices participating in the communication session,
The second providing means registers the second minutes data in the search system,
The second minutes generating means determines whether or not to generate the second minutes data based on attributes of the users of the plurality of terminal devices, and does not generate the second minutes data. A program in which the second minutes generating means does not generate the second minutes data when it is determined that data receiving means for receiving text data generated by an information processing device from the information processing device that generated the text data via a network;
display control means for displaying the text data received by the data receiving means on a display screen;
extracting means for extracting part of the displayed text data in accordance with the input received from the user on the display screen, via a communication session of a conference established between a plurality of terminal devices; An information generation method performed by an information processing device that generates information about data transmitted or received by
a step in which a text data acquisition means acquires text data by recognizing voice data acquired by one of the plurality of terminal devices participating in the communication session;
a step of storing a plurality of pieces of text data acquired by said text data acquisition means between the start and end of said communication session in a storage means;
a step of transmitting the text data acquired by the text data acquisition means to any one of the plurality of terminal devices participating in the same communication session via a network;
a first minutes generation means generating first minutes data to be transmitted to participants who have participated in the communication session based on the partial text data extracted by the extraction means;
a first providing means providing information on the generated first minutes data to users of terminal devices participating in the communication session;
A search target is second minutes data generated by a second providing means by combining a plurality of text data stored in the storage means during the period from the start of the communication session to the end of the communication session. providing the second minutes data to an in-house search system that allows a person other than the participants who participated in the communication session to search for the second minutes data in order to
A second minutes generation means determines whether or not to generate the second minutes data based on an instruction from a user of a terminal device participating in the communication session, and if it is determined to generate the second minutes data is generated, and if determined not to be generated, not generated ,
The first minutes generation means generates the first minutes data summarizing the text data extracted by the extraction means and displayed by the display control means,
The second minutes generation means generates the second minutes data that enables utilization of the text data displayed by the display control means,
The first providing means transmits the first minutes data to users of terminal devices participating in the communication session,
The second providing means registers the second minutes data in the search system,
The second minutes generating means determines whether or not to generate the second minutes data based on attributes of the users of the plurality of terminal devices, and does not generate the second minutes data. The information generating method , wherein the second minutes generating means does not generate the second minutes data when it is determined that

Images