JPH11272663A - Device and method for preparing minutes and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the system which can generate the high precision minutes of even though a personal computer conference system using a communication network has narrow transmission bandwidth and can easily judge who of conference participants is speaking in a conference. SOLUTION: Voice data inputted from a voice input/output part 3 are stored in a voice storage part 9 and converted by a voice-text conversion part 6 into text data, which are sent to the minutes generation part 7 of the speaker's or other-party's personal computer. The voice input/output part 3 is triggered by detecting voice input to photograph and send a picture of the speaker's face to the minutes generation part 7. The image data and text data sent to the minutes generation part 7 are used by the minute generation part 7 to generate the conference minutes, which are recorded in a minutes recording part 8.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a personal computer conference system using an information processing device such as a personal computer (hereinafter referred to as a personal computer). The present invention relates to a personal computer conference system and a recording medium having an automatic creation function as minutes.

[0002]

2. Description of the Related Art Normally, in a video conference or a video phone, data is transmitted and received using dedicated hardware, and a TV monitor is used for image display. This adopts a method based on the ITU-T international standard, and transmits and receives images and sounds after compressing the data.

[0003] Also, when a normal personal computer conference is held, FIG.
As shown in FIG. 4, each conference participant is connecting to a server that controls the conference participation terminal and is proceeding with the conference. Also,
Patents have been filed for creating conference proceedings in video conferences (Japanese Patent Application Laid-Open Nos. Hei 6-22530 and Hei 7-18236).
No. 5), voice and image data are recorded on this server,
It is only accumulated and created later.

[0004]

However, when such a personal computer conference server is used, the minutes of the minutes are created after the conference, which is troublesome. In addition, although it is conceivable to use means for converting all the contents of speech of the conference participants into text, the transmitted voice signal is not always of high quality. Generally, audio signals used in such a personal computer conference system conform to ITU-T Recommendation G. 711.G. Speech compression encoding such as 722 is performed.

[0005] Depending on the state of the communication network used between the terminals, it is extremely difficult to convert the speech signal received and decoded by the server into text, and the minutes of misrecognized minutes are created. is there. It is also difficult to immediately determine who is speaking during the meeting.

[0006]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides an input means for inputting voice data, and assigns an identifier to the voice data input by the input means and stores the data. Voice storage means for converting the voice data stored in the voice storage means into character data, and an identifier assigned to the voice data converted from the character data converted by the conversion means. And means for creating a minutes of the meeting.

[0007] Further, image input means for inputting audio data and image data corresponding to the audio data, and identifiers are assigned to the audio data input by the input means and the image data corresponding to the audio data and stored. A storage unit, a conversion unit that converts the voice data stored in the storage unit into character data, and the storage unit corresponding to the character data converted by the conversion unit and the voice data converted to the character data. Means for rearranging the stored image data based on the identifier and creating a minutes.

[0008] Further, an identifier is added to the input voice data and stored, the stored voice data is converted into character data, and the converted character data is arranged based on the identifier provided to the voice data. In other words, the minutes are prepared.

An identifier is assigned to the input audio data and the image data corresponding to the audio data and stored, and the stored audio data is converted into character data.
The converted character data and the stored image data corresponding to the audio data converted to the character data are rearranged based on the identifier, and a minutes is created.

[0010] Further, the present invention is a recording medium on which a program for exchanging data via a plurality of computers and creating minutes is recorded so as to be executable by the computers,
An identifier is added to the input voice data and stored, the stored voice data is converted into character data, the converted character data is rearranged based on the identifier provided to the voice data, and the minutes of the minutes are recorded. This is a recording medium on which a program to be created is recorded so as to be executable by a computer.

[0011] Also, a recording medium in which a program for exchanging data via a plurality of computers and creating minutes is recorded so as to be executed by the computers,
The input audio data and the image data corresponding to the audio data are given identifiers and stored, and the stored audio data is converted into character data, and the converted character data and the character data are converted. A recording medium in which a program for rearranging the stored image data corresponding to the generated audio data and rearranging the minutes based on the identifier is recorded so as to be executable by a computer.

According to the invention of the present application, the voice data spoken during the conference is subjected to text conversion processing on the voice input side, thereby making it possible to create a highly accurate minutes with few errors.

Since the minutes of the meeting are created on the personal computer of the person participating in the meeting, there is no need to prepare a separate personal computer conference server. In addition, by attaching the face image of the speaker to the minutes, when the minutes are read after the meeting, it is possible to easily know who of the meeting participants has spoken.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings. (First Embodiment) FIG. 1 is a diagram showing the configuration of a personal computer conference system according to the present invention. In FIG. 1, a personal computer 1 is a personal computer body which is an information processing device used in the present invention.

The image input unit 2 is a digital video camera, a digital camera, or the like for photographing a conference attendee who is a personal computer user. The audio input / output unit 3 includes a microphone, a speaker, and the like.

The data input section 4 is for inputting data other than the image input section 2 and the voice input / output section 3. Specifically, the data input section 4 includes a keyboard of a personal computer, a digitizer for drawing input, and other auxiliary means. Input means.

The personal computer conference processing unit 5 provides a television / data conference service on a personal computer and has means for compressing / decompressing image / audio / data. The personal computer conference processing unit 5 includes an image input unit 2, an audio input / output unit 3,
By transmitting / receiving various data obtained from the data input unit 4 to other PC users, it is possible to share the same data in real time while viewing the other person's face between remote PCs. I do.

The image to be transmitted and received is represented by an ITU
-T International Standard H. H.261 compliant system and H.264. H.263 compliant coding. G. For voice. 711 and G. 728 compliant coding or the like.

The voice / text conversion processing section 6 recognizes the voice signal of the speaker input from the voice input / output section 3 and converts it into text data. The minutes creating unit 7 records the face image data of the personal computer user input from the image input unit 2, the text data of the speaker obtained from the voice-to-text conversion processing unit 6, and other data, and creates the minutes data. I do. The minutes creating unit 7 also uses data sent from the other party having the meeting.

The minutes recording section 8 is where the minutes created by the minutes creating section 7 are recorded. Voice storage unit 9
Is where audio signal data is temporarily recorded before being processed by the audio-text conversion processing unit 6.

A communication interface (hereinafter referred to as a communication IF) 10 is a network interface for connecting to a communication network 11. Communication network is ISDN, telephone network, LAN
・ This is a network line such as the Internet. The monitor 12 displays an output screen of the personal computer conference processing section 5, a minutes result image, a face image of another conference participant, and the like. The other party holding the PC conference also has the same configuration.

The flow of voice signal data in which a speaker speaks in a normal personal computer conference is as shown in the flowchart of FIG. First, in step S21, the speech of the conference participant is input from the voice input / output unit 3. The input data is digital data. Next, in step S22, the aforementioned S2
The audio data obtained in step 1 is compressed by the personal computer conference processing unit 5 using an encoding system conforming to international standards. In the next step S23, the data is transmitted to the personal computer of each conference participant via the communication IF10.

FIG. 3 is a flowchart showing the flow of converting an audio signal into text data and transmitting the text data. First, in step S31, the speech of the conference participant is input from the voice input / output unit 2. In step S32, the audio data is stored in the audio storage unit 9 in the audio data format as shown in FIG.

The voice data format shown in FIG. 4 includes an ID number 41, time data 42 when voice is input at S31,
It is composed of an audio digital signal 43. ID number 41 is
An ID that identifies a participant in the conference
Are the user ID and the utterance number assigned.

The utterance number is a number assigned in the order of utterances uttered during the conference, and the same numerical value is not present in the utterance number during the same conference. For example, if the statement number of the first speaker is “00001”, “00002” is assigned to the next statement.

The user ID used is automatically assigned when the personal computer conference processing unit 5 is started.
It is also possible for the meeting participants to make any changes.
The time data 42 uses the time of the personal computer when the speech of the speaker is input from the voice input / output unit 3 in S31.

Next, in step S33, the voice data of FIG. 4 recorded in the voice storage unit 9 in S32 is converted into text data by the voice / text conversion processing unit 6. The audio digital signal 43 of the audio data in FIG. 4 is converted from the audio digital data to text data. Whether the voice text data is sent to the minutes creating unit 7 of another conference participant via the communication IF 10 in the next step S34,
Alternatively, it is sent to the minutes preparing section 7 of the personal computer. May be sent to both. The destination of the text data in S34 is set by the conference participant at the start of the conference.

For example, when a conference is held at three personal computers A, B, and C. If A sets to create minutes,
A text data transmission request is issued to conference participants B and C. In step S34, A sends the text data to its own minutes creator, and sends B and C to its minutes creator. Finally, the minutes are created in the minutes creating section 7 of the personal computer of A, and are recorded in the minutes recording section 8 of the personal computer of A. B and C may refer to the minutes of A.

Next, a case is set in which the minutes are prepared by A and B. A issues a text data transmission request to B and C, and B issues a text data transmission request to A and C. The minutes are created by the minutes creating unit 7 of the personal computers A and B, and C can refer to both the minutes A and B.

If all participants do not set the minutes, no text data transmission request will be sent to ABC. At that time, the minutes creating unit 7 issues a request to the meeting participant, "Do you want to create minutes? Y / N". If all participants answer NO (rejection) to this question, no text data transmission request signal is sent from any personal computer, and no minutes are created. If YES (agree), the personal computer issues a text data transmission request to another personal computer.

Further, the process of converting the voice data of FIG. 4 recorded in the voice storage unit 9 in step S32 into text data by the voice / text conversion processing unit 6 is efficient when the CPU utilization is low. .

For example, if the processing is performed when the speaker is not speaking and the audio signal is not transmitted to the other conference participants, the deterioration of the image and sound quality and the response during the PC conference are reduced. Can be held down.

Further, when text data is transmitted to another personal computer, the transmission state of the image, voice, and data on the communication network is monitored, and the transmission band is reallocated.
Deterioration of image and sound quality and response during computer conference can be suppressed. As an example, when there is no remark from a conference participant while monitoring the transmission state, there is no transmission of the audio signal, so the transmission band of the audio signal is reduced and the transmission band for transmitting the text data is increased to transmit.

Next, a method for creating minutes will be described in detail. The data converted from the audio data in the format of FIG. 4 to text and transmitted to the minutes creating unit 7 in S34 described above is stored in a state as shown in FIG.

As described above, since the transmission of text data is not always performed in real time, the stored data is not arranged in time sequence. First, rearrangement is performed in the order of the time sequence according to the time code added to the comment text data. As a result, the contents of the remarks are arranged in the order in which the remarks were made during the meeting, and are finally reorganized into the format shown in FIG. As a result, the utterance contents are stored in a state where they are arranged in time sequence.

When the user requests the display of the minutes of the meeting, the minutes creating unit 7 displays the minutes on the monitor 12 of the personal computer in a layout as shown in FIG. The display on the monitor displays the content 72 spoken during the meeting and the name 71 of the speaker who spoke. The name 71 of the speaker uses the ID information 41 of the voice data in FIG.

As described above, according to the first embodiment, since the contents of remarks during the meeting necessary for the minutes of the meeting are converted into text sentences on the side of the speaker, high accuracy is obtained even if the state of the communication network is poor. Minutes can be created.

(Second Embodiment) Next, a second embodiment of the present invention will be described in detail. The process from photographing the face of the speaker using the speech of the speaker during the meeting as a trigger and using the data in the minutes of the meeting will be described with reference to the flowchart of FIG.

First, in step S81, it is detected that the speech of the conference participant has been input from the voice input / output unit 3.
The image input unit 2 always captures the face of the personal computer user and displays the data on the screen of the terminal participating in the conference.

In step S82, the speech is detected in S81, and using this as a trigger, the ID and time data are added to the face image data of the speaker input from the image input unit 2 at the next timing.

FIG. 9 shows the format of face image data to which a user ID has been added. The same numerical value as the speaker number obtained in S32 is given to the ID 91 of the face image data. The image data 92 is image data of the face of the speaker photographed in S82.

Next, in step S83, the image data obtained in step S82 is sent to the minutes creating section 7 of the personal computer that issued the text data transmission request signal via the communication IF 10.
Next, in step S84, the minutes are created by associating the voice data stored in S34 stored in the minutes creation unit 7 with the face image data obtained in S83.

Next, a method for creating minutes will be described. The face image data of the speaker in the format of FIG. 9 obtained in S82 and the voice data in the format of FIG. 4 obtained in S32 are stored in a state as shown in FIG.

In the same manner as in the above-described minutes creating method, first, the order is rearranged in the time sequence according to the time code added to the comment text data. This means that the remarks are arranged in the order in which the remarks were made during the meeting. Next, the utterance data and the utterance numbers added to the utterer's face data are compared, and rearranged by the same number.
It is reorganized into a state like 1. As a result, the speaker's face image data / speech contents are recorded in the minutes recording unit 8 in a state of being arranged in time sequence.

When the user requests the display of the minutes of the meeting, the minutes of the minutes recording means 8 are converted by
It is displayed on the monitor 12 of the personal computer in a layout as shown in FIG. The monitor 12 displays a sentence 121 of the content spoken during the meeting, and an image 122 of the expression of the speaking speaker at the time of speaking.

In the above description, the image data of the next one frame is used at the next timing with the detection of the speech as a trigger.
U.T. Recommendation H. When H.263 or MPEG4 is used, the above-mentioned effect can be obtained by using the I-picture at the next timing, triggered by the speech of the speaker.

It is also conceivable to obtain a still image in advance and use it repeatedly. Before entering the actual conference, even one of the conference participants sets the minutes, and when another person refers to it (when a text transmission request is issued to another participant's computer), each image input section In step 2, the participant's face is photographed, the speaker's ID 91 is added, and the participant's face is transmitted to the personal computer that has issued the text transmission request, and is used for the face image data used in the minutes.

This image becomes a still image having higher image quality than the moving image data transmitted during the personal computer conference, and there is no need to repeatedly exchange image data. The time may be added to the format of the face image data shown in FIG. The format at this time is shown in FIG. If this time data is used, the rearrangement described with reference to FIGS. 10 and 11 can be performed more efficiently.

As described above, according to the second embodiment of the present invention, the contents spoken in a meeting are automatically created as minutes, thereby reducing the work of the minutes creator. In addition, by displaying the face image of the speaker next to the content of the comment, it becomes easy to understand who the utterance is later.

[0050]

As described above, according to the present invention, high-precision text conversion can be performed by performing voice-to-text conversion processing on one's own personal computer, and can be used for the minutes of a conference. . Also, by attaching the face image of the speaker to the minutes, the speaker can be easily confirmed when the minutes are referred to later.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a conference system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a flow of processing of a normal audio signal in a personal computer conference according to an embodiment of the present invention.

FIG. 3 is a flowchart of a flow in which an audio signal is subjected to text conversion processing and transmitted according to a first example which is an embodiment of the present invention.

FIG. 4 is a schematic diagram of a format of an audio signal according to a first example which is an embodiment of the present invention.

FIG. 5 is a schematic diagram of text data stored in minutes creating means before minutes of the first example which is an embodiment of the present invention.

FIG. 6 is a schematic diagram of audio data stored in the minutes recording means after the minutes of the first example, which is one embodiment of the present invention, is created.

FIG. 7 is a diagram when a text-converted minutes of the first example that is an embodiment of the present invention is displayed on a monitor.

FIG. 8 is a flowchart showing a flow of creating minutes from input face image data and text data according to a second example that is an embodiment of the present invention.

FIG. 9 is a schematic diagram of a format of an image obtained by photographing a face according to a second example which is an embodiment of the present invention.

FIG. 10 is a schematic diagram of image data and text data stored in the minutes creating means before creating minutes in the second example which is an embodiment of the present invention.

FIG. 11 is a schematic diagram of image data and text data stored in the minutes recording means after the minutes of the second example, which is one embodiment of the present invention, is created.

FIG. 12 is a diagram when a minute including a face image and text data is displayed on a monitor according to a second example which is an embodiment of the present invention.

FIG. 13 is a schematic diagram of a format in which a time code is added to an image of a face taken according to the second example, which is one embodiment of the present invention.

FIG. 14 is a schematic diagram of a conventional conference system.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... PC, 2 ... Image input part, 3 ... Voice input / output part 4 ... Data input part, 5 ... Personal computer conference processing part 6 ... Voice text conversion processing part, 7 ... Minutes preparation part 8 ... Minutes recording part, 9 ... voice storage unit 10 ... communication interface, 11 ... communication network, 12 ... monitor

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Toshihiro Moroboshi 1st place, Komukai Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Inside Toshiba R & D Center (72) Inventor Tomiyoshi Fukumoto Toshiba Komukai-shi, Kawasaki-shi, Kanagawa No. 1 town Toshiba R & D Center

Claims (6)

[Claims] An input means for inputting voice data, a voice storage means for assigning an identifier to voice data input by the input means and storing the voice data, and a voice data stored in the voice storage means as characters. Conversion means for converting the data into data, and means for rearranging the character data converted by the conversion means based on the identifier given to the audio data converted to the character data, and creating a minutes. Minutes creation device. 2. An image input means for inputting audio data and image data corresponding to the audio data, and an identifier assigned to the audio data input by the input means and the image data corresponding to the audio data, and stored. Storage means; conversion means for converting voice data stored in the storage means into character data; and the storage means corresponding to the character data converted by the conversion means and the voice data converted to the character data. Means for rearranging the stored image data on the basis of the identifier to generate a minutes. 3. An input voice data is provided with an identifier and stored, the stored voice data is converted into character data, and the converted character data is arranged based on the identifier provided to the voice data. A minutes creation method characterized in that the minutes are replaced. 4. An apparatus according to claim 1, wherein said input voice data and image data corresponding to said voice data are provided with identifiers and stored, and said stored voice data is converted into character data. A minutes creating method, wherein a minutes is created by rearranging stored image data corresponding to voice data converted into character data based on the identifier. 5. A recording medium in which a program for exchanging data via a plurality of computers and creating a minutes is recorded so as to be executed by a computer, wherein an identifier is assigned to input audio data. A program for storing and converting the stored voice data to character data, rearranging the converted character data based on the identifier assigned to the voice data, and creating a minutes of the minutes so that the computer can execute the program. A recorded recording medium. 6. A recording medium in which a program for exchanging data via a plurality of computers and creating a minutes is recorded so as to be executed by a computer, the input audio data and the audio data corresponding to the input audio data. Assigning an identifier to the image data to be stored, converting the stored voice data into character data, and storing the converted character data and the stored image data corresponding to the voice data converted into the character data And a recording medium in which a program for creating minutes is rearranged on the basis of the identifier and recorded so as to be executable by a computer.