JP7471979B2 - Meeting Support System - Google Patents

Description

The present invention relates to a conference support system, a conference support method, and a program.

In a conference where hearing people and hearing-impaired people participate together, it has been proposed to convert the speech of each presenter and the speech input by the hearing-impaired person into text by operating a terminal, and to display the text on a display device and on the terminal owned by each user (see, for example, Patent Document 1). In the technology described in Patent Document 1, when a hearing-impaired person participates in a conversation using text, the parent unit is controlled to put the other conference participants on hold for speech, and information on the hold for speech is sent to the child unit used by the hearing-impaired person.

JP 2019-179480 A

However, with conventional technology, when a hearing person is presenting and facing the audience, they may speak without checking the display screen.

The present invention has been made in consideration of the above problems, and aims to provide a meeting support system, meeting support method, and program that can clearly make the presenter and the audience aware that text input is taking place.

(1) In order to achieve the above object, a conference support system according to one aspect of the present invention includes a terminal having a text input unit, a display device that displays images, and a conference support device that is connected to the terminal and causes the display device to display text information input from the terminal and information that is a text version of an audio signal input from an audio pickup unit, and when text input is started by the terminal, the conference support device changes the overall display of the display device for a predetermined period of time.

(2) In addition, in a conference support system according to one aspect of the present invention, when text input is started by the terminal, the conference support device may change the overall display of the display device for a predetermined period of time by at least one of changing the brightness of the overall display of the display device, changing the display color, changing the saturation, and changing the contrast.

(3) In addition, in a conference support system according to one aspect of the present invention, the conference support device may detect a speech section in the audio signal, and if speech continues after changing the overall display of the display device for a predetermined period of time, change the overall display of the display device again for a predetermined period of time.

(4) In order to achieve the above object, a conference support method according to one aspect of the present invention is a conference support method in a conference support system having a terminal equipped with a text input unit, a display device that displays images, and a conference support device that is connected to the terminal and causes the display device to display text information input from the terminal and information that has been converted from audio input from an audio pickup unit into text, and when text input is started by the terminal, the conference support device changes the overall display of the display device for a predetermined period of time.

(5) In order to achieve the above object, a program according to one aspect of the present invention is a conference support system having a terminal equipped with a text input unit, a display device that displays images, and a conference support device that is connected to the terminal and causes the display device to display text information input from the terminal and information that has been converted from audio input from an audio pickup unit into text. The program causes a computer of the conference support device in the conference support system to change the overall display of the display device for a predetermined period of time when text input is started by the terminal.

By using (1) to (5), the presenter and the audience can be made aware that text is being entered in an easy-to-understand manner.

FIG. 13 is a diagram illustrating a case where a person with a speech impediment is not making an input to a terminal according to the embodiment. 11 is a diagram illustrating a case where a person with a speech impediment inputs information into a terminal according to the embodiment. FIG. 1 is a block diagram showing an example of the configuration of a conference support system according to an embodiment; FIG. 11 is a diagram showing an example of an image displayed on a terminal according to the embodiment; 5A to 5C are diagrams illustrating an example of an image displayed on a first display device according to the embodiment. 13A and 13B are diagrams illustrating examples of changes to the image displayed on the first display device when text input is started according to the embodiment. 6 is a diagram showing an example of a display change period of the first display device according to the embodiment; FIG. 4 is a flowchart of a processing procedure of the conference supporting device according to the embodiment. FIG. 13 is a block diagram showing a configuration example of a modified example of the conference support system.

The following describes an embodiment of the present invention with reference to the drawings. Note that in the drawings used in the following description, the scale of each component has been appropriately altered so that each component is of a recognizable size.

[Overview of the conference support system, overview of this embodiment]
First, an overview of the conference support system and the present embodiment will be described.
FIG. 1 is a diagram showing a case in which a person with a speech disability according to the present embodiment is not making any input to a terminal.
The conference support system of this embodiment is used in a conference with, for example, two or more participants. Among the participants US1 to US3, a person with a speech impediment (e.g., a person with a hearing impairment) US2 may also participate in the conference. Note that the participants do not all need to be in the same conference room, and may participate from another conference room or from home, for example, via a network NW.

Each of the participants US1 and US3 who are able to speak wears a sound pickup unit 11. The person US2 who is unable to speak carries a terminal (smartphone, tablet terminal, personal computer, etc.) 20. The conference support device 30 performs voice recognition on the voice signals spoken by the participants, converts them into text, and displays the text on the first display device 60 (display device) and the terminal 20. The second display device 70 is connected to a PC (personal computer, etc.) 80 that displays explanatory materials. Participant US2 uses the terminal 2, for example, by placing it on a table Tb.

FIG. 2 is a diagram showing a case in which a person with a speech disability inputs information into a terminal according to this embodiment.
As shown in FIG. 2, when the speech-impaired person US2 starts inputting text using the terminal 20, the brightness or color of the screen of the first display device 60 changes for a first predetermined time (predetermined period of time).
As a result, according to this embodiment, it is possible to make the participant US1 who is speaking aware that input is being made by the terminal 20. For example, when the participant US1 finishes speaking the phrase he or she is currently speaking, he or she stops speaking until input is completed by the terminal 20. As a result, according to this embodiment, it is possible to wait for input by the terminal 20.

[Example of a conference support system configuration]
Next, an example of the configuration of a conference support system will be described.
Fig. 3 is a block diagram showing an example of the configuration of a conference support system 1 according to this embodiment. As shown in Fig. 3, the conference support system 1 includes a sound collection device 10, a terminal 20, a conference support device 30, an acoustic model and dictionary DB 40, a minutes and audio log storage unit 50, a first display device 60, a second display device 70, and a PC 80. The terminals 20 include terminals 20-1, 20-2, .... Hereinafter, when one of terminals 20-1 and 20-2 is not specified, it will be referred to as "terminal 20".

The sound collection device 10 includes a sound collection unit 11-1, a sound collection unit 11-2, a sound collection unit 11-3, etc. Hereinafter, when one of the sound collection units 11-1, 11-2, 11-3, etc. is not specified, it will be referred to as "sound collection unit 11".

The terminal 20 includes an input unit 201, a processing unit 202, a display unit 203, and a communication unit 204.

The conference support device 30 includes an acquisition unit 301, a voice recognition unit 302, a text conversion unit 303 (voice recognition unit), a dependency analysis unit 304, a minutes creation unit 306, a communication unit 307, an authentication unit 308, an operation unit 309, and a processing unit 310.

The sound collection device 10 and the conference support device 30 are connected by wire or wirelessly. The terminal 20 and the conference support device 30 are connected by a wire or wireless network NW.

[Sound recording device]
The sound collection device 10 collects a voice signal uttered by a user and outputs the collected voice signal to the conference support device 30. The sound collection device 10 may be a single microphone array. In this case, the sound collection device 10 has P microphones arranged at different positions. The sound collection device 10 generates P channel voice signals (P is an integer equal to or greater than 2) from the collected sound and outputs the generated P channel voice signals to the conference support device 30.

The sound collection unit 11 is a microphone. The sound collection unit 11 collects the user's voice signal, converts the collected voice signal from an analog signal to a digital signal, and outputs the converted digital voice signal to the conference support device 30. Note that the sound collection unit 11 may also be configured to output the analog voice signal to the conference support device 30.

[Device]
The terminal 20 is, for example, a smartphone, a tablet terminal, a personal computer, etc. The terminal 20 may include an audio output unit, a motion sensor, a Global Positioning System (GPS), etc.

The input unit 201 is, for example, a touch panel sensor (including a touch panel pencil) or a keyboard provided on the display unit 203. The input unit 201 detects input entered by the user and outputs the detected result to the processing unit 202. Note that the user of the terminal 20 inputs text by handwriting, for example, with a touch panel pencil, inputting by operating a software keyboard displayed on the display unit 203, or inputting by operating a mechanical keyboard.

Processing unit 202 detects that text input has started based on the result output by input unit 201, generates text input start information indicating that text input has started, and outputs the generated text input start information to communication unit 204. Processing unit 202 may detect that text input has started, for example, when it detects that one character has been handwritten or when one character has been input on a keyboard. When text has been input by handwriting, processing unit 202 recognizes the handwritten character using a known method and converts it into text.
The processing unit 202 generates transmission information in accordance with the result output by the input unit 201, and outputs the generated transmission information to the communication unit 204. The transmission information includes the input text information and identification information for identifying the terminal 20.
The processing unit 202 acquires text information output by the communication unit 204, converts the acquired text information into image data, and outputs the converted image data to the display unit 203. Note that the image displayed on the display unit 203 will be described later with reference to FIG.

The display unit 203 displays the image data output by the processing unit 202. The display unit 203 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, etc.

The communication unit 204 receives text information or minutes information from the conference supporting device 30 and outputs the received information to the processing unit 202. The communication unit 204 transmits the text input start information and transmission information output by the processing unit 202 to the conference supporting device 30.

[Acoustic model/dictionary DB, minutes/speech log storage unit]
The acoustic model/dictionary DB 40 stores, for example, an acoustic model, a language model, a word dictionary, etc. An acoustic model is a model based on sound features, and a language model is a model of information on words and their arrangement. A word dictionary is a dictionary with a large vocabulary, for example a large vocabulary word dictionary. The conference support device 30 may store words and the like that are not stored in the speech recognition dictionary 13 in the acoustic model/dictionary DB 40 for updating.

The minutes/audio log storage unit 50 stores minutes (including audio signals).

The first display device 60 displays the image data output by the conference support device 30. The first display device 60 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, etc. The first display device 60 may be provided in the conference support device 30.

The second display device 70 displays image data output by the PC 80. The second display device 70 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, or the like. The second display device 70 may be provided in the PC 80.

The PC 80 is, for example, one of a personal computer, a smartphone, a tablet terminal, etc.

[Conference Support Device]
The conference support device 30 is, for example, any one of a personal computer, a server, a smartphone, a tablet terminal, etc. When the sound collection device 10 is a microphone array, the conference support device 30 further includes a sound source localization unit, a sound source separation unit, and a sound source identification unit. The conference support device 30 converts the voice signals uttered by the participants into text by performing voice recognition, for example, at predetermined intervals. Then, the conference support device 30 transmits the text information of the converted utterances to each of the terminals 20 of the participants.

The acquisition unit 301 acquires the audio signal output by the sound collection unit 11, and outputs the acquired audio signal to the voice recognition unit 302. If the acquired audio signal is an analog signal, the acquisition unit 301 converts the analog signal into a digital signal, and outputs the converted digital audio signal to the voice recognition unit 302.

When there are multiple sound collection units 11, the voice recognition unit 302 performs voice recognition for each speaker who uses the sound collection unit 11.
The voice recognition unit 302 acquires the voice signal output by the acquisition unit 301. The voice recognition unit 302 detects a voice signal of a speech section from the voice signal output by the acquisition unit 301. The detection of the speech section may be performed by detecting a voice signal equal to or greater than a predetermined threshold value as the speech section, or by detecting the on and off states of the sound collection unit 11. The voice recognition unit 302 may detect the speech section using other known methods. The voice recognition unit 302 performs voice recognition on the voice signal of the detected speech section by referring to the acoustic model/dictionary DB 40 using a known method. The voice recognition unit 302 performs voice recognition using, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2015-64554. The voice recognition unit 302 outputs the recognized recognition result and the voice signal to the text conversion unit 303. The voice recognition unit 302 outputs the recognition result and the voice signal in association with each other, for example, for each sentence, for each speech phrase, or for each speaker.

The text conversion unit 303 converts the speech into text based on the recognition result output by the speech recognition unit 302. The text conversion unit 303 outputs the converted text information and the speech signal to the dependency analysis unit 304. Note that the text conversion unit 303 may convert the speech into text by deleting interjections such as "ah," "um," "eh," and "well."

The dependency analysis unit 304 performs morphological analysis and dependency analysis on the text information output by the text conversion unit 303. For dependency analysis, for example, SVM (Support Vector Machines) is used in the shift-reduce method, the spanning tree method, or the stepwise application method of chunk identification. If correction is required based on the analysis results, the dependency analysis unit 304 corrects the voice-recognized text information by referring to the acoustic model and dictionary DB 40 and stores it in the minutes and voice log storage unit 50. The dependency analysis unit 304 outputs the text information and voice signal resulting from the dependency analysis to the minutes creation unit 306.

The minutes creation unit 306 creates minutes, dividing them into groups for each speaker, such as a presenter, based on the text information and audio signals output by the dependency analysis unit 304. The minutes creation unit 306 stores the created minutes and the audio signals corresponding to them in the minutes/audio log storage unit 50. The minutes creation unit 306 may also create the minutes by deleting interjections such as "ah," "um," "eh," and "well."

The communication unit 307 transmits and receives information to and from the terminal 20. The information received from the terminal 20 includes, for example, a request to participate in a conference, text input start information, text transmission information, and instruction information requesting transmission of past minutes. The communication unit 307 extracts, for example, identification information for identifying the terminal 20 from the participation request received from the terminal 20, and outputs the extracted identification information to the authentication unit 308. The identification information is, for example, the serial number of the terminal 20, a Media Access Control address (MAC address), an Internet Protocol (IP) address, etc. When the authentication unit 308 outputs an instruction to permit communication participation, the communication unit 307 communicates with the terminal 20 that requested participation in the conference. When the authentication unit 308 outputs an instruction not to permit communication participation, the communication unit 307 does not communicate with the terminal 20 that requested participation in the conference. The communication unit 307 outputs the received information to the processing unit 310. The communication unit 307 transmits the text information or past minutes information output by the processing unit 310 to the terminal 20 that has requested participation.

The authentication unit 308 receives the identification information output by the communication unit 307 and determines whether or not to permit communication. The conference support device 30, for example, accepts registration of the terminals 20 used by participants in the conference and registers them in the authentication unit 308. Depending on the result of the determination, the authentication unit 308 outputs to the communication unit 307 an instruction to permit participation in communication or an instruction not to permit participation in communication.

The operation unit 309 is, for example, a keyboard, a mouse, a touch panel sensor provided on the first display device 60, etc. The operation unit 309 detects the result of the user's operation and outputs the detected operation result to the processing unit 310.

When the processing unit 310 receives text input start information from the terminal 20, it changes the image to be displayed on the first display device 60. The processing unit 310 changes the display, for example, by darkening the backlight (reducing the brightness), changing the background to black or the like to make the screen darker or brighter, increasing or decreasing the saturation, changing the contrast to make it darker or brighter, etc. After changing the image to be displayed on the first display device 60, the processing unit 310 ends the image change of the first display device 60, for example, after a first predetermined time has elapsed.
The processing unit 310 reads the minutes from the minutes/voice log storage unit 50 in response to instruction information requesting transmission of past minutes, and outputs the read information of the minutes to the communication unit 307. Note that the information of the minutes may include information indicating the speaker, information indicating the result of dependency analysis, etc.

When the sound collection device 10 is a microphone array, the conference support device 30 further includes a sound source localization unit, a sound source separation unit, and a sound source identification unit. In this case, the conference support device 30 has the sound source localization unit perform sound source localization using a transfer function generated in advance for the audio signal acquired by the acquisition unit 301. Then, the conference support device 30 performs speaker identification using the localization result of the sound source localization unit. The conference support device 30 performs sound source separation on the audio signal acquired by the acquisition unit 301 using the localization result of the sound source localization unit. Then, the voice recognition unit 302 of the conference support device 30 performs speech section detection and voice recognition on the separated audio signal (see, for example, JP 2017-9657 A). The conference support device 30 may also perform reverberation suppression processing.

[Example of image displayed on device]
FIG. 4 is a diagram showing an example of an image displayed on the terminal 20 according to the present embodiment.
The text display fields g11 to g13 are images of text obtained by voice recognition of speech that has already been spoken, or input text.
The text input field g14 is an area where the user of the terminal 20 inputs text.
The send button image g15 is a button image that is selected by the user of the terminal 20 when completing text input and sending the inputted text.

As shown in FIG. 4, the display image may include an icon g16 corresponding to the speaker, a speaker name image g17, a speaking time image g18, and a text input start button image g19 selected by the user when starting text input. The text input start button image g19 and the send button image g15 may be displayed in a switched manner.

[Example of image displayed on first display device]
FIG. 5 is a diagram showing an example of an image displayed on the first display device 60 according to this embodiment.

The image in area g100 is an area where participant information is edited.
The area g101 is an area for participant information. The icon image g102 is an image corresponding to a participant. The name image g103 is an image of the name of the participant. The microphone number image g105 is an image of the number (or identification information) of the sound pickup unit 11 used by the participant. The terminal number image g106 is an image of the number (or identification information) of the terminal 20 used by the participant.

The image in the area g200 is an area for displaying spoken text, input text, or minutes of a meeting. Note that Fig. 5 shows a state during login.
The button image g201 is a login/logout image.
The button image g202 is a button for starting/ending the conference support system 1. The display image g203 is an image that lights up while the conference support system 1 is being used.
The button image g204 is an image for displaying the minutes stored in the minutes/audio log storage unit 50 and playing back audio signals.

Icon image g211 is an image corresponding to the participant who has spoken.
Reference characters g212, g221, and g231 denote text information obtained by voice recognition of the contents of speech by the participants.
The speech time image g214 is information indicating the time when the participant spoke or input text.
Name image g215 is an image of the name of the first participant.

Note that the image shown in FIG. 5 is an example, and the image displayed on the first display device 60 is not limited to this. For example, during a meeting, only the image of area g200 may be displayed.

[Example of image on first display device]
Next, an example of changing the image displayed on the first display device 60 when text input is started will be described.
Fig. 6 is a diagram showing an example of a change in the image displayed on the first display device 60 when text input according to this embodiment is started. Note that Fig. 6 shows one text image among the displayed images, and omits icon images, button images, etc.

In an image g501 before text input starts, for example, a text image is displayed in black on a white background.
In contrast, in the first image g502 when text input is started, the entire screen of the first display device 60 is changed and displayed.
Alternatively, in the second image g503 when text input is started, the entire screen of the first display device 60 is darkened, and further the text image is changed from black to white and displayed.

The modification example shown in Fig. 6 is merely an example, and is not limited thereto. For example, a part of the screen may be modified, such as by darkening 2/3 of the screen.
By changing the screen in this way, it is possible to make the presenter or other person who is speaking aware that text input has started.

[Display change period of the first display device]
Next, an example of the display change period of the first display device will be described.
FIG. 7 is a diagram showing an example of a display change period of the first display device 60 according to the present embodiment.
In FIG. 7, the horizontal axis represents time (seconds).
The first display change period is a period from time t1 when the start of text input is detected to time t2 when a first predetermined time has elapsed. The first predetermined time is, for example, several seconds.
The second display change period is the period (second specified time) from time t1 when the start of text input is detected to time t3 when the icon image of the send button is transmitted by terminal 20, i.e., when text input ends.

The display change period shown in FIG. 7 is an example, and is not limited thereto. For example, if the speech does not end even after the first display change period has elapsed and the display change is returned, for example, from image g502 to image g501 in FIG. 6, the speaker may not have noticed the display change. In such a case, the display change may be performed again for the first display change period, and in this case, the display change may be performed for a time longer than the first predetermined time. The processing unit 310 may check the end of the speech after the second predetermined time has elapsed after the change is completed.
Alternatively, if the speech does not end even after the first display change period has elapsed and the display change has been returned, a second display change period may be performed.

[Example of processing procedure]
Next, a description will be given of an example of a processing procedure of the conference supporting device 30. Fig. 8 is a flowchart of the processing procedure of the conference supporting device 30 according to the present embodiment. Note that the following processing example is an example in which the screen display of the first display device 60 is changed for a predetermined time when text input is started.

(Step S1) The processing unit 310 of the conference support device 30 registers the audio pickup unit 11 and the terminal 20 to be used in the conference based on the operation result of the user operating the operation unit 309, etc.

(Step S2) The processing unit 310 detects the start of the conference based on the operation result of the user operating the operation unit 309, etc.

(Step S3) The processing unit 310 detects an utterance.
(Step S4) The voice recognition unit 302, the text conversion unit 303, etc. of the conference support device 30 perform processes such as voice recognition processing and text conversion on the voice signal collected by the sound collection unit 11.

(Step S5) The processing unit 310 determines whether the speech has ended based on the results of speech section detection, etc. If the processing unit 310 determines that the speech has ended (Step S5; YES), the processing returns to step S3. If the processing unit 310 determines that the speech has not ended (Step S5; NO), the processing proceeds to step S6.

(Step S6) The processing unit 310 determines whether or not text input has been started by the terminal 20. If the processing unit 310 determines that text input has been started (Step S6; YES), the processing unit 310 proceeds to the process of Step S7. If the processing unit 310 determines that text input has not been started (Step S6; NO), the processing unit 310 returns to the process of Step S5.

(Step S7) The processing unit 310 starts changing the image displayed on the first display device 60, for example, by darkening the entire screen.

(Step S8) The processing unit 310 determines whether a predetermined time has elapsed after starting the image change. If the processing unit 310 determines that the predetermined time has elapsed (Step S8; YES), the processing unit 310 proceeds to the processing of Step S9. If the processing unit 310 determines that the predetermined time has not elapsed (Step S8; NO), the processing unit 310 repeats the processing of Step S8.

(Step S9) The processing unit 310 ends the screen change on the first display device 60 and returns to the original display.

(Step S10) The processing unit 310 determines whether or not text input by the terminal 20 has ended. If the processing unit 310 determines that text input has ended (Step S10; YES), the processing unit 310 returns to the processing of Step S3. If the processing unit 310 determines that text input has not ended (Step S10; NO), the processing unit 310 proceeds to the processing of Step S11.

(Step S11) The processing unit 310 determines whether the speech has ended. If the processing unit 310 determines that the speech has ended (Step S11; YES), the processing unit 310 returns to the processing of Step S3. If the processing unit 310 determines that the speech has not ended (Step S11; NO), the processing unit 310 returns to the processing of Step S7.

Even if the display on the first display device 60 is changed to indicate that text is being input through the processing of steps S10 and S11, and the speaker is not aware of this, according to this embodiment, the display on the first display device 60 can be changed again to indicate this.

The above-described processing procedure is merely an example, and is not limiting. For example, the start of text input may be detected by an interrupt process.
In the above example, the screen display is changed when text input is started while speaking, but the present invention is not limited to this. Even when no speech is being made, the processing unit 310 may change the display of the entire screen of the first display device 60 to notify that text input has started. Furthermore, the processing unit 310 may not change the screen before the speech starts, but may start changing the screen after the speech starts.

As described above, in this embodiment, when text input via the terminal 20 is started, the screen of the first display device 60 is changed.
Furthermore, in this embodiment, even if the first screen change is missed, if speech and text input continue, the screen of the first display device 60 is further changed.

As a result, according to this embodiment, it is possible to make the presenter and the audience aware that text input is being performed in an easily understandable manner. Also, according to this embodiment, the presenter is aware that text input is being performed, and can pause his/her speech and wait for the end of text input. As a result, according to this embodiment, the hearing impaired person or the like can calmly perform text input using the terminal 20 without worrying about the speech.
Furthermore, according to this embodiment, even if the first screen change is missed, the screen of the first display device 60 is changed again, so that the user can notice that text input has started.

[Modification]
Next, a modified example will be described.
9 is a block diagram showing a configuration example of a modified example of the conference support system. As shown in Fig. 9, the conference support system 1A includes a sound collection device 10A, a terminal 20, a conference support device 30A, an acoustic model and dictionary DB 40, a minutes and voice log storage unit 50, a first display device 60, a second display device 70, and a PC 80.

The sound collection device 10A includes a sound collection unit 11A-1, a sound collection unit 11A-2, a sound collection unit 11A-3, etc. Hereinafter, when one of the sound collection units 11A-1, 11A-2, 11A-3, etc. is not specified, it will be referred to as "sound collection unit 11A".

The conference support device 30A includes an acquisition unit 301, a voice recognition unit 302, a text conversion unit 303 (voice recognition unit), a dependency analysis unit 304, a minutes creation unit 306, a communication unit 307, an authentication unit 308, an operation unit 309, and a processing unit 310A.

The sound collection unit 11A includes an alarm unit 111A (111A-1, ...). The alarm unit 111A is, for example, at least one of a vibration device or a light-emitting device. The alarm unit 111A issues an alarm for, for example, a predetermined period of time, in response to the control of the processing unit 310A of the conference support device 30A. This alarm indicates that text input has begun by the terminal 20.

When the processing unit 310A receives text input start information from the terminal 20, it notifies the notification unit 111A of the sound collection unit 11A. After the notification, for example, after a first predetermined time has elapsed, the processing unit 310A ends the notification to the notification unit 111A.

Note that, if the speech does not end even after the first predetermined time has elapsed after the notification and text input is still being continued, processing unit 310A may issue another notification.
Furthermore, the processing unit 310A may notify the sound collection units 11A carried by all participants, or may notify only the sound collection unit 11A of the participant who is speaking.
In this embodiment, the screen of the first display device 60 is also changed.

As described above, in this modified example, the sound pickup unit 11A used by at least the participant who is speaking is notified by vibration or the like.

As a result, according to the modified example, even if the user does not notice the notification from the first display device 60, the user can notice that text input has started by the notification from the sound collection unit 11A.

In addition, a program for realizing all or part of the functions of the conference support device 30 (30A) in the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform all or part of the processing performed by the conference support device 30 (30A). Note that the term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a home page providing environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes those that hold a program for a certain period of time, such as volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, it may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system, a so-called difference file (difference program).

The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1, 1A...conference support system, 10, 10A...sound collection device, 11...sound collection section, 20...terminal, 30, 30A...conference support device, 40...acoustic model/dictionary DB 40, 50...minutes/audio log storage section, 60...first display device, 70...second display device, 80...PC, 111A...notification section

Claims (2)

A terminal equipped with a text input unit;
A display device for displaying an image;
a conference support device that is connected to the terminal and displays, on the display device, text information input from the terminal and information obtained by converting an audio signal input from a sound pickup unit into text;
Equipped with
The conference support device includes:
when text input is started by the terminal, changing the entire display of the display device for a first predetermined time;
the first predetermined time is either a predetermined time from a start of the text input or a time from a start of the text input to an end of the text input,
detecting a speech section in the voice signal of a second person different from the first person who is inputting the text, and if the speech of the second person is continuing even after the predetermined time period is changed, changing the entire display on the display device for a second predetermined time period that is longer than the first predetermined time period;
Meeting support system. A terminal equipped with a text input unit;
A display device for displaying an image;
a conference support device that is connected to the terminal and displays, on the display device, text information input from the terminal and information obtained by converting an audio signal input from a sound pickup unit into text;
Equipped with
The conference support device includes:
when text input is started by the terminal, changing the entire display of the display device for a first predetermined time;
detecting a speech segment in the voice signal of a second person different from the first person who is inputting the text, and if the speech of the second person continues after the first predetermined time has elapsed, changing the entire display on the display device for a second predetermined time that is longer than the first predetermined time;
Meeting support system.

Images