JP2019179480A - Conference system, control method therefor, and program - Google Patents

Abstract

To provide a conference system which makes it easier for normal-hearing and hearing-impaired people to attend a conference together, and to provide a method of controlling the same and a program.SOLUTION: A conference system is provided, comprising: sub units, each having a voice acquisition unit, a text input unit, and a display unit; and a base unit connected with the sub units and configured to generate minutes using text information obtained through voice recognition of voices entered through the sub units or text information entered through the sub units, and to share the generated minutes with the sub units. When somebody participates in a dialog by means of text, the base unit controls the base unit to restrict other conference attendees from speaking and sends information for restricting speech to the sub units.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a conference system, a conference system control method, and a program.

  In the case where a plurality of people hold a meeting, it has been proposed that the utterance content of each speaker is converted into text and the utterance content is converted into text and displayed on a playback device owned by each user (see, for example, Patent Document 1). . In the technique described in Patent Document 1, the utterance is recorded as a voice memo for each topic, and the minutes creator reproduces the recorded voice memo and converts it into text. And with the technique of patent document 1, the minutes are created using what was converted into text.

Japanese Patent Laid-Open No. 08-194492

  However, when a hearing-impaired person and a normal hearing person hold a meeting together, the normal hearing person participates in the meeting by voice utterance, and the hearing-impaired person participates in the meeting by inputting text from the handset. With the technique described in Patent Document 1, if a hearing impaired person speaks a voice while a person with hearing impairment is inputting text on the handset, the person with hearing impairment may not be able to keep up with the content of the conference.

  The present invention has been made in view of the above problems, and provides a conference system, a conference system control method, and a program capable of facilitating a normal hearing person and a hearing impaired person to participate in a conference together. With the goal.

  (1) In order to achieve the above object, a conference system (1) according to an aspect of the present invention includes a sound collection unit (301), a text input unit (operation unit 302), and a display unit (display unit 304). The slave unit (3, 3a, 3b,...) Is connected to the slave unit, and the text information obtained by voice recognition of the voice input from the slave unit or the text information input from the slave unit is used. In a conference system comprising a master unit (2) that creates a minutes and shares the created minutes with the slave unit, when the master unit participates in a conversation by text, The master unit is controlled to wait for the speech of another conference participant, and information for waiting for the speech is transmitted to the slave unit.

  (2) Also, in the conference system according to one aspect of the present invention, the master unit automatically determines whether the slave unit has been input by voice or text and indicates that fact as the master unit. You may make it display on the said subunit | mobile_unit.

  (3) Further, in the conference system according to one aspect of the present invention, when the parent device is inputting text on the parent device side or the child device side, a dialog to that effect is sent to the parent device and the child device. You may make it display with.

  (4) In the conference system according to one aspect of the present invention, when the text input is completed, the slave device transmits information indicating the completion of the text input to the master device. Based on the information indicating that the text input received from the slave unit has been completed, voice input may be permitted to the master unit and the plurality of slave units.

  (5) In the conference system according to one aspect of the present invention, the conference system includes a sound collection device including a communication unit, a notification unit, and a sound collection unit, and the parent device participates in a conversation by text. In addition, information for waiting for the speech of another conference participant is transmitted to the handset and the sound collecting device, and the sound collecting device is prohibited from speaking based on the information for waiting for the speech from the parent device. You may make it alert | report that it is.

  (6) In order to achieve the above object, a conference system control method according to an aspect of the present invention includes a sound collection unit, a text input unit, a slave unit including a text display unit, and a connection to the slave unit. Using the text information obtained by voice recognition of the voice input from the slave unit or the text information input from the slave unit, a minutes is created, and the master unit that shares the created minutes with the slave unit, A control method for a conference system in a conference system comprising the steps of: controlling the master unit to wait for the speech of another conference participant when the master unit participates in a conversation by text; And when the base unit participates in a conversation by text, the base unit transmits information for waiting for a speech to wait for the speech of another conference participant to the base unit.

  (7) In order to achieve the above object, a program according to one aspect of the present invention connects a slave unit including a sound collection unit, a text input unit, and a text display unit, and the slave unit, and the slave unit Using the text information obtained by voice recognition of the speech input from the text or the text information input from the slave unit, and a master unit for sharing the created minutes with the slave unit When the computer of the parent device of the conference system participates in the conversation by text, the step of controlling the parent device to wait for the utterance of another conference participant, and the case where the conversation is joined by text And a step of transmitting to the slave unit information for waiting to speak so as to wait for the speech of another conference participant.

  According to the above (1), (6), or (7), there is an effect that it becomes easy for the normal hearing person and the hearing impaired person to participate in the conference together.

According to the above (2) or (3), when a text is input by the slave unit, the speech and text input of other participants can be stopped.
According to (4) described above, other participants can resume speech and text input.
According to (5) described above, a participant who is wearing a sound collecting device such as a pin microphone without using a handset can be prohibited from speaking during text input.

It is a block diagram which shows the structural example of the conference system which concerns on this embodiment. It is a figure which shows the example of an image which the main | base station which concerns on this embodiment displays on a display apparatus. It is a figure which shows the example of an image of the participant information displayed on the display part of the participant's subunit | mobile_unit based on this embodiment. It is a figure which shows the example of an image of the minutes information displayed on the display part of the subunit | mobile_unit of the participant who is performing the text input which concerns on this embodiment. It is a figure which shows the example of an image of the minutes information displayed on the display part of the subunit | mobile_unit of the participant who is not performing the text input which concerns on this embodiment. It is a figure which shows the example of an icon image displayed on the display apparatus of the main | base station which concerns on this embodiment, or the display part of a subunit | mobile_unit. It is a figure which shows the example of information which the memory | storage part of the main | base station which concerns on this embodiment memorize | stores. It is a flowchart which shows the example of a process sequence which the main | base station which concerns on this embodiment performs. It is a block diagram which shows the structural example of the sound collection part in a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, an example of a situation in which the conference system of this embodiment is used will be described.
The conference system according to the present embodiment is used in a conference held by two or more people. Among the participants, a person who is not able to speak may be participating in the conference. Participants who can speak use a microphone for each participant or use a terminal (smart phone, tablet terminal, personal computer, etc.) equipped with a microphone. Participants with hearing impairments use terminals capable of inputting text. The conference system recognizes the voice signal uttered by the participant and converts it into text, and displays the text on each terminal. In addition, the conference system displays text information input by the hearing impaired person on each terminal. When the hearing impaired person is inputting text information, the conference system displays that the text is being input on each terminal.

FIG. 1 is a block diagram illustrating a configuration example of a conference system 1 according to the present embodiment.
As shown in FIG. 1, the conference system 1 includes a master unit 2, a slave unit 3a, a slave unit 3b,. In addition, when not specifying one of the subunit | mobile_unit 3a, the subunit | mobile_unit 3b, ..., it is only called the subunit | mobile_unit 3. FIG.
The parent device 2 and the child device 3 are connected via a wired or wireless network 4.
A display device 21 is connected to the parent device 2. The display device 21 may be included in the parent device 2.

  The master unit 2 includes a sound collection unit 201, an operation unit 202, a communication unit 211, an authentication unit 212, an acoustic model / dictionary storage unit 221, a voice recognition unit 222, a text conversion unit 223, a dependency analysis unit 224, and a minutes creation unit. 225, a minutes storage unit 226, a text acquisition unit 231, a storage unit 240, an input determination unit 241, an icon setting unit 242, an icon storage unit 243, a warning generation unit 244, and an image output unit 245.

  The subunit | mobile_unit 3 is provided with the sound collection part 301, the operation part 302 (text input part), the process part 303, the display part 304, and the communication part 305. The sound collection unit 301, the operation unit 302, the processing unit 303, the display unit 304, and the communication unit 305 are connected via a bus 306.

<Slave unit 3>
First, the subunit | mobile_unit 3 is demonstrated.
The subunit | mobile_unit 3 is a smart phone, a tablet terminal, a personal computer etc., for example. In addition, the subunit | mobile_unit 3 may be provided with the audio | voice output part, the motion sensor, GPS (Global Positioning System; Global positioning system) etc.

  The sound collection unit 301 is a microphone. The sound collection unit 301 collects a user's sound signal, converts the collected sound signal from an analog signal to a digital signal, and outputs the sound signal converted into the digital signal to the processing unit 303.

  The operation unit 302 detects a user operation and outputs the detected result to the processing unit 303. The operation unit 302 is, for example, a touch panel sensor provided on the display unit 304 or a priority connection or wireless connection keyboard.

  The processing unit 303 generates setting information based on the operation result of the operation unit 302 being operated, and outputs the generated setting information to the communication unit 305. Here, the setting information includes participant identification information, information indicating whether the sound collection unit is used, and information indicating whether the operation unit is used. The processing unit 303 generates a login instruction based on the operation result of the operation unit 302 being operated, and outputs the generated login instruction to the communication unit 305. Here, the login instruction includes the identification information of the participant and the identification information of the handset 3. The processing unit 303 adds identification information to text information based on the operation result of the operation unit 202 being operated, and outputs the text information to the communication unit 305. The processing unit 303 adds the identification information to the audio signal output from the sound collection unit 301 and outputs it to the communication unit 305. The processing unit 303 acquires the image data output from the communication unit 305 and outputs the acquired image data to the display unit 304. The processing unit 303 establishes communication with the parent device 2 based on the information permitting login output from the communication unit 305.

  The display unit 304 displays the image data output from the processing unit 303. The display unit 304 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, or the like. The image displayed on the display unit 304 will be described later.

  The communication unit 305 transmits the setting information output from the processing unit 303 to the parent device 2 via the network 4. The communication unit 305 transmits a login instruction output from the processing unit 303 to the parent device 2 via the network 4. The communication unit 305 transmits text information or an audio signal output from the processing unit 303 to the parent device 2 via the network 4. Note that the text information or audio signal to be transmitted includes user identification information and slave unit 3 identification information. The communication unit 305 receives the image data transmitted from the parent device 2 and outputs the received image data to the processing unit 303. When the communication unit 305 receives the information allowing the login transmitted by the parent device 2, the communication unit 305 outputs the received information allowing the login to the processing unit 303.

<Master unit 2>
Next, the master unit 2 will be described.
The base unit 2 is, for example, a notebook personal computer.

  The sound collection unit 201 is a microphone. The sound collection unit 201 collects a user's voice signal, converts the collected voice signal from an analog signal to a digital signal, and outputs the converted voice signal to the voice recognition unit 222.

  The operation unit 202 detects a user operation and outputs the detected result to the text acquisition unit 231 and the input determination unit 241. The operation unit 202 is, for example, a touch panel sensor provided on the display device 21 or a keyboard. The operation unit 202 outputs an operation detection result to the authentication unit 212 during the login process.

  The communication unit 211 receives the audio signal transmitted from the child device 3 and outputs the received audio signal to the audio recognition unit 222. The communication unit 211 receives the text information transmitted from the handset 3 and outputs the received text information to the dependency analysis unit 224 and the input determination unit 241. The communication unit 211 receives the login instruction transmitted from the child device 3 and outputs the received login instruction to the authentication unit 212. The communication unit 211 transmits the image data output from the image output unit 245 to the child device 3 via the network 4. The communication unit 211 transmits information that allows the login output from the authentication unit 212 to the child device 3 via the network 4.

  Authentication unit 212 determines whether or not to permit login based on the identification information of the participant and the identification information of handset 3 included in the login instruction output from communication unit 211. When the authentication unit 212 permits login, the authentication unit 212 outputs information permitting login to the communication unit 211. The authentication unit 212 determines whether to permit the login of the user of the parent device 2 based on the result of the operation of the operation unit 202. When permitting login, the authentication unit 212 outputs information for permitting login to each functional unit, and permits the operation of each functional unit. The function units include a communication unit 211, an authentication unit 212, an acoustic model / dictionary storage unit 221, a voice recognition unit 222, a text conversion unit 223, a dependency analysis unit 224, a minutes creation unit 225, and a minutes storage unit. 226, a text acquisition unit 231, a storage unit 240, an input determination unit 241, an icon setting unit 242, an icon storage unit 243, a warning generation unit 244, and an image output unit 245.

  The acoustic model / dictionary storage unit 221 stores, for example, an acoustic model, a language model, a word dictionary, and the like. The acoustic model is a model based on the feature amount of sound, and the language model is a model of information on words and how to arrange them. The word dictionary is a dictionary with a large number of vocabularies, for example, a large vocabulary word dictionary. The base unit 2 may store and update words and the like that are not stored in the acoustic model / dictionary storage unit 221.

The voice recognition unit 222 acquires the voice signal output from the sound collection unit 201 or the voice signal output from the communication unit 211. The voice recognition unit 222 detects a voice signal in the utterance section from the voice signal. For the detection of the utterance section, for example, an audio signal having a predetermined threshold value or more is detected as the utterance section. Note that the speech recognition unit 222 may perform detection of the utterance section using another known method. The speech recognition unit 222 performs speech recognition on the detected speech signal of the utterance section with reference to the acoustic model / dictionary storage unit 221 using a known method. Note that the voice recognition unit 222 performs voice recognition using, for example, a method disclosed in JP-A-2015-64554. The voice recognition unit 222 outputs the recognized recognition result and the voice signal to the text conversion unit 223. Note that the speech recognition unit 222 outputs the recognition result and the speech signal in association with each other, for example, for each sentence, between utterance phrases, or for each utterance.
Note that the voice recognition unit 222 performs voice recognition for each sound collection unit (201 or 301), for example, by time division processing when voice signals are input simultaneously. In addition, when the microphone is a microphone array, the speech recognition unit 222 performs known speech recognition processing such as sound source separation processing, sound source localization processing, and sound source identification processing.

  The text conversion unit 223 converts the recognition result output by the speech recognition unit 222 into text with reference to the acoustic model / dictionary storage unit 221. Note that the text information includes information of at least one character. The text conversion unit 223 outputs the converted text information and the acquired voice signal to the dependency analysis unit 224. Note that the text conversion unit 223 may delete interjections such as “Ah”, “Ut”, “Eh”, and “Well” from the result of recognizing the utterance information and convert it to text.

The dependency analysis unit 224 performs morphological analysis and dependency analysis on the text information output by the text conversion unit 223 or the text information output by the communication unit 211 with reference to the acoustic model / dictionary storage unit 221. In the dependency analysis, for example, SVM (Support Vector Machines) is used in the shift-reduce method, the spanning tree method, and the chunk identification step application method.
Alternatively, the dependency analysis unit 224 acquires text information output from the text acquisition unit 231. The dependency analysis unit 224 performs morphological analysis and dependency analysis on the text information with reference to the acoustic model / dictionary storage unit 221.
The dependency analysis unit 224 outputs the text information subjected to morphological analysis and dependency analysis, and the analysis result to the minutes creation unit 225. When the dependency analysis unit 224 acquires the audio signal output from the text conversion unit 223, the dependency analysis unit 224 also outputs the acquired audio signal to the minutes creation unit 225.

  The minutes creation unit 225 creates minutes for each speaker based on the text information output by the dependency analysis unit 224. The minutes creation unit 225 causes the minutes storage unit 226 to store an audio signal corresponding to the created minutes. In addition, the minutes creation unit 225 outputs the created minutes to the image output unit 245. The minutes creation unit 225 may create minutes by deleting the interjections such as “Ah”, “Et”, “Eh”, “Well”.

  The minutes storage unit 226 stores the minutes and audio signals in association with each other.

  The text acquisition unit 231 acquires the operation result output by the operation unit 202, and generates text information based on the acquired result. The text acquisition unit 231 outputs the generated text information to the text conversion unit 223.

  The storage unit 240 stores setting information of participants or planned participants. The setting information includes a participant name, an identifier for identifying the participant, an icon associated with the participant, information indicating whether or not the participant's sound collection unit (201 or 301) is used, and operation of the participant Information indicating whether or not the part (202 or 302) is used is included.

The input determination unit 241 determines whether or not the sound collection unit 201 is used for the parent device 2 based on the audio signal output from the sound collection unit 201. Further, the input determination unit 241 determines whether or not the sound collection unit 301 is used for the child device 3 based on the audio signal output from the communication unit 211. For example, if the size of the audio signal continues for a predetermined time and is less than a predetermined threshold, the input determination unit 241 determines that no audio signal is input and does not use the sound collection unit (201 or 301). May be determined. Further, the input determination unit 241 determines that the audio signal is being input and uses the sound collection unit (201 or 301) when the size of the audio signal continues for a predetermined time and is equal to or greater than a predetermined threshold. May be determined.
The input determination unit 241 determines whether or not the operation unit 202 is used for the base unit 2 based on the operation result output from the operation unit 202. Further, the input determination unit 241 determines whether or not the operation unit 302 is used for the slave unit 3 based on the text information output from the communication unit 211. For example, the input determination unit 241 may determine that the operation unit (202 or 302) is not used when an operation result is not detected from the operation unit (202 or 302) for a predetermined time. The input determination unit 241 may determine that the operation unit (202 or 302) is being used when an operation result is detected from the operation unit (202 or 302) continuously for a predetermined time.
When it is determined that the sound collection unit (201 or 301) is to be used, the input determination unit 241 outputs information indicating the use of the sound collection unit (201 or 301) to the icon setting unit 242. When the input determination unit 241 determines to use the operation unit (202 or 302), the input determination unit 241 outputs information indicating use of the operation unit (202 or 302) to the icon setting unit 242.
The input determination unit 241 generates an utterance prohibition instruction (information for waiting for utterance) as a warning generation unit in order to prohibit other utterances while the operation unit 202 is operated or text information is input from the communication unit 211. To 244. Note that the utterance prohibition instruction includes information indicating the user who is using the parent device 2 when the operation unit 202 is operated, and when text information is input from the communication unit 211, 3 is included to indicate the user who is inputting text using 3.

  The icon setting unit 242 reads an icon image corresponding to the participant from the icon storage unit 243, and outputs the read icon image to the image output unit 245. In addition, when a participant logs in to a conference system for the first time and starts using, the participant selects an icon image corresponding to him / her from among the stored icons. The icon setting unit 242 outputs an image of an icon indicating use of the sound collection unit (201 or 301) based on information indicating use of the sound collection unit (201 or 301) output from the input determination unit 241. Output to the unit 245. The icon setting unit 242 outputs an image of an icon indicating use of the operation unit (202 or 302) to the image output unit 245 based on information indicating use of the operation unit (202 or 302) output from the input determination unit 241. To do. The icons will be described later.

  The icon storage unit 243 stores an icon corresponding to the participant. The icon storage unit 243 stores an icon indicating whether or not the sound collection unit (201 or 301) is used. The icon storage unit 243 stores an icon indicating whether or not the operation unit (202 or 302) is used.

  The warning generation unit 244 generates warning information (information for waiting to speak) based on the utterance prohibition instruction output by the input determination unit 241. The warning information includes information about the user who is inputting text. The warning generation unit 244 outputs the generated warning information to the image output unit 245.

  The image output unit 245 acquires the minutes information output by the minutes generation unit 225 and the warning information output by the warning generation unit 244. The image output unit 245 generates image data based on the minutes information, and outputs the generated image data to the communication unit 211 and the display device 21. When the warning information is acquired from the warning generation unit 244, the image output unit 245 displays an alert on the display device 21 based on the acquired warning information. When the warning information is acquired from the warning generation unit 244, the image output unit 245 transmits the acquired warning information to the child device 3 via the communication unit 211.

  The display device 21 is, for example, a liquid crystal display device, an organic EL display device, an electronic ink display device, or the like. The display device 21 displays the image data output from the image output unit. The display device 21 may be included in the parent device 2.

<Display image of base unit 2>
Next, an example of an image that the master unit 2 displays on the display device 21 will be described.
FIG. 2 is a diagram illustrating an image example displayed on the display device 21 by the parent device 2 according to the present embodiment.
The image g10 is an image displayed on the display device 21 of the parent device 2.

The image in the area g100 is an area for editing participant information.
An area g101 is an area for participant information. A symbol g102 is an icon image corresponding to the participant. Symbol g103 is the name of the participant. Reference numeral 104 denotes an icon indicating that the participant inputs text using the operation unit 202 of the parent device 2 or the operation unit 302 of the child device 3. The symbol g105 is an icon indicating that the participant speaks with the sound collecting unit 201 of the parent device 2 or the sound collecting unit 301 of the child device 3. A symbol g106 is a microphone number (or identification information) used by the participant.

The area g200 is an area for displaying minutes. FIG. 2 shows a state after login.
Reference numeral g201 denotes a login / logout button image.
A symbol g202 is a button image for starting / ending the conference system 1. A symbol g203 is a display that is lit when the conference system 1 is in use.
A symbol g204 is a button image for displaying the minutes stored in the minutes storage unit 226 and reproducing the audio signal.
A symbol g205 is a button image for the user of the parent device 2 to select whether or not the sound collection unit 201 is used.

A symbol g211 is an icon image corresponding to a person who speaks among participants.
Symbol g212 is text information input by the first participant operating the operation unit (202 or 302).
Reference numeral g213 is a pictograph input by the first participant operating the operation unit (202 or 302).
Reference numeral g214 is information indicating the date and time when the first participant speaks.
Symbol g215 is the name of the first participant.

Symbol g221 is text information input by the second participant operating the operation unit (202 or 302).
A symbol g222 is text information obtained by voice recognition of the content uttered by the second participant.
A symbol g223 is an icon indicating that text has been input by operating the operation unit (202 or 302).
A symbol g224 is an icon indicating that an utterance is input by the sound collection unit (201 or 301).

Symbol g231 is text information obtained by voice recognition of the content uttered by the third participant.
Symbol g241 is text information input by the first participant operating the operation unit (202 or 302). Note that this text information indicates the middle of input.

  Symbol g251 is an example of an alert for prohibiting other participants from speaking because the first participant is inputting text. The alert is, for example, “Please wait while Mr. Suzuki (Note; first participant) is entering text”. Thus, by prohibiting the speech of other participants during text input, according to this embodiment, it is possible to wait for the speech of other participants.

  Reference g261 is an example of an alert displayed on the display device 21 when the first participant, Mr. Suzuki, is using the parent device 2. The alert is, for example, “Mr. Suzuki (first participant);“ Please resume when input is completed ””. When the participant inputs text using the handset 3, an alert indicated by reference numeral g261 is displayed on the display unit 304 of the handset 3 used by the participant. In this case, the parent device 2 determines which child device 3 is inputting the text based on the information received by the communication unit 211, and based on the determined result, sets the warning information to the corresponding child device. 3 to send. The subunit | mobile_unit 3 displays the alert of the code | symbol g261 based on the received warning information.

  The image shown in FIG. 2 is an example, and the image displayed on the display device 21 is not limited to this. For example, when a participant who uses the parent device 2 performs text input using the operation unit 202, a “Resume OK” button image (FIG. 4, g310) is further displayed on the image g10. When the text input is completed, the participant instructs to allow other participants to input the text by pressing the “resume OK” button image. Thereby, the input determination unit 241 stops outputting the utterance prohibition instruction.

<Display screen of handset 3>
Next, an example of an image displayed on the display unit 304 of the slave unit 3 will be described.
FIG. 3 is a diagram showing an example of participant information images displayed on the display unit 304 of the child device 3 according to the present embodiment.
The image g30 is an image displayed on the display unit 304 of the child device 3 for displaying participant information.

An area g301 is an area for displaying a conference discussion theme.
The area g302 is an area for displaying information related to the user of the child device 3. This area is an example in which an icon corresponding to the participant, the name of the participant, and an icon indicating that the participant uses the operation unit 302 are displayed.

  A region g303 is a region indicating a state. In the example shown in FIG. 3, the participant is inputting text.

  An area g304 is an area for displaying participant information. In this area, an icon corresponding to each participant, an icon indicating that each participant uses the sound collection unit (201 or 301), and an operation unit (202 or 302) are used for input. An icon and the name of each participant are displayed.

  An area g305 is an area for displaying an operation button image or the like. The operation buttons are, for example, a button for returning to the previous screen, a button for returning to the home screen, and the like.

FIG. 4 is a diagram illustrating an image example of minutes information displayed on the display unit 304 of the child device 3 of the participant who is performing text input according to the present embodiment. FIG. 4 is a screen displayed when a participant who is a user of the handset 3 is inputting text.
The image g30a is an image displayed on the display unit 304 of the child device 3 at the time of minutes information.

Reference g310 is a restart OK button image that permits the restart of another participant's speech after the text input is completed.
Symbol g311 is an alert displayed on the child device 3 of the participant who is inputting the text. The alert is, for example, “Please issue“ Restart OK when input is completed ””.

Reference numeral g312 is text information obtained by voice recognition of the content uttered by the second participant.
Symbol g313 is text information input by the first participant.
A region g314 is a region for the text input unit. The image in this area is when text input is not prohibited, that is, when no other participant is inputting text.
A symbol g315 is text information input by a participant who is a user of the child device 3. The operation unit 302 may be a software keyboard displayed on the display unit 304, and may be connected to the slave unit 3 by wire or wirelessly.

FIG. 5 is a diagram showing an example of minutes information displayed on the display unit 304 of the child device 3 of the participant who is not performing text input according to the present embodiment. FIG. 5 is a screen on which the second participant (Mr. Fujisawa) is displayed while the first participant (Mr. Suzuki) is inputting text.
The image g40a is an image displayed on the display unit 304 of the handset 3 of the second participant during the minutes information.

  Reference sign g411 is an alert displayed on the child device 3 of the second participant (Mr. Fujisawa) when the first participant (Mr. Suzuki) is inputting text. The alert is, for example, “Please wait for a while while Mr. Suzuki is entering text”. This alert is displayed, for example, in an area (g302, FIG. 3) for displaying information related to the user of the slave unit 3. In the example of FIG. 5, an icon image indicating that the participant inputs using the sound collection unit 301 and the operation unit 302 is displayed in an area for displaying information regarding the user of the handset 3.

  Reference numeral g412 is an image of an alert that prohibits the user of the handset 3 from entering text because the first participant (Mr. Suzuki) has entered text. The alert is “Please wait while Mr. Suzuki enters text”. This alert is displayed, for example, in the text input area.

  3 to 5 are examples, and the image displayed on the display unit 304 is not limited to this.

FIG. 6 is a diagram illustrating an example of icon images displayed on the display device 21 of the parent device 2 or the display unit 304 of the child device 3 according to the present embodiment.
The image in the region indicated by reference sign g51 is an example of an icon image for a participant who is inputting through the sound collection unit (201 or 301; also referred to as a microphone). In this way, the display of the user using the sound collection unit corresponds to, for example, the third participant (Mr. Yoshida) in FIG.

  The image in the region indicated by reference sign g52 is an example of an icon image for a participant who is inputting through the operation unit (202 or 302; also referred to as a slave unit). Thus, the display of the user using the operation unit corresponds to, for example, the first participant (Mr. Suzuki) in FIG.

  The image in the area indicated by reference sign g53 is an example of an icon image for a participant who is inputting through the sound collection unit (201 or 301; also referred to as a microphone) and the operation unit (202 or 302). In this way, the display of the user using the sound collection unit and the operation unit corresponds to, for example, the second participant (Mr. Fujisawa) in FIG.

<Information stored in the storage unit 240>
Next, an example of information stored in the storage unit 240 of the parent device 2 will be described.
FIG. 7 is a diagram illustrating an example of information stored in the storage unit 240 of the parent device 2 according to the present embodiment.
As illustrated in FIG. 7, the storage unit 240 stores, for each participant, an icon and a use unit (sound collection unit or operation unit) associated with the participant name. In the example illustrated in FIG. 7, the first participant name is “Suzuki”, the icon is “icon 1”, and the usage unit is “operation unit”.

<Processing performed by base unit 2>
Next, an example of a processing procedure performed by the parent device 2 will be described.
FIG. 8 is a flowchart illustrating an example of a processing procedure performed by the parent device 2 according to the present embodiment.

  (Step S1) The user operates the operation unit 202 to perform a login procedure. For example, the user inputs identification information (user ID) and a password for identifying the user. Subsequently, the authentication unit 212 performs a login procedure based on the operation result of the operation unit 202 being operated.

  (Step S2) The user operates the operation unit 202 to perform settings such as icon selection and name input. The user may operate the operation unit 202 to set whether input is performed by the sound collection unit 201 or the operation unit 202. Alternatively, the user may set whether input is performed by the sound collection unit 201 or the operation unit 202 by utterance. The input determination unit 241 stores the set information in the storage unit 240. Note that the input determination unit 241 acquires setting information of other participants via the communication unit 211, and stores the acquired setting information of other participants in the storage unit 240.

  (Step S <b> 3) The input determination unit 241 determines whether text input is being performed based on the operation result output from the operation unit 202 of the parent device 2 or the text information output from the communication unit 211. If the input determination unit 241 determines that text input is being performed (step S3; YES), the process proceeds to step S4. If it is determined that text input is not being performed (step S3; NO), the process proceeds to step S6. Proceed.

  (Step S4) The input determination unit 241 determines whether or not the text input is completed. Specifically, the input determination unit 241 determines that the text input is completed by detecting that the “resume OK” button image (FIG. 4, g310) is pressed. If the input determination unit 241 determines that the text input has ended (step S4; YES), the process proceeds to step S8. If the input determination unit 241 determines that the text input has not ended (step S4; NO), the input determination unit 241 proceeds to step S5. Proceed to processing.

  (Step S5) The input determination unit 241 continues outputting the utterance prohibition instruction. Subsequently, the warning generation unit 244 continues to output warning information. Thereby, the main | base station 2 alert | reports an alert to the display apparatus 21. FIG. In addition, the slave unit 3 causes an alert to be notified on the display unit 304. After the process, the input determination unit 241 returns to the process of step S3.

(Step S <b> 6) The voice recognition unit 222 performs voice recognition processing on the voice signal collected by the sound collection unit 201 and the sound collection unit 301. After the process, the speech recognition unit 222 proceeds to the process of step S7.
(Step S7) The text conversion unit 223 performs text conversion processing on the result of speech recognition. After the process, the text conversion unit 223 proceeds to the process of step S8.

(Step S8) The dependency analysis unit 224 performs dependency analysis and morphological analysis processing on the text information subjected to text conversion for each speaker. After the process, the dependency analysis unit 224 proceeds to the process of step S9.
(Step S9) The minutes creation unit 225 creates the minutes based on the text information output by the dependency analysis unit 224.

  (Step S <b> 10) The communication unit 211 transmits the created minutes data to the child device 3. As described above, the master unit 2 creates the minutes based on the text information obtained by voice recognition of the voice signal input by the sound collection unit (201 or 301) or the text information input by the operation unit (202 or 302). Is transmitted to the handset 3 possessed by the participant, the minutes can be shared.

The base unit 2 repeats the processes of steps S3 to S10 below.
Note that the processing of FIG. 8 is an example, and the present invention is not limited to this. When the participant participates in the conference from the middle, the setting process in step S2 may be performed during the processes in steps S3 to S10.

As described above, in the present embodiment, during the period when the participant who uses the handset 3 or the base unit 2 is inputting text, the speech (utterance, text input) of other participants is prohibited.
Thereby, according to this embodiment, since a hearing impaired person prohibits a speech utterance of a normal hearing person while inputting text with the subunit | mobile_unit 3, a hearing impaired person can follow the content of a meeting. Thereby, according to this embodiment, a hearing impaired person becomes easy to participate in a meeting with a normal hearing person.

In the above-described example, an example is described in which an alert is displayed in order to wait for another participant's speech (utterance, text input) during text input. However, the present invention is not limited to this.
For example, the processing unit 303 of the child device 3 may not display the text input field (FIG. 5, g412) while another participant is inputting text based on the warning information received from the parent device 2. .
Further, the parent device 2 or the child device 3 may not accept text input while other participants are inputting text, and may not accept voice input. Even in this case, since there is anxiety that there is no reaction even if the user inputs, it is preferable that the parent device 2 and the child device 3 display an alert that prohibits speaking.

<Modification>
In the above-described example, the master unit 2 controls the display on the display unit 304 of the slave unit 3 to limit the text input of other participants during text input. However, the present invention is not limited to this. .
For example, a normal hearing person may use only the sound collection unit without using the handset 3.
FIG. 9 is a block diagram illustrating a configuration example of the sound collection unit in the modification. As illustrated in FIG. 9, the sound collection unit 501 of the modification includes a notification unit 511 and a communication unit 512.

When receiving the warning information from the parent device 2, the communication unit 512 generates alert information based on the received warning information, and outputs the generated alert information to the notification unit 511.
The notification unit 511 notifies that the utterance (utterance, text input) is restricted based on the alert information output from the communication unit 512. The notification unit 511 is, for example, a lamp, an LED (light emitting diode), a liquid crystal display device, or the like.
The user can know that the input of the utterance (utterance, text input) is restricted by this notification. As a result, during the period when the user is restricted, according to the present embodiment, when the participant is inputting text, speech (utterance, text input) can be restricted.

  In the modification, the example in which the sound collection unit includes the communication unit and the notification unit has been described, but the operation unit 302 (FIG. 1) may include the communication unit and the notification unit. In this case, the participant wearing the operation unit 302 can know that speech (utterance, text input) is restricted based on information notified by the notification unit of the operation unit 302.

  Note that a program for realizing all or part of the functions of the parent device 2 or the child device 3 in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. By executing, all or part of the processing performed by the parent device 2 or the child device 3 may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, and various deformation | transformation and substitution are within the range which does not deviate from the summary of this invention. Can be added.

DESCRIPTION OF SYMBOLS 1 ... Conference system, 2 ... Master machine, 201 ... Sound collection part, 202 ... Operation part, 211 ... Communication part, 212 ... Authentication part, 221 ... Acoustic model and dictionary memory | storage part, 222 ... Voice recognition part, 223 ... Text conversion 224 ... Dependency analysis part 225 ... Minutes creation part 226 ... Minutes storage part 231 ... Text acquisition part 240 ... Storage part 241 ... Input determination part 242 ... Icon setting part 243 ... Icon storage 244 ... Warning generator 245 ... Image output unit 3 ... Slave unit 301 ... Sound pickup unit 302 ... Operation unit 303 ... Processing unit 304 ... Display unit 305 ... Communication unit

Claims (7)

A handset equipped with a sound collection unit, a text input unit, and a display unit;
Using the text information that is connected to the slave unit and voice-recognized the voice input from the slave unit or the text information input from the slave unit, the minutes are created and the created minutes are stored in the slave unit In a conference system equipped with a parent machine shared with the machine,
When the base unit participates in a conversation by text, the base unit controls the base unit to wait for the speech of another conference participant, and transmits information for waiting for the speech to the slave unit. .   The conference system according to claim 1, wherein the parent device automatically determines whether the input is made by voice or text on the child device side and displays the fact on the parent device and the child device.   3. The conference system according to claim 1, wherein, when text is input on the parent device side or the child device side, the parent device displays a message to that effect on the parent device and the child device in a dialog. 4. . When the text input is completed, the slave unit transmits information indicating that the text input is completed to the master unit,
The conference according to claim 3, wherein the master unit permits voice input to the master unit and the plurality of slave units based on information indicating completion of text input received from the slave unit. system. A sound collection device including a communication unit, a notification unit, and a sound collection unit;
When the base unit participates in a conversation by text, the base unit transmits information to wait for the speech of another conference participant to the sub unit and the sound collection device,
The conference system according to any one of claims 1 to 4, wherein the sound collection device notifies that a speech is prohibited based on information for waiting for a speech from the parent device. Text information obtained by voice recognition of voice input from the slave unit connected to the slave unit and a slave unit including a sound collection unit, a text input unit, and a display unit, or text information input from the slave unit A conference system control method in a conference system comprising: a master device that creates a minutes using the master device, and a master device that shares the created minutes with the slave device,
Controlling the parent machine to wait for the utterances of other conference participants when the parent machine participates in a conversation by text; and
When the parent device participates in a conversation by text, transmitting information to make the utterance stand by so as to wait for the utterance of another conference participant, to the child device;
A conference system control method. Text information obtained by voice recognition of voice input from the slave unit connected to the slave unit and a slave unit including a sound collection unit, a text input unit, and a display unit, or text information input from the slave unit To the computer of the parent machine of the conference system comprising the parent machine that creates the minutes and shares the created minutes with the child machine,
Controlling the parent machine to wait for the speech of another conference participant when participating in a conversation by text;
Sending information to the slave unit to wait for speech to wait for the speech of another conference participant when participating in a conversation by text; and
A program that executes

Images