JP2019179081A - Conference support device, conference support control method, and program - Google Patents

Abstract

To provide a conference support device, a conference support control method, and a program that can select a conference desired to be continued with a voice even after the conference is restarted after being interrupted.SOLUTION: A conference support device comprises: a sound collection part; a voice recognition part which performs voice recognition on an utterance voice-input by the sound collection part using a dictionary for voice recognition; a display part which displays a voice-recognized text as a minute book; a storage part which stores minute books and also stores information representing interruption of a minute book of a conference being interrupted associated with the minute book; and a narrowing-down selection part which displays candidates for interrupted conferences when an interrupted conference is restarted, and narrows down the displayed candidates based upon a result of the voice recognition on the voice utterance to determine a candidate for a conference desired to be continued.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a conference support apparatus, a conference support control method, and a program.

  In the case where a plurality of people hold a meeting, it is proposed that the utterance content of each speaker is converted into text, and the utterance content is converted into text and displayed on a playback device owned by each user (see, for example, Patent Document 1). . In the technique described in Patent Document 1, the utterance is recorded as a voice memo for each topic, and the minutes creator reproduces the recorded voice memo and converts it into text. And with the technique of patent document 1, the minutes are created using what was converted into text.

Japanese Patent Laid-Open No. 08-194492

The conference may end with a single session, but may be interrupted or may not end with a single session. In such a meeting, the meeting is resumed separately.
However, in the technique described in Patent Document 1, since the minutes are created for each meeting, even if the meeting is resumed, the minutes are different.

  The present invention has been made in view of the above problems, and can provide a conference support apparatus, a conference support control method, and a conference support control method that can select a conference to be continued even if the conference is resumed after being suspended. The purpose is to provide a program.

  (1) In order to achieve the above object, a conference support apparatus (1) according to an aspect of the present invention provides a sound collection unit (101) and a speech recognition unit for speech input by the sound collection unit. A speech recognition unit (speech recognition unit 104, text conversion unit 105, dependency analysis unit 106) that performs speech recognition using a dictionary (speech recognition dictionary DB 103), and a display unit that displays the speech-recognized text as minutes (The minutes creation unit 107, the image generation unit 110, the display unit 111) and the minutes are stored in the minutes, and information indicating that the minutes for the meeting that has been suspended is suspended is stored in the minutes. In the case of resuming an interrupted meeting with a storage section (the minutes storage section 108) that stores the information in association, the interrupted meeting candidate is displayed, and the displayed meeting candidate is recognized as a voice utterance. Based on the results Comprising a selection unit to narrow down the candidates of the conference to be (Minutes selection unit 109), the.

  (2) Moreover, in the conference support apparatus according to an aspect of the present invention, the selection unit is configured to select a conference to be continued, a meeting attendee, a time related to the conference, and a keyword extracted from the minutes. You may make it narrow down by at least one of them.

  (3) Moreover, in the conference support apparatus according to an aspect of the present invention, the voice recognition unit may use a voice recognition dictionary used in a continuous meeting.

  (4) Further, in the conference support apparatus according to one aspect of the present invention, the selection unit additionally presents the attendees of the conference when narrowing down the conference to be continued for the second time or later, and the minutes It is also possible to add at least one of the presentations by adding the keywords extracted from.

  (5) In the conference support device according to one aspect of the present invention, the slave device (3, 3a, 3b,...) Connected to the conference support device by communication, A sound unit (301), a slave unit display unit (display unit 304), and an operation unit (302) are provided, and the voice recognition unit uses the voice recognition dictionary for voice signals transmitted by the slave unit. Performing speech recognition, and performing speech recognition on the speech signal collected by the sound collection unit and the new speech word collected by the sound collection unit of the handset, the display unit and the handset display unit You may make it display on.

  (6) In order to achieve the above object, a conference support control method according to an aspect of the present invention is a conference support method in a conference support apparatus including a storage unit that stores the minutes of a conference, and the sound collection unit includes: A sound collection step for acquiring a speech input speech, a speech recognition unit for performing speech recognition using a speech recognition dictionary for the speech input in the sound collection step, and a minutes A creation section for creating a minutes of the text recognized by the speech recognition step and storing information indicating that the meeting is interrupted in association with the meeting when the meeting is interrupted; When the selection unit resumes the suspended meeting, the meeting candidate is displayed, and the displayed meeting candidate is to be continued based on the result of voice recognition of voice utterance. Including a selection step to narrow down the candidate, the.

  (7) In order to achieve the above object, a program according to an aspect of the present invention includes a sound collection step of acquiring an utterance input by voice to a computer of a conference support apparatus including a storage unit that stores conference minutes. A speech recognition step of performing speech recognition using a speech recognition dictionary for the speech input in the sound collection step, and creating a minutes of the text recognized by the speech recognition step, When a meeting is interrupted, a minutes creation step for storing information indicating that the meeting has been suspended is stored in association with the minutes, and when the suspended meeting is resumed, the suspended meeting candidate is displayed and displayed. A selection step of narrowing down the conference candidates to be continued based on the result of speech recognition of the speech utterance.

  According to the above (1), (6) or (7), it is possible to select a conference to be continued by a simple method by voice input.

According to the above (2), there is an effect that it is easy to narrow down.
According to (3) described above, conversion of homonyms and the like can be performed appropriately.
According to the above (4), when candidates cannot be narrowed down by a single voice input, information related to the conference can be added and presented, so that it is easy to narrow down.
According to (5) described above, since the minutes of the conference can be displayed on the slave unit, it is possible to make it easier for a normal hearing person and a hearing impaired person to participate in the conference together.

It is a block diagram which shows the structural example of the meeting assistance apparatus which concerns on this embodiment. It is a figure which shows the example of information which the minutes memory | storage part which concerns on this embodiment memorize | stores. It is a figure which shows the example of creation of the meeting state and minutes based on this embodiment. It is a figure which shows the example of information shown on a display part when restarting the interrupted meeting which concerns on this embodiment. It is a figure for demonstrating the information which the conference record memory | storage part concerning this embodiment memorize | stores. It is a flowchart which shows the example of a process sequence which a meeting assistance apparatus performs at the time of the meeting start which concerns on this embodiment. It is a figure which shows the example of an image displayed on a display part during the meeting which concerns on this embodiment, and the meeting end. FIG. 7 is an example of an image displayed on the display unit when conference end or conference continuation is selected. It is a figure which shows the example of an image displayed on a display part at the time of the meeting start which concerns on this embodiment. It is another example of the image of the interrupted meeting list which concerns on this embodiment. It is a figure which shows the example of an image displayed when the minutes cannot be narrowed down at the time of the meeting continuation in the 1st modification of this embodiment. It is a block diagram which shows the structural example of the meeting assistance system in the 2nd modification which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, an example of a situation in which the conference support apparatus according to the present embodiment is used will be described.
The conference support apparatus according to the present embodiment is used in a conference held by two or more people, for example. When the conference ends, the conference support device assigns and stores a flag indicating whether the conference is interrupted or ended. The conference support apparatus stores information related to the conference (participants, keywords, etc.) in association with the minutes. When the conference is resumed, the conference support device presents candidate minutes from the conference minutes that have been suspended based on the result of voice recognition of the user's voice. The conference support apparatus selects a conference to be resumed from the candidates according to the user's voice, and continues to create the minutes of the resumed conference.

FIG. 1 is a block diagram illustrating a configuration example of a conference support apparatus 1 according to the present embodiment.
As shown in FIG. 1, the conference support apparatus 1 includes a sound collection unit 101, an acquisition unit 102, a speech recognition dictionary DB 103 (speech recognition dictionary), a speech recognition unit 104, a text conversion unit 105 (speech recognition unit), a dependency. Analysis unit 106 (voice recognition unit), minutes creation unit 107 (keyword extraction unit), minutes storage unit 108, minutes selection unit 109 (selection unit), image generation unit 110, display unit 111, and operation unit 112 Prepare.

  The meeting support apparatus 1 is, for example, a tablet terminal, a smartphone, a notebook computer, or the like.

  The sound collection unit 101 is a microphone. The sound collection unit 101 collects the user's voice signal and outputs the collected voice signal to the acquisition unit 102. The sound collection unit 101 is attached to each participant. The sound collection unit 101 may be a microphone array.

  The acquisition unit 102 converts the audio signal output from the sound collection unit 101 from an analog signal to a digital signal, and outputs the audio signal converted into the digital signal to the audio recognition unit 104. When there are a plurality of sound collection units 101, the acquisition unit 102 performs, for example, time-division processing for each sound collection unit 101.

  The speech recognition dictionary DB 103 is a database that stores, for example, an acoustic model, a language model, a word dictionary, and the like. The acoustic model is a model based on the feature amount of sound, and the language model is a model of information on words and how to arrange them. The word dictionary is a dictionary with a large number of vocabularies, for example, a large vocabulary word dictionary. The conference support apparatus 1 may store and update words and the like that are not stored in the speech recognition dictionary DB 103. Note that the speech recognition dictionary DB 103 includes a DB (database) for each conference, for example. For example, the first DB is for a general conference, the second DB is for a presentation, and the third DB is for an international conference. In this way, conversion of homonyms and the like can be performed appropriately by using a DB that matches the conference.

The voice recognition unit 104 acquires a voice signal output from the acquisition unit 102. The voice recognition unit 104 detects a voice signal in the utterance section from the voice signal. For the detection of the utterance section, for example, an audio signal having a predetermined threshold value or more is detected as the utterance section. Note that the speech recognition unit 104 may detect the utterance section using another known method. The speech recognition unit 104 performs speech recognition on the detected speech signal in the utterance section with reference to the speech recognition dictionary DB 103 using a known method. Note that the voice recognition unit 104 performs voice recognition using, for example, a method disclosed in JP-A-2015-64554. The voice recognition unit 104 outputs the recognized recognition result and the voice signal to the text conversion unit 105. Note that the speech recognition unit 104 outputs the recognition result and the speech signal in association with each other, for example, for each sentence, between utterance phrases, or for each speaker.
Note that the speech recognition unit 104 performs speech recognition for each sound collection unit 101 by, for example, time division processing when speech signals are input simultaneously. In addition, when the microphone is a microphone array, the voice recognition unit 104 also performs known voice recognition processing such as sound source separation processing, sound source localization processing, and sound source identification processing.
The speech recognition unit 104 switches the speech recognition dictionary DB 103 used for speech recognition processing and the like based on information indicating the speech recognition dictionary DB 103 to be used output from the minutes selection unit 109.

  The text conversion unit 105 converts the recognition result output from the voice recognition unit 104 into text with reference to the voice recognition dictionary DB 103. Note that the text information includes information of at least one character. The text conversion unit 105 outputs the converted text information and the acquired voice signal to the dependency analysis unit 106. Note that the text conversion unit 105 may delete the interjections such as “Ah”, “Ut”, “Eh”, “Oh” from the result of recognizing the utterance information and convert it to text.

The dependency analysis unit 106 performs morphological analysis and dependency analysis on the text information output from the text conversion unit 105 with reference to the speech recognition dictionary DB 103. In the dependency analysis, for example, SVM (Support Vector Machines) is used in the shift-reduce method, the spanning tree method, and the chunk identification step application method.
When resuming the conference, the dependency analysis unit 106 outputs the text information subjected to the morphological analysis and dependency analysis and the analysis result to the minutes selection unit 109.
The minutes are created after detecting a predetermined keyword such as “Start meeting”. Alternatively, the creation of the minutes is started based on the result of the user operating the operation unit 112 with an icon (for example, see FIG. 7) displayed on the display unit 111. After the start of the meeting, the dependency analysis unit 106 outputs the text information subjected to morphological analysis and dependency analysis and the analysis result to the minutes creation unit 107. The dependency analysis unit 106 also outputs the audio signal output from the text conversion unit 105 to the minutes creation unit 107.
Further, when the minutes creation unit 107 creates the minutes after the meeting is ended or interrupted, the dependency analysis unit 106 gives the speaker and the duration of the analysis to the analysis result and outputs the result to the image generation unit 110. Note that the speaker (participant) name may be stored in advance by the dependency analysis unit 106 for each sound collection unit 101 used by the participant, or may be input by the participant at the start of the conference.

The minutes creation unit 107 creates minutes separately for each speaker based on the text information output by the dependency analysis unit 106. The minutes creation unit 107 stores the created minutes (text information) and the corresponding audio signal in the minutes storage unit 108. Note that the minutes creation unit 107 may create the minutes during the conference, or may create the minutes after the conference is ended or interrupted. The minutes creation unit 107 extracts keywords in the meeting when creating the minutes or after the minutes are created, and stores the extracted keywords in association with the minutes. The minutes creation unit 107 extracts keywords using, for example, keyword matching, surface layer patterns, the method of the invention described in Japanese Patent Laid-Open No. 2008-146150, and the like. Alternatively, the minutes creation unit 107 may extract words having a large number of appearances as keywords. In addition, the minutes creation unit 107 assigns the utterer and the utterance date and time for each utterance. Note that the name of the speaker (participant) may be stored in advance by the minutes creation unit 107 for each sound collection unit 101 used by the participant, or may be input by the participant by speaking at the start of the conference. The minutes creation unit 107 may create minutes by deleting the interjections such as “Ah”, “Et”, “Eh”, “Well”.
The minutes creation unit 107 continues (restarts) the creation of the selected minutes in accordance with the information indicating the selected minutes output from the minutes selection unit 109 and the continuation instruction.

  The minutes storage unit 108 stores the minutes and audio signals in association with each other. Note that the minutes storage unit 108 associates and stores the minutes and the keywords, and further associates the meeting state (finished or suspended) with the minutes. Information stored in the minutes storage unit 108 will be described later.

The minutes selection unit 109 acquires the morphological analysis output by the dependency analysis unit 106 and the text information subjected to the dependency analysis, and the analysis result. The minutes selection unit 109 searches the minutes storage unit 108 for the minutes meeting the conditions based on the acquired information. The minutes selection unit 109 outputs information on the searched minutes to the image generation unit 110. In addition, the minutes selection unit 109 outputs the text information of the utterances made for selecting the minutes before the start of the meeting to the image generation unit 110.
In addition, when the minutes creation unit 107 creates the minutes during the meeting, the minutes selection unit 109 reads the created minutes from the minutes storage unit 108 and generates an image of the read minutes Output to the unit 110.
The minutes selection unit 109 selects a meeting to be continued based on a result of performing speech recognition processing or the like on the voice signal input by the utterance from the suspended meetings. The minutes selection unit 109 outputs information indicating the minutes of the selected meeting and a continuation instruction to the minutes creation unit 107. In addition, the minutes selection unit 109 outputs information indicating the voice recognition dictionary DB 103 to be used included in the minutes of the selected meeting to the voice recognition unit 104.

  The image generation unit 110 generates image data based on the information on the minutes output from the minutes selection unit 109 and the text information. The image generation unit 110 outputs the generated image data to the display unit 111.

  The display unit 111 is, for example, a liquid crystal display device, an organic EL display device, an electronic ink display device, or the like. The display unit 111 displays the image data output from the image generation unit 110.

  The operation unit 112 is, for example, a touch panel sensor provided on the display unit 111 or a priority connection or wireless connection keyboard. The operation unit 112 detects a user operation and outputs the detected result to the minutes selection unit 109.

<Information stored in the minutes storage unit 108>
Next, an example of information stored in the minutes storage unit 108 will be described.
FIG. 2 is a diagram illustrating an example of information stored in the minutes storage unit 108 according to the present embodiment.
As shown in FIG. 2, the minutes storage unit 108 stores the number of the meeting, the attendees, the latest update date / time of the minutes, the meeting state, and the keywords in association with each other. For example, the minutes storage unit 108 has the name of the meeting “Management Meeting A”, the number “1”, the attendees “Yamada, Sato, Takeda”, and the last update date “2018.1.13 17:30”. And the conference state “suspended” and the keyword “development policy, cost reduction, business improvement,...” Are stored in association with each other. Note that the name of the conference may be input by the user by voice, or may be input by operating the operation unit 112. The keyword may be input by the user by voice, or may be input by operating the operation unit 112. The last update date / time is, for example, the date / time when the conference is ended or interrupted.

<Conference status>
Next, an example of creating a meeting state and minutes will be described.
FIG. 3 is a diagram showing an example of creating a meeting state and minutes according to the present embodiment. In FIG. 3, the horizontal axis represents time t.
In the example indicated by reference sign g11, the new conference is started at time t1 and ended at time t2.
The example indicated by reference sign g12 is an example in which a new conference is started at time t11, interrupted at time t12, rested during the period from time 12 to t13, resumed at time t13, and ended at time t14.

  In the example indicated by reference sign g13, the new meeting A is started at time t21 and is interrupted at time t22, a break is entered during the period of time 22 to t23, and the new meeting B is started at time t23 and is interrupted at time t24. This is an example in which a break is entered during the period of t25, and the new meeting A is resumed at time t25 on the next day and ended at time t26.

Thus, depending on the conference, resumption may be on another day. In addition, another conference may be held between the interrupted conferences, and other conferences may be in a suspended state. In the case of the symbol g13, since there are two suspended conferences, it is necessary to select which conference to resume when the conference is resumed. When there are many suspended meetings, or when the suspended date and time is several days ago or several weeks ago, it is difficult to find the minutes that are the data of the meeting that you want to continue with just the meeting name.
For this reason, in this embodiment, according to the user's voice input, information (meeting name, attendee, last update date / time of meeting, meeting status, keyword) regarding the meeting minutes that have been interrupted is presented. Further, narrowing down is performed by voice input of the user from the presented information.

<Information presented on the display when resuming an interrupted meeting>
Next, an example of information presented on the display unit when resuming the interrupted conference will be described.
FIG. 4 is a diagram illustrating an example of information presented on the display unit when resuming the interrupted conference according to the present embodiment.

  The image g20 is an image example first presented on the display unit 111 when resuming the interrupted conference. Reference g21 is a list of interrupted meetings, and the list can be scrolled up and down by operating the operation unit 112 on the right scroll unit. Reference sign g22 is a result of the voice recognition process performed on the voice input by the user to select the conference to be resumed. In this case, the voice input is “I want to resume the continuation of the meeting with Mr. Ito”.

The image g30 is an image presenting candidate minutes extracted based on the voice input in the image g20.
The symbol g31 is a first candidate and includes a number, a meeting name, a participant, a last update date and time, and a keyword. The symbol g32 is a second candidate and includes a number, a meeting name, a participant, a last update date and time, and a keyword.
A symbol g33 is a result of performing speech recognition processing on the speech input by the user. In this case, the voice input is “Please give manager meeting B”.

The image g40 is an image presenting candidate minutes extracted based on the voice input in the image g30.
A symbol g41 is a candidate, and includes a number, a meeting name, a participant, a last update date and time, and a keyword.
The symbol g42 is the result of the user performing speech recognition processing on the speech input by the user. In this case, the voice input is “Thank you.”

The image g50 is an image displayed on the display unit 111 when the meeting is resumed after the minutes of the meeting to be resumed are selected.
Symbol g51 is text information obtained as a result of voice recognition of the speech of the conference participant. The text information includes, for example, the speaker and the utterance date / time.
The display example shown in FIG. 4 is an example, and the present invention is not limited to this.

FIG. 5 is a diagram for explaining information stored in the conference record storage unit 118 according to the present embodiment.
As shown in FIG. 5, when the conference state is ended, the minutes storage unit 118 stores (saves) the minutes of the meeting, but cannot continuously record the minutes.
When the meeting state is interrupted, the minutes storage unit 118 stores (saves) the minutes of the meeting, and when the resume is selected for the minutes, the minutes are resumed to the interrupted minutes. Minutes can be added to the meeting and recorded.

<Processing of Conference Support Device 1>
Next, an example of a processing procedure performed by the conference support apparatus 1 will be described.
FIG. 6 is a flowchart illustrating an example of a processing procedure performed by the conference support apparatus 1 at the start of the conference according to the present embodiment.

  (Step S1) The user performs a login procedure by voice input. For example, the user inputs identification information (user ID) and a password for identifying the user. Alternatively, the user operates the operation unit 112 to perform a login procedure. The conference support apparatus 1 performs a login procedure for each participant.

  (Step S2) The minutes selection unit 109 sets whether to start a new meeting or resume an interrupted meeting based on the voice of the participant or the result of the operation unit 112 being pointed. In addition, the process of step S3-S11 is a process at the time of restarting a meeting. In the case of a new conference, steps S3 to S1 are not performed. In addition, the conference support apparatus 1 performs conference settings such as registration of names of participants who participate in the conference and registration of the sound collection unit 101 used by the participants. The conference setting includes settings such as whether the sound collection unit 101 is a pin microphone or a microphone array. Note that the minutes creation unit 107 stores the setting information of these meetings in the minutes storage unit 108 in association with the minutes. Note that the minutes creation unit 107 may not store conference setting information for a completed conference.

  (Step S3) The minutes selection unit 109 searches the information stored in the minutes storage unit 108, and the image data of the list of minutes of the meeting that has been interrupted (eg, FIG. 4, image g20, code g21) , And the generated image data is output to the image generation unit 110 and displayed on the display unit 111. Subsequently, the minutes selector 109 selects image data for prompting speech in order to select which conference to continue (for example, “Continuing conference in FIG. 4, image g20 is as follows. Please generate ") and output the generated image data to the image generation unit 110 to display on the display unit 111.

  (Step S4) The participant speaks (eg; FIG. 4, image g20, “I want to resume the continuation of the meeting with Mr. Ito”). The sound collection unit 101 acquires the spoken voice.

  (Step S5) The conference support apparatus 1 performs voice recognition processing, text conversion processing, and dependency analysis processing on the voice signal picked up by the sound pickup unit 101.

  (Step S6) The minutes selection unit 109 is suspended according to the conditions based on the text information acquired by the process of step S5 (eg, “I want to resume the continuation of the meeting with Mr. Ito”). The minutes of the conference being narrowed down are read out from the minutes storage unit 108. The word used for narrowing down is at least one of the participant name, the time related to the conference (the start date and time or the end date and time), and the keyword used in the conference.

  (Step S7) The minutes selection unit 109 generates image data (for example, FIG. 5, image g30, code g31, code g32) of the meeting minutes meeting the read conditions, and the generated image data is an image generation unit. 110 to display on the display unit 111.

  (Step S8) The participant speaks (eg; FIG. 4, reference g33 of the image g30 “Please give me a managerial meeting B” or FIG. 4, reference g42 of the image g40 “Thank you.”) I do. The sound collection unit 101 acquires the spoken voice.

  (Step S9) The conference support apparatus 1 performs voice recognition processing, text conversion processing, and dependency analysis processing on the voice signal picked up by the sound pickup unit 101.

  (Step S10) The minutes selection unit 109 determines whether or not the meetings that are to be resumed (required to be continued) have been narrowed down. For example, the minutes selection unit 109 determines that the utterance content agrees with the displayed result, and determines that the utterance has not been narrowed down. Alternatively, if the number of candidate minutes displayed on the display unit 111 in step S7 is one, the minutes selection unit 109 determines that the meeting to be resumed (to be continued) has been narrowed down and displayed in step S7. If there are two or more candidate minutes displayed on the section 111, it is determined that the meeting to be resumed (to be continued) has not been narrowed down. When the minutes selection unit 109 determines that the meetings that it wants to resume (want to continue) have been narrowed down (step S10; YES), it proceeds to the process of step S11 and determines that the meetings that it wants to resume (want to continue) have not been narrowed down In the case (step S10; NO), the process returns to step S7.

  (Step S11) When the conference is resumed by voice input or operation of the operation unit 112, the conference support apparatus 1 resumes the creation of the minutes for the selected minutes. Note that, when the conference is resumed, the conference support apparatus 1 reads the conference settings set in step S2 from the minutes storage unit 108. Thereby, the participant does not need to reset the conference to be resumed. Subsequently, the conference support apparatus 1 continues the conference, and continues to create a single minutes using the content of the continued conference for the selected minutes. Subsequently, the conference support apparatus 1 displays information indicating whether the conference is to be ended or interrupted at the end of the conference (see FIG. 7). Subsequently, when the conference is interrupted, the conference support apparatus 1 adds information (flag) indicating that the conference is interrupted to the minutes and stores the information in the minutes storage unit 108. The conference support apparatus 1 displays conference setting information on the display unit 111. Thereby, the participant can confirm the participant and the number of the sound collection unit 101 used by the participant.

<Images displayed on the display unit 111 during and at the end of the meeting>
Next, an example of an image displayed on the display unit 111 during the meeting and at the end of the meeting will be described.
FIG. 7 is a diagram illustrating an example of an image displayed on the display unit 111 during the meeting and at the end of the meeting according to the present embodiment.
The image g100 is an image displayed on the display unit 111.

  An area g101 is an area for editing participant information. The image in the region g101 includes an icon image corresponding to the participant, the name of the participant, a microphone number (or identification information) used by the participant, and the like.

  The area g102 is an area for displaying minutes. FIG. 7 shows a state after login. The image in the region g102 includes a login / logout button image, a conference start / end button image, a display that lights up during the conference, a button image for displaying the minutes and playing the audio signal, and the sound collection unit 101. A button image for selecting whether or not to use, an image indicating the number of participants, an image obtained as a result of performing speech recognition processing on the speech input of the participants, and the like are included.

When the end of the conference is instructed, for example, when the button image g103 of the end of the conference is selected by operating the operation unit 112, the minutes selection unit 109 outputs an instruction to display the image g110 to the image generation unit 110. Thus, the image g110 is displayed on the display unit 111.
The image g110 includes a conference stop button image g111, a conference end button image g112, and a conference end selection cancel button image g113. The participant operates the operation unit 112 or selects one of these three by voice input.
During the conference, the image g110 is not displayed, and the images of the areas g101 and g102 are displayed on the display unit 111. Participants who are hearing impaired can recognize the speech contents of other participants by recognizing the voice displayed on the display unit 111 and viewing the result.

FIG. 8 is an example of an image displayed on the display unit 111 when the conference end or the conference continuation is selected in FIG.
The image g200 is an image displayed on the display unit 111.

  The image g210 is an image indicating that logout processing is started. The image g210 includes an OK button image g211 that agrees to the processing. Even when the OK button image g211 is not selected, the conference support apparatus 1 performs logout processing after a predetermined time (for example, 3 seconds).

  Note that the images shown in FIGS. 7 and 8 are examples, and the images displayed on the display unit 111 differ depending on the conference. Further, the image g110 and the image g210 are also examples, and the images may be different. For example, the image g210 may include a cancel button image.

<Image displayed on display unit 111 at the start of the conference>
Next, an example of an image displayed on the display unit 111 at the start of the conference will be described.
FIG. 9 is a diagram illustrating an example of an image displayed on the display unit 111 at the start of the conference according to the present embodiment. The user operates the operation unit 112 to make each setting. The image g300 is an image displayed on the display unit 111.
The image in the region g301 is an image for performing conference setting. The image in the area g301 includes, for example, a microphone setting area for voice recognition, a setting area for selecting voice recording, and a setting area for selecting a meeting form (conference, video conference, Waigaya, presentation, etc.). It is. Note that “waigaya” is an abbreviation for “waiwaigayayaya”, and is a conference where people can speak freely.
Each setting area is an example, and the present invention is not limited to this.

As shown in FIG. 9, the conference support apparatus 1 displays an image g310 at the start of the conference. The image g310 includes a new button image g311 that is selected when a new conference is held, and a continuation button image g312 that is selected when the suspended conference is resumed.
The user selects whether to hold a new meeting or continue a suspended meeting by operating the operation unit 112 or by voice input. When continuation is selected, the image g20 shown in FIG. 4 is displayed.

FIG. 10 is another example of an interrupted conference list image according to the present embodiment.
The image g400 is an image displayed on the display unit 111.
An area 301 is an image of a conference setting area as in FIG.
Reference numeral g410 is another example of an image of an interrupted conference list. The suspended conference list may include information on the name of the conference, the location where the conference was held, and the date and time (start time) of the conference. Note that the image shown in FIG. 10 is an example, and the information included in the suspended conference list is not limited to this.

  According to the above-described embodiment, the conference that the user wants to resume (continue) from the suspended conferences is presented based on the utterances of the participants. Thereby, the participant can select the conference that he / she wants to resume (continue) from among the suspended conferences by voice.

<First Modification>
In the example illustrated in FIG. 4, the example in which the minutes of the image g30 are narrowed down to one minutes (the meeting is narrowed down) as in the image g40 by voice input has been described. There may be cases. In the modification, a processing example in such a case will be described.

FIG. 11 is a diagram showing an example of an image displayed when the minutes cannot be narrowed down during the conference in the first modified example of the present embodiment.
The image g30a is an example of an image that is displayed in an example in which the conference cannot be narrowed down by two voice inputs, for example. The image g30a includes a first candidate image g31a and a second candidate image g32a. From this image, the participant has not been able to narrow down the conference to be resumed, and therefore performs voice input “cannot be narrowed down” (image g33a).

The minutes selection unit 109 adds information about the first candidate and the second candidate and re-displays them based on the voice input “cannot be narrowed down”. The image g30b is an image example in which information is added and redisplayed. In the image g30b, “Suzuki” is added as a participant to the first candidate image g31a, and “Issue, Europe” is added as a keyword, as in the image g31b. In the image g30b, “Watanabe” is added as a participant and “US, China” is added as a keyword, as in the image g32b, with respect to the second candidate image g32a.
Note that the number of information to be added may be one or more. Further, the minutes selection unit 109 may display only the added information.

  The user can narrow down the conference to be continued to one by this added information. It should be noted that the user who is a participant may input information desired to be added by voice when displaying the image g30a. For example, the user may make an utterance such as “please display more keywords because it cannot be narrowed down”. The meeting support apparatus 1 may perform processing such as voice recognition on the utterance and display the image g30b.

  According to the first modified example, when the conference cannot be narrowed down by the content uttered by the participant, information on the minutes of the candidate conference is additionally presented. Thereby, the participant can select the conference that he / she wants to resume (continue) from among the suspended conferences by voice.

<Second Modification>
In the conference support apparatus 1 described above, the configuration example illustrated in FIG. 1 has been described. However, a slave unit may be connected to the conference support apparatus.
In this case, for example, a conference proceeding person uses the conference support apparatus, and other participants use the slave units.

  Among the participants, a person who is not able to speak may be participating in the conference. Participants who can speak use a sound collection unit for each participant or use a terminal (smart phone, tablet terminal, personal computer, etc.) equipped with a sound collection unit. Participants with hearing impairments use terminals capable of inputting text. The conference support apparatus recognizes voices of voice signals uttered by the participants, converts the voice signals into text, and displays the text on each terminal. In addition, the conference support apparatus displays text information input by the hearing impaired person on each terminal.

FIG. 12 is a block diagram illustrating a configuration example of the conference support system 2 in the second modification example according to the present embodiment.
As shown in FIG. 12, the conference support system 2 includes a conference support device 1A and slave units 3a, 3b,. In addition, when not specifying one of the subunit | mobile_unit 3a, the subunit | mobile_unit 3b, ..., it is only called the subunit | mobile_unit 3. FIG.
The conference support apparatus 1A and the child device 3 are connected via a wired or wireless network.

  The conference support apparatus 1A includes a sound collection unit 101, an acquisition unit 102A, a speech recognition dictionary DB 103, a speech recognition unit 104, a text conversion unit 105, a dependency analysis unit 106A, a minutes creation unit 107A, a minutes storage unit 108, and minutes. A selection unit 109, an image generation unit 110A, a display unit 111, an operation unit 112, and a communication unit 113 are provided. In addition, the description which abbreviate | omits the function part which has a function similar to the meeting assistance apparatus 1 using the same code | symbol.

  The subunit | mobile_unit 3 is provided with the sound collection part 301, the operation part 302 (text input part), the process part 303, the display part 304, and the communication part 305. The sound collection unit 301, the operation unit 302, the processing unit 303, the display unit 304 (child device display unit), and the communication unit 305 are connected via a bus 306.

  The subunit | mobile_unit 3 is a smart phone, a tablet terminal, a personal computer etc., for example. In addition, the subunit | mobile_unit 3 may be provided with the audio | voice output part, the motion sensor, GPS (Global Positioning System; Global positioning system) etc.

  The sound collection unit 301 is a microphone. The sound collection unit 301 collects a user's sound signal, converts the collected sound signal from an analog signal to a digital signal, and outputs the sound signal converted into the digital signal to the processing unit 303.

  The operation unit 302 detects a user operation and outputs the detected result to the processing unit 303. The operation unit 302 is, for example, a touch panel sensor provided on the display unit 304 or a priority connection or wireless connection keyboard.

  The processing unit 303 generates setting information based on the operation result of the operation unit 302 being operated, and outputs the generated setting information to the communication unit 305. Here, the setting information includes identification information of the participant. The processing unit 303 generates a login instruction based on the operation result of the operation unit 302 being operated, and outputs the generated login instruction to the communication unit 305. Here, the login instruction includes the identification information of the participant and the identification information of the handset 3. The processing unit 303 adds identification information to text information based on the operation result of the operation unit 302 being operated, and outputs the text information to the communication unit 305. The processing unit 303 adds the identification information to the audio signal output from the sound collection unit 301 and outputs it to the communication unit 305. The processing unit 303 acquires the image data output from the communication unit 305 and outputs the acquired image data to the display unit 304. The processing unit 303 establishes communication with the conference support apparatus 1 </ b> A based on the information permitting login output from the communication unit 305. When the processing unit 303 receives a speech restriction instruction (input restriction instruction) from the conference support apparatus 1A, the processing unit 303 restricts text input. In addition, when the processing unit 303 receives a speech restriction instruction from the conference support apparatus 1A, the processing unit 303 may perform restriction on voice input.

  The display unit 304 displays the image data output from the processing unit 303. The display unit 304 is, for example, a liquid crystal display device, an organic EL (electroluminescence) display device, an electronic ink display device, or the like. The image displayed on the display unit 304 will be described later.

  The communication unit 305 transmits the setting information output from the processing unit 303 to the conference support apparatus 1A via the network. The communication unit 305 transmits a login instruction output from the processing unit 303 to the conference support apparatus 1A via the network. The communication unit 305 transmits the text information or the audio signal output from the processing unit 303 to the conference support apparatus 1A via the network. Note that the text information or audio signal to be transmitted includes user identification information and slave unit 3 identification information. The communication unit 305 receives the image data transmitted by the conference support apparatus 1A and outputs the received image data to the processing unit 303. When the communication unit 305 receives information permitting login transmitted from the conference support apparatus 1 </ b> A, the communication unit 305 outputs the received information permitting login to the processing unit 303.

  In addition to the process performed by the acquisition unit 102, the acquisition unit 102A acquires the audio signal transmitted by the slave unit 3 output from the communication unit 113.

  In addition to the processing performed by the dependency analysis unit 106, the dependency analysis unit 106A acquires text information transmitted from the slave unit 3 and output from the communication unit 113.

  In addition to the processing performed by the minutes creation unit 107, the minutes creation unit 107A acquires the setting information transmitted by the handset 3 output from the communication unit 113.

  The image generation unit 110A outputs the generated image data to the communication unit 113. The image data includes a result (for example, a region indicated by reference sign g102 in FIG. 7) obtained by performing processing such as speech recognition on the result of the participant speaking during the conference.

  The communication unit 113 acquires the setting information transmitted by the child device 3, and outputs the acquired setting information to the minutes creation unit 107A. The communication unit 113 receives the login instruction transmitted by the child device 3. The conference support apparatus 1A performs a login procedure for the slave unit 3 in response to the received login instruction. Communication unit 113 outputs the audio signal received from handset 3 to acquisition unit 102A. The communication unit 113 outputs the audio signal received from the child device 3 to the dependency analysis unit 106A. The communication unit 113 transmits the image data output from the image generation unit 110A to the child device 3.

  With this configuration, according to the second modification, the participant can understand the utterance contents of other participants by displaying the display unit 304 of the child device 3. As a result, according to the second modification, it is possible to make it easier for the normal hearing person and the hearing impaired person to participate in the conference together.

  A program for realizing all or part of the functions of the conference support apparatus 1 (or 1A) in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is stored in a computer system. You may perform all or one part of the process which the meeting assistance apparatus 1 (or 1A) performs by reading and performing. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, and various deformation | transformation and substitution are within the range which does not deviate from the summary of this invention. Can be added.

DESCRIPTION OF SYMBOLS 1,1A ... Conference support apparatus, 2 ... Conference support system, 101 ... Sound collection part, 102, 102A ... Acquisition part, 103 ... Speech recognition dictionary DB, 104 ... Speech recognition part, 105 ... Text conversion part, 106, 106A ... Dependency analysis unit, 107, 107A ... Minutes creation unit, 108 ... Minutes storage unit, 109 ... Minutes selection unit, 110, 110A ... Image generation unit, 111 ... Display unit, 112 ... Operation unit, 113 ... Communication unit 3 ... Slave unit, 301 ... Sound collection unit, 302 ... Operation unit, 303 ... Processing unit, 304 ... Display unit, 305 ... Communication unit

Claims (7)

A sound collection unit;
A speech recognition unit that performs speech recognition using a speech recognition dictionary for speech input by the sound collection unit;
A display unit for displaying the voice-recognized text as a minutes;
A storage unit for storing the minutes and storing information indicating that the meeting is suspended with respect to the minutes of the suspended meeting;
In the case of resuming a suspended conference, the selection unit displays the suspended conference candidates, and narrows down the conference candidates to be displayed based on the result of speech recognition of the displayed conference candidates.
A meeting support apparatus.   The conference support according to claim 1, wherein the selection unit narrows down a conference to be continued by at least one of a participant extracted from the conference, a time related to the conference, and a keyword extracted from the minutes. apparatus.   The conference support apparatus according to claim 1, wherein the voice recognition unit uses a voice recognition dictionary used in a continuous conference.   The selection unit performs at least one of presentation by adding the attendees of the conference and adding a keyword extracted from the minutes when narrowing down the conference to be continued for the second time or later. The conference support device according to claim 1, wherein the conference support device is configured as described above. A slave unit connected to the conference support apparatus by communication, wherein the slave unit includes a sound collection unit, a slave unit display unit, and an operation unit,
The voice recognition unit performs voice recognition on the voice signal transmitted by the slave using the voice recognition dictionary,
A voice recognition result of the voice signal picked up by the sound pickup unit and a new voice word picked up by the sound pickup unit of the slave unit is displayed on the display unit and the slave unit display unit; The conference support device according to any one of claims 1 to 4. A meeting support method in a meeting support apparatus having a storage unit for storing meeting minutes,
A sound collection step for obtaining a speech input by the sound collection unit;
A voice recognition step in which a voice recognition unit performs voice recognition using the voice recognition dictionary for the utterance inputted by voice in the sound collecting step;
Minutes creation of the minutes of the text recognized by the voice recognition step in the minutes creation unit, and storing the information indicating that the meeting is interrupted in association with the minutes when the meeting is interrupted Steps,
When the selection unit resumes an interrupted conference, the candidate for the interrupted conference is displayed, and the candidates for the conference to be continued are narrowed down based on the result of voice recognition of the speech utterance. A selection step;
Meeting support method including In the computer of the conference support apparatus having a storage unit for storing the minutes of the conference,
A sound collection step for acquiring speech input speech;
A speech recognition step of performing speech recognition using the speech recognition dictionary for the utterance input by the sound collection step;
Creating a minutes of the text recognized by the speech recognition step, and storing the information indicating that the meeting is interrupted in association with the meeting when the meeting is interrupted;
In the case of resuming an interrupted meeting, a selection step of displaying the interrupted meeting candidates and narrowing down the meeting candidates to be continued based on a result of voice recognition of the displayed speech candidates,
A program that executes

Images