JP2023178102A - Information display control device and information display method - Google Patents

Abstract

To appropriately display reference information from a keyword based on contents of speeches made by a plurality of persons.SOLUTION: A control device 20 comprises: a speech recognition unit 21 which recognizes speeches of a plurality of participants participating in a communication with at least speeches; an identification unit 22 which identifies a main speaker among the participants; an extraction unit 26 which extracts a keyword needing reference information out of the speeches recognized by the speech recognition unit 21; an information acquisition unit 27 which acquires the reference information for the keyword extracted by the extraction unit 26; and a display control unit 28 which displays the reference information acquired by the information acquisition unit 27. The display control unit 28 displays a condition for displaying the reference condition by weighting the keyword based on a speech made by the main speaker.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information display control device and an information display method.

2. Description of the Related Art In a meeting or conversation, a technique for displaying additional information on words and phrases uttered by the other party is known (for example, see Patent Document 1).

JP 2021-007003 Publication

In a meeting or conversation where multiple people speak, a large amount of reference information such as additional information based on the comments of various speakers is displayed. In such a case, viewing the displayed information may cause the user to ignore what is being said in the meeting or conversation.

The present disclosure has been made in view of the above, and an object of the present disclosure is to appropriately display reference information from keywords based on the content of utterances by a plurality of people.

In order to solve the above-mentioned problems and achieve the purpose, an information display control device according to the present disclosure includes at least a speech recognition unit that recognizes the utterances of a plurality of participants participating in speech communication; an identification unit that identifies a main speaker in a participant; an extraction unit that extracts a keyword requiring reference information from the utterance recognized by the utterance recognition unit; and reference information for the keyword extracted by the extraction unit. and a display control unit that displays the reference information acquired by the information acquisition unit, and the display control unit sets conditions for displaying the reference information according to the main speaker. Keywords based on utterances are weighted and displayed.

The information display method according to the present disclosure includes at least a speech recognition step of recognizing utterances of a plurality of participants participating in speech communication, and a specifying step of identifying a main speaker among the plurality of participants. , an extraction step of extracting a keyword requiring reference information from the utterance recognized in the utterance recognition step, an information acquisition step of acquiring the reference information for the keyword extracted in the extraction step, and an information acquisition step of acquiring the reference information for the keyword extracted in the extraction step; a displaying step of displaying the reference information; in the displaying step, conditions for displaying the reference information are displayed by weighting keywords based on utterances of the main speaker.

According to the present disclosure, it is possible to appropriately display reference information based on keywords based on the content of utterances by a plurality of people.

FIG. 1 is a schematic diagram showing an example of how an information display device is used. FIG. 2 is a schematic diagram showing another example of how the information display device is used. FIG. 3 is a block diagram showing a configuration example of an information display device having an information display control device according to the first embodiment. FIG. 4 is a diagram showing an example of keyword information. FIG. 5 is a diagram illustrating the necessity level of keywords. FIG. 6 is a flowchart showing an example of the flow of processing in the information display control device according to the first embodiment. FIG. 7 is a flowchart illustrating an example of the flow of processing in the information display control device according to the second embodiment. FIG. 8 is a flowchart showing an example of the flow of processing in the information display control device according to the third embodiment.

Embodiments of an information display control device and an information display method according to the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

[First embodiment]
<Information display device>
FIG. 1 is a schematic diagram showing an example of how an information display device is used. FIG. 2 is a schematic diagram showing another example of how the information display device is used. FIG. 3 is a block diagram showing a configuration example of an information display device 10 having an information display control device (hereinafter referred to as "control device") 20 according to the first embodiment. The information display device 10 displays reference information of keywords included in utterances of participants in at least utterance communication such as a meeting or conversation.

As shown in FIG. 1, the information display device 10 is used in a conference (hereinafter referred to as a "real conference") or conversation in which a plurality of participants gather in one place. The information display device 10 used in this case may be implemented as one of the functions of an information terminal device such as a smartphone used by the participant, for example. The real conference shown in FIG. 1 includes a presenter M11, another participant M12, another participant M13, another participant M14, and another participant M15. Another participant M15 is using the information display device 10.

As shown in FIG. 2, the information display device 10 is used in an online conference (hereinafter referred to as a "WEB conference") or conversation in which a plurality of participants participate. The information display device 10 used in this case is, for example, an information terminal device such as a notebook personal computer or a smartphone used when participating in an online web conference or conversation via a web conference system (not shown). It may be implemented as one of the functions.

In the following description, a case will be described in which the system is used in either a real conference or a web conference. When there is no particular need to distinguish between a real conference and a web conference, they are referred to as a conference.

The information display device 10 extracts keywords based on the content of utterances by a plurality of people in a conference by weighting the utterances of the main speaker. The information display device 10 includes a microphone 11, a display section 12, a keyword information storage section 18, a communication section 19, and a control device 20, as shown in FIG. The information display device 10 used for the web conference further includes a camera (not shown).

The microphone 11 is a microphone that picks up the voices of participants in the conference who are using the information display device 10 . The microphone 11 outputs the collected audio data to the speech recognition unit 21 of the control device 20 . As shown in FIG. 1, in the case of an information display device 10 used for a real conference, a microphone 11 picks up the voice of a speaker. As shown in FIG. 2, when the information display device 10 is used for a web conference, the microphone 11 picks up the voices of the participants using the information display device 10.

The display unit 12 is a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 12 displays video based on the video signal output from the display control unit 28 of the control device 20.

The display unit 12 displays reference information corresponding to keywords included in the utterances of conference participants. The range in which reference information is displayed on the display unit 12 is referred to as a reference information display range 110. As shown in FIG. 1, in the case of the information display device 10 used for a real conference, the reference information display range 110 may be displayed on the entire surface of the display unit 12. As shown in FIG. 2, in the case of the information display device 10 used for a web conference, the reference information display range 110 is displayed on a part of the display unit 12, for example, at the bottom, top, left side, or right side. Good too. In the case of the information display device 10 used for a web conference, the reference information display range 110 may be displayed as a pop-up on the display unit 12. The reference information display range 110 is, for example, a range in which about three lines or less of characters are displayed.

As shown in FIG. 2, in the case of the information display device 10 used for a web conference, the display unit 12 displays images of participants, screen-shared materials, and the like. The display unit 12 displays, for example, a video 101 of a main speaker (hereinafter referred to as "main speaker") larger than videos 102 and 103 of other participants.

The keyword information storage unit 18 stores reference information for each keyword as keyword information. The keyword information storage unit 18 is, for example, a recording medium such as a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory. The keyword information storage section 18 may be an external recording section wirelessly connected via the communication section 19.

The keyword information storage unit 18 stores, as keyword information, a necessity level and reference information for each keyword. The necessity level is a level indicating a guideline for displaying reference information for each keyword, and includes, for example, a difficulty level and a well-known level.

Keywords are words that require supplementary explanation. Keywords include, for example, words that are not generally known, technical terms, abbreviations, difficult words, and buzzwords. The difficulty level indicates the difficulty of a keyword that is a difficult word. For example, the difficulty level is indicated in five stages using numerical values from 1 to 5. Regarding the difficulty level, the higher the difficulty, that is, the more difficult the word is, the larger the numerical value becomes. The popularity level indicates the popularity of a keyword, such as a phrase, a technical term, or an abbreviation that is not generally known. The well-known level is indicated by a numerical value from 1 to 5, for example, in five stages. As for the well-known level, the higher the well-known word, that is, the more well-known the word, the larger the numerical value becomes.

The reference information is, for example, the meaning of the keyword and a specific explanation. If the keyword is an abbreviation, the reference information may be the full, abbreviated name. Note that the explanation may be, for example, a photograph or a diagram in addition to text. The reference information may be, for example, additional information that is additional information about the keyword.

The reference information may have a number of characters, for example, about 100 characters or less, which can be read by a person in about a few seconds.

FIG. 4 is a diagram showing an example of keyword information. In the keyword information shown in FIG. 4, a necessity level is stored for each keyword, and reference information is searched based on the keyword. The reference information may be stored in advance in addition to information searched using the communication unit 19, which will be described later. For example, for the keyword "○○○", difficulty level "5", well-known level "1", and reference information "○...○" are stored. For example, for the keyword "△△△", difficulty level "3", well-known level "5", and reference information "△...△" are stored.

In the example of FIG. 4, for example, five levels of necessity are associated with various keywords. In this case, regarding the difficulty level, which is an example of the necessity level, a difficulty level of "1" indicates that the difficulty is low; in other words, it is an easy-to-understand keyword. Further, the difficulty level "5" indicates that the keyword is highly difficult to understand, in other words, it is a keyword that cannot be easily understood. Furthermore, regarding the well-known level, which is an example of the necessity level, a well-known level "1" indicates that the well-known level is low. In other words, it is a keyword that is not generally known. Further, a well-known level of "5" indicates that the keyword is highly well-known; in other words, it is a keyword that is generally well-known.

When reference information is searched on the web by the communication unit 19, for example, information from a specific web media such as Wikipedia (registered trademark) for a keyword, the latest news regarding the keyword, etc. are searched. Furthermore, if the keyword is a product name or service name, the website of the provider that provides the product or service is searched. Furthermore, the content displayed at the top of the results searched by a search engine on the web, excluding advertisements, may be used as reference information.

The communication section 19 is a communication unit for performing wireless communication and the like. The communication unit 19 communicates using a communication method using, for example, Wi-Fi (registered trademark) or a mobile phone line, and is connected to an intranet or the Internet. Communication of the communication unit 19 is controlled by a communication control unit 29 of the control device 20 . As shown in FIG. 1, in the case of the information display device 10 used for a real conference, the communication section 19 acquires keyword information from the keyword information storage section 18, which is an external recording section, and searches for reference information. As shown in FIG. 2, in the case of the information display device 10 used for a web conference, the communication unit 19 communicates audio and video of participants with the information display device 10 used by other participants. and searches for reference information. More specifically, the communication unit 19 transmits the participant's voice captured by the microphone 11 and the participant's video captured by the camera to the information display device 10 used by other participants. The communication unit 19 receives audio and video of other participants from the information display device 10 used by the other participants.

In the following description, the input voice is the voice picked up by the microphone 11 in the case of the information display device 10 used for a real conference. Inputted audio includes, in the case of the information display device 10 used for the web conference, the audio collected by the microphone 11 and the information display device used by other participants acquired via the communication unit 19. This is the voice of another participant obtained from 10.

<Information display control device>
The control device 20 controls each part of the information display device 10. The control device 20 is, for example, an arithmetic processing device (control device) configured with a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc., and has a storage device such as a RAM or a ROM (Read Only Memory). The control device 20 loads a stored program into memory and executes instructions included in the program. The control device 20 includes an internal memory such as the above-mentioned RAM, and the internal memory is used for temporary storage of data in the control device 20. The control device 20 includes a speech recognition section 21, an identification section 22, an extraction section 26, an information acquisition section 27, a display control section 28, and a communication control section, as components such as functional blocks realized by executing a program. 29.

The utterance recognition unit 21 recognizes at least the utterances of a plurality of participants participating in communication through speech. The speech recognition unit 21 analyzes the input speech and recognizes the content of the speech contained in the speech. The utterance recognition unit 21 performs acoustic model analysis for each phoneme or word on input speech, and recognizes the content of the utterance by comparing the acoustic model with a phoneme model and a language model.

The specifying unit 22 specifies the main speaker among the plurality of participants. The specifying unit 22 may specify, for example, a main participant such as a facilitator, a presenter, a leader, or a conference organizer in a web conference or a real conference as the main speaker.

The identifying unit 22 identifies, from the input audio, the person who first spoke in the web conference or real conference, or the person who first spoke for a predetermined period of time, such as 10 seconds or more, as the main speaker. It's okay. For example, in the case of a real conference, the identification unit 22 may identify the characteristics of the voice of the person who spoke first in the real conference from the input voice by voiceprint analysis, and identify the person as the main speaker.

For example, in the case of a web conference, the identifying unit 22 may identify the person who convened the web conference as the main speaker from the web conference holding information acquired from the web conference system. In the case of a real conference, the identification unit 22 may acquire holding information of the real conference from participants' email information or schedule information, and identify the main speaker.

In the case of a web conference, the identifying unit 22 may identify the person who is sharing the screen, that is, the display of materials for speaking, as the main speaker from the information acquired from the web conference system.

In the case of a web conference, the identification unit 22 may identify, as the main speaker, a person who has performed a predetermined operation such as a speak button or a raise hand button in order to speak from information acquired from the web conference system.

In the case of a web conference, the identification unit 22 may identify the other participant whose image is displayed the largest, such as the image 101 shown in FIG. 2, as the main speaker.

In the case of a real conference, the identification unit 22 identifies people who are standing up, people who have raised their hands, people who are standing in front of a whiteboard or a projection screen projected by a projector, etc. from images of participants captured by a camera. It may also be specified as the main speaker.

The extraction unit 26 extracts keywords requiring reference information from the utterance recognized by the utterance recognition unit 21. The extraction unit 26 extracts conditions for extracting keywords by weighting the utterances of the main speaker. More specifically, the extraction unit 26 extracts keywords by giving priority to the utterances of the main speaker over the utterances of other speakers other than the main speaker, in other words, giving higher weight to the utterances of the main speaker. do.

Compared to other speakers, the main speaker often makes important statements that are in line with the agenda of the meeting. Therefore, by extracting keywords by giving priority to the utterances of the main speaker over the utterances of other speakers other than the main speaker, keywords important for understanding the content of the meeting are preferentially extracted.

The extraction unit 26 determines whether the extracted keyword is uttered by the main speaker depending on whether the keyword extracted by the extraction unit 26 matches the voice characteristics of the main speaker identified by the identification unit 22. All you have to do is decide whether or not.

In the present embodiment, the extraction unit 26 may extract keywords that require reference information depending on the level of necessity of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21. More specifically, when the necessity level of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21 is equal to or higher than a predetermined threshold, the extraction unit 26 extracts the keyword as a keyword requiring reference information. If the keyword is based on the utterance of the main speaker, the extraction unit 26 lowers the necessity level threshold for speakers other than the main speaker.

FIG. 5 is a diagram illustrating a difficulty level and a well-known level, which are examples of the necessity level and necessity level of reference information in a keyword. If the necessity level of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21 is equal to or higher than the threshold t1, the extraction unit 26 extracts the keyword as a keyword requiring reference information. In the case of a keyword based on the utterance of the main speaker, the extraction unit 26 extracts the keyword as a keyword requiring reference information if the necessity level of reference information for the extracted keyword is equal to or higher than a threshold value t2 (t2<t1). For example, when the necessity level is classified into five stages as shown in FIG. A keyword is extracted as a keyword requiring information, and if the necessity level is, for example, 4 or higher for a keyword based on an utterance by a person other than the main speaker, the keyword is extracted as a keyword requiring reference information.

In the present embodiment, when the difficulty level as the necessity level of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21 is equal to or higher than a predetermined threshold, the extraction unit 26 extracts the keyword as a keyword requiring reference information. Good too. For example, when the difficulty level is classified into five levels as shown in FIG. The keyword is extracted as a keyword requiring information, and if the difficulty level is, for example, 4 or higher for a keyword based on an utterance by a person other than the main speaker, the keyword is extracted as a keyword requiring reference information. In other words, keywords based on the utterances of the main speaker are keywords that require reference information, including keywords with a low difficulty level, that is, keywords that are less difficult to understand.

In the present embodiment, when the well-known level as the necessity level of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21 is less than a predetermined threshold, the extraction unit 26 extracts the keyword as a keyword requiring reference information. Good too. For example, when the well-known level is classified into five levels as shown in FIG. 5, the extraction unit 26 extracts the keyword as requiring reference information if the well-known level is less than 3, for example. The extraction unit 26 extracts a keyword based on an utterance by a person other than the main speaker as a keyword requiring reference information when the well-known level is, for example, 3 or less, and extracts the keyword based on an utterance by a person other than the main speaker as a keyword that requires reference information. If the well-known level is, for example, 2 or less, the keyword is extracted as requiring reference information. In other words, keywords based on the utterances of the main speaker are keywords that require reference information, including keywords with a high level of well-known knowledge, that is, keywords that are more well-known.

In FIG. 5, the difficulty level being greater than or equal to the threshold t1 corresponds to the necessity level being greater than or equal to the threshold t1, and the difficulty level being greater than or equal to the threshold t2 means that the necessity level is greater than or equal to the threshold t2. respond to something. Furthermore, the fact that the well-known level is below the threshold t1 corresponds to the necessity level being above the threshold t1, and the well-known level below the threshold t2 means that the necessity level is above the threshold t2. corresponds to

The information acquisition unit 27 acquires reference information for the keyword extracted by the extraction unit 26, that is, the keyword for which the extraction unit 26 determines that reference information is required. In this embodiment, the information acquisition unit 27 acquires reference information for keywords based on information on the web.

The display control unit 28 controls the display of images on the display unit 12. The display control unit 28 outputs a video signal of a video to the display unit 12.

The display control unit 28 controls the display of the reference information acquired by the information acquisition unit 27 on the display unit 12. The display control unit 28 outputs a video signal for displaying the reference information corresponding to the keyword extracted by the extraction unit 26 in the reference information display range 110 of the display unit 12. The display control unit 28 displays the reference information for a predetermined period of time, for example, about 5 seconds to 10 seconds. When displaying the next reference information while displaying the reference information, the display control unit 28 may change the display to the next reference information even before the predetermined period has elapsed. When displaying the next reference information while displaying the reference information, the display control unit 28 may display reference information with a high necessity level, for example.

When the information display device 10 is used for a web conference, the display control unit 28 controls the display of images of the web conference. The display control unit 28 outputs a video signal of the web conference video including the video captured by the camera.

The communication control unit 29 controls communication by the communication unit 19. As shown in FIG. 1, in the case of the information display device 10 used in a real conference, the communication control unit 29 receives keyword information from the keyword information storage unit 18, which is an external recording unit, via the communication unit 19. , controls to search for reference information. As shown in FIG. 2, when the information display device 10 is used for a web conference, the communication control unit 29 communicates with the information display device 10 used by other participants via the communication unit 19. It communicates the audio and video of the participants with each other and controls the search for reference information. More specifically, the communication control unit 29 transmits, via the communication unit 19, the participant's voice captured by the microphone 11 and the participant's video captured by the camera to an information display device used by other participants. 10. The communication control unit 29 controls the communication unit 19 to receive audio and video of other participants from the information display device 10 used by the other participants.

<Information processing in the information display control device>
Next, the flow of processing in the control device 20 will be explained using FIG. 6. The process shown in FIG. 6 is started by starting an application for displaying reference information in the case of the information display device 10 used for a real conference as shown in FIG. As shown in FIG. 2, in the case of the information display device 10 used for a web conference, the process starts by activating an application for holding a web conference or by starting a web conference. In this case, the application for conducting the web conference may include a function for displaying reference information, and the function for displaying reference information may be turned on or off by user operation. When the information display device 10 is activated, the processing of the flowchart shown in FIG. 6 is started.

The control device 20 identifies the main speaker (step S101). More specifically, the control device 20 uses the specifying unit 22 to specify, for example, a main participant such as a facilitator, a presenter, a leader, or a conference organizer as the main speaker from the input voice. Note that the method for identifying the main speaker is not limited to this. The control device 20 proceeds to step S102.

The control device 20 starts speech recognition (step S102). More specifically, the control device 20 uses the speech recognition unit 21 to analyze the input speech and recognize the speech content included in the speech. The control device 20 proceeds to step S103.

The control device 20 determines whether a keyword has been extracted (step S103). More specifically, the control device 20 uses the extraction unit 26 to extract keywords requiring reference information from the utterance recognized by the utterance recognition unit 21. If the necessity level of reference information for the keyword extracted from the utterance recognized by the utterance recognition unit 21 is equal to or higher than a predetermined threshold, the control device 20 extracts the keyword as a keyword requiring reference information by the extraction unit 26 . If the control device 20 determines that the keyword has been extracted (Yes in step S103), the process proceeds to step S104. If the control device 20 does not determine that the keyword has been extracted (No in step S103), the process proceeds to step S108.

When determining that the keyword has been extracted (Yes in step S103), the control device 20 determines whether the extracted keyword is uttered by the main speaker (step S104). More specifically, the control device 20 causes the extraction unit 26 to determine whether the extracted keyword is the main speaker based on whether or not the extracted keyword matches the voice characteristics of the main speaker specified by the identification unit 22. It is determined whether the utterance is by the speaker. When the control device 20 determines that the extracted keyword is uttered by the main speaker (Yes in step S104), the process proceeds to step S106. If the control device 20 does not determine that the extracted keyword is uttered by the main speaker (No in step S104), the process proceeds to step S105.

If it is not determined that the extracted keyword is uttered by the main speaker (No in step S104), the control device 20 determines whether the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold value t1. (Step S105). More specifically, the control device 20 uses the information acquisition unit 27 to acquire the necessity level of reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . In this manner, when the keyword is based on the utterance of the main speaker, the control device 20 uses the extraction unit 26 to lower the threshold value for speakers other than the main speaker and extract the keyword as requiring reference information. do. When the control device 20 determines that the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold value t1 (Yes in step S105), the process proceeds to step S107. If the control device 20 does not determine that the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold value t1 (No in step S105), the process proceeds to step S108.

When determining that the extracted keyword is uttered by the main speaker (Yes in step S104), the control device 20 determines whether the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold t2. (Step S106). More specifically, the control device 20 uses the information acquisition unit 27 to acquire the necessity level of reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . When the control device 20 determines that the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold value t2 (Yes in step S106), the process proceeds to step S107. If the control device 20 does not determine that the necessity level of the reference information of the extracted keyword is equal to or higher than the threshold t2 (No in step S106), the process proceeds to step S108.

The control device 20 acquires the reference information and displays it for a predetermined period of time (step S107). More specifically, the control device 20 uses the information acquisition unit 27 to acquire reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . The control device 20 causes the display control unit 28 to output a video signal of reference information corresponding to the keyword extracted by the extraction unit 26 for a predetermined period of time. The control device 20 proceeds to step S108.

The control device 20 determines whether to end the process (step S108). For example, the information display device 10 may be powered off or an operation unit (not shown) may be operated to terminate the application that triggered the start of the flowchart shown in FIG. 6. As a result, it is determined that the process is to be terminated. When the control device 20 determines to end the process (Yes in step S108), it ends the process. If the control device 20 does not determine to end the process (No in step S108), it executes the process in step S103 again.

A case will be described in which, in steps S105 and S106, the necessity level is set to be whether the difficulty level is equal to or higher than a predetermined threshold. If difficulty levels are associated with various keywords in 5 levels, the larger the numerical value indicating the difficulty level, the more difficult it is, in step S105, the control device 20 controls the difficulty level of the reference information of the extracted keyword. It is determined whether the sex level is 4 or higher. In step S106, the control device 20 determines whether the difficulty level of the extracted keyword reference information is 3 or higher.

A case will be described in which, in steps S105 and S106, whether or not the well-known level is less than a predetermined threshold is set as the necessity level. If various keywords are associated with a well-known level in five stages, the larger the value indicating the well-known level, the more well-known the keyword is, in step S105, the control device 20 controls the public knowledge of the reference information of the extracted keyword. It is determined whether the sex level is 2 or less. In step S106, the control device 20 determines whether the well-known level of the reference information of the extracted keyword is 3 or less.

<Effect>
As described above, in this embodiment, the conditions for extracting keywords are extracted by weighting the utterances of the main speaker. According to this embodiment, reference information can be appropriately displayed by weighting the utterances of the main speaker based on keywords based on the content of utterances by a plurality of people. In the present embodiment, by narrowing down the reference information to be displayed, it is possible to reduce the possibility that participants will take time to check the displayed reference information and will not be able to listen to what is being said in the meeting.

In this embodiment, when the necessity level of reference information for a keyword extracted from a recognized utterance is equal to or higher than a predetermined threshold, the keyword is extracted as a keyword requiring reference information. In this embodiment, if the keyword is based on the utterance of the main speaker, the keyword is extracted by lowering the threshold value for speakers other than the main speaker. In this embodiment, keyword reference information based on the utterance of the main speaker can be displayed with priority over keyword reference information based on the utterances of other speakers. In this embodiment, reference information can be appropriately displayed with priority given to the main speaker.

In this embodiment, when the difficulty level as the necessity level of reference information for a keyword extracted from a recognized utterance is equal to or higher than a predetermined threshold, the keyword is extracted as a keyword requiring reference information. This embodiment can appropriately display reference information for keywords with a high level of difficulty.

In this embodiment, when the well-known level as the necessity level of reference information for a keyword extracted from a recognized utterance is less than or equal to a predetermined threshold, the keyword is extracted as a keyword requiring reference information. The present embodiment can appropriately display reference information for keywords with a low level of public awareness.

In this embodiment, the extraction unit 26 extracts keywords requiring reference information using different weighting conditions for keywords based on the utterances of the main speaker and keywords based on the utterances of speakers other than the main speaker. It was configured to extract. In this embodiment, instead of the extraction section 26, the display control section 28 may have an equivalent function.

In this case, the display control unit 28 weights the utterances of the main speaker to display the keyword display conditions. For example, when the necessity level of the keyword based on the utterance of the main speaker is equal to or higher than the threshold value t2, the display control unit 28 displays reference information, and displays the necessity level of the keyword based on the utterance of other speakers other than the main speaker. When the gender level is equal to or higher than the threshold t1, a process of displaying reference information is performed.

In this case, in step S107 of FIG. 6, the extraction unit 26 acquires reference information for keywords whose necessity level is t2 or higher. In addition, in step S107, the display control unit 28 displays reference information for keywords whose necessity level is t2 or higher when step S104 is Yes, and displays reference information for keywords whose necessity level is t1 or higher when step S104 is No. Display reference information.

[Second embodiment]
The information display device 10 according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart illustrating an example of the flow of processing in the information display control device according to the second embodiment. The basic configuration of the information display device 10 is the same as that of the information display device 10 of the first embodiment. In the following description, components similar to those of the information display device 10 are denoted by the same or corresponding symbols, and detailed description thereof will be omitted. In the present embodiment, the information display device 10 displays reference information in a conference by weighting the utterances of the main speaker based on the content of utterances by a plurality of people and changing the display mode. In the present embodiment, the information display device 10 displays reference information for a keyword based on the utterance of the main speaker for a longer time than reference information for a keyword based on the utterances of other speakers. In this embodiment, the processing in the display control unit 28 is different from the first embodiment.

In this embodiment, the keyword information storage unit 18 only needs to store reference information for each keyword as keyword information. In this embodiment, the keyword information does not need to include the necessity level.

The display control unit 28 displays the conditions for displaying the reference information by weighting the keywords based on the utterances of the main speaker. More specifically, the display control unit 28 gives priority to the utterances of the main speaker over the utterances of other speakers other than the main speaker, in other words, gives higher weight to the utterances of the main speaker and displays keywords. Display reference information.

In this embodiment, the display control unit 28 displays the reference information for the keyword based on the utterance of the main speaker in a different display manner than when the reference information is for the keyword based on the utterances of other speakers. . When the display control unit 28 is the reference information for the keyword based on the utterance of the main speaker, the display control unit 28 makes the display more conspicuous or easier to confirm than when the reference information is for the keyword based on the utterances of other speakers. Change the display mode to display.

Changing the display mode means, for example, changing the display time or changing the amount of information to be displayed. Changing the display mode may include, for example, changing the size or color of the displayed characters, changing the size of the reference information display range 110, or changing the display position.

In the present embodiment, the display control unit 28 determines the time to display the reference information, when the reference information is for a keyword based on the utterance of the main speaker, and when the reference information is for the keyword based on the utterance of another speaker. to be displayed for a long time. The display control unit 28 displays the reference information by setting the display time of the keyword reference information based on the utterance of the main speaker to 5 seconds, for example, and setting the display time of the keyword reference information based on the utterances of other speakers to 3 seconds, for example. do.

The main speaker often makes important statements, such as statements that are related to the topic of the meeting, more often than other speakers. Therefore, by displaying reference information for keywords based on the utterances of the main speaker for a longer period of time than reference information for keywords based on the utterances of other speakers other than the main speaker, we can display reference information for keywords based on the utterances of other speakers other than the main speaker. Make it easier to check.

Next, the flow of processing in the control device 20 will be described using FIG. 7. The processing from step S111 to step S114 and step S117 shown in FIG. 7 is the same as the processing from step S101 to step S104 and step S108 of the flowchart shown in FIG.

When determining that the extracted keyword is uttered by the main speaker (Yes in step S114), the control device 20 acquires reference information and displays it for a first period tm1 (tm1<tm2) (step S115). . More specifically, the control device 20 uses the information acquisition unit 27 to acquire reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . The control device 20 causes the display control unit 28 to output a video signal of reference information corresponding to the keyword extracted by the extraction unit 26 for a first period tm1. The control device 20 proceeds to step S117.

If it is not determined that the extracted keyword is uttered by the main speaker (No in step S114), the control device 20 acquires the reference information and displays it for the second period tm2 (step S116). More specifically, the control device 20 uses the information acquisition unit 27 to acquire reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . The control device 20 causes the display control unit 28 to output a video signal of reference information corresponding to the keyword extracted by the extraction unit 26 for a second period tm2. The control device 20 proceeds to step S117.

<Effect>
As described above, the conditions for displaying reference information are displayed by weighting the keywords based on the utterances of the main speaker. According to this embodiment, reference information can be appropriately displayed based on keywords based on the content of utterances by a plurality of people.

In this embodiment, when the reference information is for a keyword based on the utterance of the main speaker, the display format is changed from that when the reference information is for the keyword based on the utterances of other speakers. In this embodiment, the keyword reference information based on the utterance of the main speaker can be displayed in a display format that is easier to confirm, for example, than the keyword reference information based on the utterances of other speakers.

In this embodiment, reference information is displayed for a longer time when the reference information is for a keyword based on the utterance of the main speaker than when the reference information is for a keyword based on the utterances of other speakers. can be done.

[Third embodiment]
FIG. 8 is a flowchart showing an example of the flow of processing in the information display control device according to the third embodiment. The information display device 10 has the same basic configuration as the information display device 10 of the second embodiment. In the following description, components similar to those of the information display device 10 are denoted by the same or corresponding symbols, and detailed description thereof will be omitted. In the present embodiment, the information display device 10 displays a larger amount of information when the reference information is for a keyword based on the utterance of the main speaker than when the reference information is for the keyword based on the utterances of other speakers. do. In this embodiment, the processing in the display control unit 28 is different from the second embodiment.

In the present embodiment, the keyword information storage unit 18 may store, for example, reference information for the maximum information amount lv1 and reference information for the maximum information amount lv2 as the reference information for the keyword information. In this case, for example, the reference information of the maximum information amount lv1 is about three lines of reference information, and the reference information of the maximum information amount lv2 is about one line of reference information.

In the present embodiment, the display control unit 28 sets the amount of information to display reference information (hereinafter referred to as "maximum amount of information") to other utterances when the reference information is for a keyword based on the utterance of the main speaker. If the information is a reference to a keyword based on a person's utterance, it will be displayed more often. The display control unit 28 sets the amount of information for displaying the keyword reference information based on the utterance of the main speaker to a maximum of three lines (maximum amount of information 3 lines), and displays the keyword reference information based on the utterances of other speakers. The amount of information to be displayed is, for example, one line at most (maximum amount of information one line).

For example, the display control unit 28 may display the maximum amount of information from the beginning of the acquired reference information. For example, the display control unit 28 may display the reference information with the maximum amount of information lv1 or the reference information with the maximum amount of information lv2, depending on the maximum amount of information to be displayed among the acquired reference information.

The main speaker often makes important statements, such as statements that are related to the topic of the meeting, more often than other speakers. Therefore, by displaying reference information for keywords based on the main speaker's utterances in a larger amount than reference information for keywords based on the utterances of other speakers other than the main speaker, we can display information that is important for understanding the content of the meeting. It is possible to deepen the understanding of keywords.

Next, the flow of processing in the control device 20 will be described using FIG. 8. The processing from step S121 to step S124 and step S127 shown in FIG. 8 is the same as the processing from step S101 to step S104 and step S108 of the flowchart shown in FIG. 6.

When determining that the extracted keyword is uttered by the main speaker (Yes in step S124), the control device 20 acquires reference information and displays the maximum information amount lv1 (lv1<lv2) (step S125). ). More specifically, the control device 20 uses the information acquisition unit 27 to acquire reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . The control device 20 causes the display control unit 28 to output a video signal of reference information with a maximum information amount lv1 corresponding to the keyword extracted by the extraction unit 26. The control device 20 proceeds to step S127.

If it is not determined that the extracted keyword is uttered by the main speaker (No in step S124), the control device 20 acquires the reference information and sets the maximum information amount lv2 (step S126). More specifically, the control device 20 uses the information acquisition unit 27 to acquire reference information for the keyword extracted by the extraction unit 26 from the keyword information in the keyword information storage unit 18 . The control device 20 causes the display control unit 28 to output a video signal of reference information with a maximum information amount lv2 corresponding to the keyword extracted by the extraction unit 26. The control device 20 proceeds to step S127.

<Effect>
As described above, in this embodiment, the amount of information to display the reference information is divided into two types: when the reference information is for a keyword based on the utterance of the main speaker, and when it is the reference information for the keyword based on the utterances of other speakers. can be displayed in large numbers.

The information display device according to the present disclosure may be implemented in various different forms other than the embodiments described above.

Each component of the illustrated information display device is functionally conceptual, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to what is shown in the diagram, and all or part of it may be functionally or physically distributed or integrated into arbitrary units depending on the processing load and usage status of each device. It's okay.

The configuration of the information display device is realized by, for example, a program loaded into a memory as software. The above embodiments have been described as functional blocks realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms using only hardware, only software, or a combination thereof.

The above-mentioned components include those that can be easily imagined by those skilled in the art and that are substantially the same. Furthermore, the above configurations can be combined as appropriate. Furthermore, various omissions, substitutions, or changes in the configuration are possible without departing from the gist of the present invention.

[Modified example]
In the above description, the display of reference information may be different between the information display device 10 used by the main speaker and the information display device 10 used by other speakers. For example, the information display device 10 used by the main speaker may not display reference information.

10 Information display device 11 Microphone 12 Display section 18 Keyword information storage section 19 Communication section 20 Control device (information display control device)
21 Speech recognition unit 22 Specification unit 26 Extraction unit 27 Information acquisition unit 28 Display control unit 29 Communication control unit

Claims (5)

a speech recognition unit that recognizes utterances of at least a plurality of participants participating in communication by speech;
a specifying unit that specifies a main speaker among the plurality of participants;
an extraction unit that extracts a keyword requiring reference information from the utterance recognized by the utterance recognition unit;
an information acquisition unit that acquires the reference information for the keyword extracted by the extraction unit;
a display control unit that displays the reference information acquired by the information acquisition unit;
Equipped with
The display control unit displays conditions for displaying the reference information by weighting keywords based on utterances of the main speaker.
Information display control device. The display control unit displays the reference information for a keyword based on the utterance of the main speaker in a different display manner than when the reference information is for the keyword based on the utterance of another speaker. ,
The information display control device according to claim 1. The display control unit determines the time to display the reference information, if the reference information is for a keyword based on the utterance of the main speaker, the reference information is for a keyword based on the utterance of another speaker. Display for a long time,
The information display control device according to claim 2. The display control unit sets the amount of information to display the reference information to, if the reference information is for a keyword based on the utterance of the main speaker, the reference information is for a keyword based on the utterance of another speaker. display more in a given case,
The information display control device according to claim 2. a speech recognition step for recognizing utterances of a plurality of participants participating in at least speech communication;
a step of identifying a main speaker among the plurality of participants;
an extraction step of extracting keywords requiring reference information from the utterances recognized in the utterance recognition step;
an information acquisition step of acquiring the reference information for the keyword extracted in the extraction step;
a display step of displaying the reference information acquired in the information acquisition step;
including;
In the displaying step, conditions for displaying the reference information are displayed by weighting keywords based on the utterances of the main speaker.
An information display method executed by an information display control device.

Images