JP2014013494A - Display control device, display system, display device, terminal device, display control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information for smoothly facilitating a conference to a speech-speaking person.SOLUTION: A display control device comprises: an information generation part that generates information to be associated with a voice uttered by the speech-speaking person from the voice uttered by the speech-speaking person; and a result output process part that outputs first search information in accordance with a first search result searched on the basis of first information to be associated with the voice uttered by a plurality of the speech-speaking persons and outputs second search information in accordance with a second search result searched on the basis of second information to be associated with the voice uttered by at least any of the plurality of the speech-speaking persons.

Description

  The present invention relates to a display control device, a display system, a display device, a terminal device, a display control method, and a program.

In recent years, in a conference where a plurality of speakers (participants) participate, a conference is held by displaying the information on a provided display device so that a plurality of speakers can share information that is the topic of the conference. It may be done. In a conference held as described above, the information displayed on the display device is sequentially switched so that one of a plurality of speakers matches the topic of the conference that changes according to the progress of the conference. Is displayed. Thus, if the information to be shared is sequentially switched and simply displayed on the display device, each speaker is displayed even though each speaker can recognize that the displayed information has been switched. It was difficult to share the information up to the editing intent that the speaker switched.
By the way, there is a work status management device that records history information of an editing work in which information is switched in an editing work for editing information (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 7-098734

  However, the work status management apparatus according to Patent Document 1 does not have a means for notifying the intention of editing in which editing work for switching information is performed. Even if the work status management device described above is applied to a conference, it is not possible to share the intention of editing by switching the information to be displayed sequentially, and information for smoothly proceeding with the conference can be provided to the speaker. There was a problem that I could not.

  The present invention has been made to solve the above-described problem, and its purpose is to provide a display control device, a display system, a display device, a terminal device, and the like, which can provide information for making a conference proceed smoothly to a speaker. It is to provide a display control method and program.

  According to an embodiment of the present invention, an information generation unit that generates information associated with the voice uttered by the speaker from the voice uttered by the speaker, and the first information associated with the voice uttered by the plurality of the speakers First search information corresponding to the first search result searched based on one information is output, and at least second information associated with a voice uttered by any one of the plurality of speakers And a result output processing unit that outputs second search information corresponding to the second search result searched based on the display control device.

  In addition, an embodiment of the present invention includes the display control device according to the above invention, wherein the display control device displays the first search information on a display unit and displays the second search information on an external device. This is a display system characterized by that.

  Moreover, one embodiment of the present invention is a display device comprising the display control device according to the above-described invention and a display unit that displays the first search information output from the display control device. .

  Moreover, one embodiment of the present invention includes the display control device according to the above invention and a terminal display unit that displays the first search information, and the result output processing unit stores the first search information in the first search information. The terminal device is characterized by being displayed on a terminal display unit.

  Further, according to an embodiment of the present invention, a process of generating information associated with the voice uttered by the speaker from the voice uttered by the speaker, and a process associated with the voice uttered by the plurality of speakers. A process of outputting first search information corresponding to a first search result searched based on one information, and a second associated with a voice uttered by at least one of the plurality of speakers And a process of outputting second search information corresponding to the second search result searched based on the information.

  In one embodiment of the present invention, a computer provided in the display control device generates information associated with the voice uttered by the speaker from the voice uttered by the speaker, and a plurality of the speakers Outputting the first search information according to the first search result searched based on the first information associated with the uttered voice; and at least one of the plurality of speakers This is a program for executing a step of further outputting the second search information corresponding to the second search result searched based on the second information associated with the voice.

  According to an embodiment of the present invention, a speaker included in a computer provided in a display device that includes a display control device and a display unit that displays information output from the display control device is obtained from a voice uttered by a speaker. Generating information associated with the uttered voice, and outputting first search information corresponding to the first search result retrieved based on the first information associated with the voice uttered by the plurality of speakers And further outputting second search information corresponding to the second search result searched based on the second information associated with the voice uttered by at least one of the plurality of speakers And a step of displaying any one of the first search information and the second search information on the display unit.

  Further, according to one embodiment of the present invention, a step of generating information associated with a voice uttered by a speaker from a voice uttered by a speaker in a computer included in a terminal device including a terminal display unit; Displaying the first search information corresponding to the first search result searched based on the first information associated with the voices uttered by the plurality of speakers on the terminal display unit, and at least the plurality of utterances Outputting the second search information corresponding to the second search result searched based on the second information associated with the voice uttered by any one of the speakers. It is.

  According to the present invention, information for smoothly proceeding with the conference can be provided to the speaker.

It is a lineblock diagram showing an outline of a display system provided with a display control device concerning a 1st embodiment of the present invention. It is a block diagram which shows an example of the utilization form of the display system in this embodiment. It is a figure which shows an example of the display in the display part 200 and the display 300 in this embodiment. It is a block diagram which shows the structure of the display system of this embodiment. It is a figure which shows an example of the display in two displays in 2nd Embodiment. It is a figure which shows the case where the information according to the utterance content which the specific speaker in 7th Embodiment uttered is displayed. It is a block diagram which shows the structure of the display system in 7th Embodiment. It is a figure which shows the outline | summary of a display system provided with the display control apparatus in 8th Embodiment. It is a block diagram which shows the structure of the display system in 8th Embodiment. It is a figure which shows the outline | summary of the display system which provided the display part in 9th Embodiment. It is a block diagram which shows an example of a structure of a conference system provided with the search result output device which concerns on the 10th Embodiment of this invention. It is a block diagram which shows an example of the utilization form of the conference system in this embodiment. It is a block diagram which shows an example of a structure of the conference system in this embodiment. It is a flowchart which shows an example of operation | movement of the search means control apparatus in this embodiment. It is a flowchart which shows an example of operation | movement of the selection part in this embodiment. It is a block diagram which shows an example of a structure of the search result output device which concerns on the 11th Embodiment of this invention. It is a block diagram which shows an example of the conference system output to the display comprised in the hemisphere in this embodiment. SS It is a 1st figure which shows an example of a structure of the search device by 12th Embodiment. It is a block diagram of the search device in this embodiment. It is a flowchart which shows an example of a process of the topic provision mode in this embodiment. It is a flowchart which shows an example of the process of conversation tracking mode in this embodiment. It is a flowchart which shows the 1st modification of the topic provision mode in this embodiment. It is a flowchart which shows the 2nd modification of the topic provision mode in this embodiment. It is a block diagram of the search device in 13th Embodiment. It is a flowchart which shows an example of the process of conversation tracking mode in this embodiment. It is a 2nd figure which shows an example of a structure of the search device by this embodiment. It is a block diagram of the search device in 14th Embodiment. It is a flowchart which shows an example of the process of conversation tracking mode in this embodiment. It is a flowchart which shows an example of the process of conversation tracking mode in 15th Embodiment. It is a block diagram of the search device in a 16th embodiment. RR It is a conceptual block diagram of the image display apparatus which is 17th Embodiment. It is explanatory drawing of the operation mode determination by a mode operation control part. It is explanatory drawing of the display of the operation mode on a display part. It is a conceptual block diagram of the image display apparatus in 18th Embodiment. It is a conceptual block diagram of the image display apparatus in 19th Embodiment. It is a conceptual block diagram of the image display apparatus in 20th Embodiment. It is a conceptual block diagram of the image display apparatus in 21st Embodiment.

[First embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram illustrating an overview of a display system including a display control device according to the present embodiment.
The display system 1000 shown in FIG. 1 includes a display control device 1100.
The display control device 1100 is connected to a sound collection unit 101, a display unit 200, and a display 300 (terminal device). In addition, the display control device 1100 is provided on the desk 3, for example, together with the sound collection unit 101 and the display unit 200. The display control device 1100 (terminal control unit) and the display unit 200 (terminal display unit) may be integrated to form the terminal device 1100S.

The display unit 200 displays an image corresponding to the input image information. The display unit 200 of the present embodiment is provided, for example, on the top plate surface of the desk 3 so that the displayed image can be viewed by a person (speaker 2) around the desk 3.
The display 300 displays an image corresponding to the input image information. The display 300 of the present embodiment is provided, for example, in the vicinity of the desk 3 so that the displayed image can be viewed by a person (speaker 2) around the desk 3. The display 300 includes a display surface having a normal line oriented in a substantially horizontal direction. For example, the normal line of the display surface is directed toward the desk 3.

  The sound collection unit 101 includes, for example, a microphone that collects sound, and collects sound emitted by a plurality of speakers 2. The sound collection unit 101 of the present embodiment includes, for example, four sound collection units 101, a sound collection unit 101A, a sound collection unit 101B, a sound collection unit 101C, and a sound collection unit 101D. The sound emitted by is picked up. Here, the sound collection unit 101A collects sound emitted in the direction DR1, for example. Similarly, the sound collection unit 101B emits, for example, a sound emitted in the direction DR2, the sound collection unit 101C, for example, a sound emitted in the direction DR3, and the sound collection unit 101D, for example, emits a sound in the direction DR4. Pick up each voice. For example, as shown in FIG. 2, the sound collection unit 101 is a speaker 2A, 2B, 2C seated at a predetermined position of the desk 3, for example, a position corresponding to the sound collection units 101A, 101B, 101C, 101D. It has a sensitivity characteristic (directivity) that collects voices uttered by two 2D speakers 2.

The display control apparatus 1100 is connected to the search unit 11 via the network 15.
The search means 11 searches for information related to the input character information. The search means 11 of this embodiment is a search engine that searches for information related to character information input via a network 15 such as the Internet or a LAN. In addition, the search unit 11 outputs a search result of information related to the input character information to the display control apparatus 1100 via the network 15 as a search result.

FIG. 2 is a configuration diagram illustrating an example of a usage form of the display system.
In FIG. 2, the desk 3 is provided so as to be surrounded by the speakers 2A, 2B, 2C, and 2D. The sound collection unit 101A provided on the desk 3 collects the sound mainly emitted by the speaker 2A. Similarly, the sound collecting unit 101B collects the sounds emitted mainly by the speaker 2B, the sound collecting unit 101C mainly by the speaker 2C, and the sound collecting unit 101D mainly collects the sounds emitted by the speaker 2D. To do. Here, for example, the display system 1000 detects which speaker 2 has collected the sound by detecting which sound collecting unit 101 out of the sound collecting units 101A, 101B, 101C, and 101D. Can be determined.
In this case, the display unit 200 displays an image for the speakers 2A, 2B, 2C, and 2D so that the displayed image is visually recognized by the speakers 2A, 2B, 2C, and 2D.

Next, display on the display unit 200 and the display 300 will be shown with reference to FIG.
FIG. 3 is a diagram illustrating an example of display on the display unit 200 and the display 300. In this FIG. 3, the state which planarly viewed the top plate surface of the desk 3 provided with the display part 200 is shown.
As shown in FIG. 3, on one display unit 200 of the two displays, for example, information corresponding to the utterance contents uttered by the speaker 2 (a plurality of speakers) is displayed. The area where the above information is displayed on the display surface of the display unit 200 is provided at a position close to each speaker 2.
For example, on the display surface of the display unit 200, regions ZA, ZB, ZC, and ZD indicating partial regions of the display surface are provided at positions close to the speakers 2A, 2B, 2C, and 2D, respectively. In this area ZA, information “A” corresponding to the utterance content (utterance content) of the speaker 2A is displayed. Similarly to the speaker 2A, information “B” corresponding to the utterance content (utterance content) of the utterer 2B is displayed in the region ZB, and the utterance content of the speaker 2C is displayed in the region ZC. Information “C” corresponding to (utterance content) is displayed, and information “D” corresponding to the speech content (utterance content) of the speaker 2D is displayed in the area ZD.

As described above, the result output processing unit 1120 (FIG. 4) displays the respective information on the display unit 200, so that, for example, the content of the utterances of the speaker 2 is expressed by the information generation unit 1110 (FIG. 4). It is possible to confirm whether or not the information according to the detected content of the message is correctly displayed.
In the other display 300 of the two displays, the result output processing unit 1120 obtains the logical product (AND condition) of information corresponding to the utterance contents of a plurality of speakers 2 among all the speakers 2. Information “E” is displayed. The information “E” may be information obtained from the logical product (AND condition) of information corresponding to the utterance contents of all the speakers 2.

As shown in FIG. 3, not only information corresponding to all the speakers 2 is displayed, but information corresponding to some of the speakers 2 may be displayed.
Note that the information displayed as “A” to “D” on the display unit 200 is the information extracted from the speech uttered by the utterer 2 as the utterance content of the utterer 2 and the utterer 2 uttered respectively. Search result (first search result) searched based on information (first information) associated with the voice (first information), information associated with the voice uttered by the speaker 2 (first information), and the search result Information selected from image information (first search information) according to (first search result) may be used.

Details of the configuration of the display system 1000 will be described with reference to FIG.
FIG. 4 is a block diagram showing the configuration of the display system of this embodiment.
4 shows a display control device 1100, a sound collection unit 101, a display unit 200, a display 300, and a search unit 11 according to the display system 1000.
As described above, the sound collection unit 101 (the sound collection unit 101A, the sound collection unit 101B, the sound collection unit 101C, and the sound collection unit 101D) collects sounds emitted by the plurality of speakers 2.
The display control apparatus 1100 obtains information based on the utterance content of the speaker 2 from the voice uttered by the speaker 2 collected by the sound collection unit 101 and displays the information on the display unit 200 and the display 300.

The display control device 1100 includes an information generation unit 1110 and a result output processing unit 1120.
The information generation unit 1110 generates information associated with the voice uttered by the speaker 2 from the voice uttered by the speaker 2.
For example, the information generated from the voice uttered by the speaker 2 and associated with the voice uttered by the speaker 2 includes any of the following information. As an example of the above information, sound piece information obtained by extracting a part of the voice uttered by the speaker 2, frequency spectrum information included in the voice uttered by the speaker 2, text information extracted from the voice uttered by the speaker 2, The word information extracted from the voice uttered by the speaker 2, the information extracted as the utterance content uttered by the speaker 2, the information associated with the above information, and the like. For the process of generating the above information from the voice uttered by the speaker 2, a known voice signal process or a process combining a plurality of known voice signal processes can be applied.
For example, the information generation unit 1110 generates the first information based on the utterance content of the utterer 2 extracted from the speech uttered by the plurality of utterers 2, and at least any utterance of the plurality of utterers 2. Second information is generated based on the utterance content of any speaker 2 extracted from the voice uttered by the person 2. In the following description of the present embodiment, processing based on information based on the utterance content of the speaker 2 extracted from the speech uttered by the speaker 2 will be mainly described.

Moreover, the information generation part 1110 detects the state of the speaker 2, and produces | generates the 2nd information matched with the audio | voice which the speaker 2 from whom the state was detected.
For example, the state of the speaker 2 detected by the information generation unit 1110 includes the posture of the speaker 2, the direction in which the face is directed, the facial expression, the frequency of speech, the volume of the voice (speech signal strength), the voice Quality, tone of voice, position of speaker 2, etc. The information generation unit 1110 identifies a specific speaker 2 to be processed from among the plurality of speakers 2 based on the determination result of determining the state of the speaker 2.

In the following description of the present embodiment, for example, the information generation unit 1110 identifies the speaker 2 by detecting the position of the speaker 2, and is associated with the voice uttered by the identified speaker 2. 2 Information shall be generated.
For the process of detecting the position of the speaker 2 in the above process, known voice signal processing that is detected based on the signal intensity and phase difference of the voice collected by the plurality of sound collection units 101 can be applied.

  Further, the information generation unit 1110 may identify the speaker 2 who is looking at the second screen, and generate second information associated with the voice uttered by the identified speaker 2.

  In addition, the information generation unit 1110 further includes information (for example, text) identified from the speech content of the utterer 2 from the voice information of the speech uttered by the utterer 2 identified as viewing the second screen. Second information (for example, word information) extracted from (information) is generated.

  Further, the information generation unit 1110 generates second information according to the state of the speaker 2. For example, the information generation unit 1110 may identify the state of the speaker 2 based on the detected posture of the speaker 2. Alternatively, the information generation unit 1110 may identify the state of the speaker 2 by the detected speech frequency (aggressiveness) of the speaker 2.

Furthermore, the information generation unit 1110 includes the first information and the first information so that the first information serving as a search condition for searching for the first search information is different from the second information serving as a search condition for searching for the second search information. 2 information is generated.
Such an information generation unit 1110 sends the generated first information and second information to the search means 11 and causes the search means 11 to execute a search based on the first information and the second information.

The result output processing unit 1120 obtains the first search result searched based on the first information from the search unit 11.
Further, the result output processing unit 1120 obtains the second search result searched based on the second information from the search unit 11. Note that the result output processing unit 1120 may obtain the second search result by selecting the first search result searched based on the first information based on the second information.

  The result output processing unit 1120 outputs the first search information corresponding to the first search result searched based on the first information associated with the voices uttered by the plurality of speakers 2. In addition, the result output processing unit 1120 outputs the first search information as described above, and based on the second information associated with the voice uttered by any one of the plurality of speakers 2. The second search information corresponding to the second search result searched is output. Here, for example, the first search information is information to be displayed on the display unit 200, and the second search information is information to be displayed on the display 300. Each of the first search information and the second search information is output from the result output processing unit 1120 as information included in image information to be displayed on the display unit 200 and the display 300. Each of the first search information and the second search information may be any of text information, graphic information, image information, and combination information thereof.

  Further, the result output processing unit 1120 outputs first search information to be displayed on the first screen and second search information to be displayed on the second screen as information to be displayed on the plurality of screens. For example, the first search information is information included in the first screen displayed on the display unit 200. More specifically: The first search information is, for example, information based on a search result searched from any of the information described below. As an example of information for performing the search for obtaining the first search information, sound piece information obtained by extracting a part of the voice uttered by the speaker 2, frequency spectrum information included in the voice uttered by the speaker 2, the speaker 2 Text information extracted from the speech uttered by the speaker, word information extracted from the speech uttered by the speaker 2, information extracted as the content of the speech uttered by the speaker 2, and information associated with the above information It is done. For example, the second search information is information included in the second screen displayed on the display 300.

The result output processing unit 1120 also detects the state when the information generation unit 1110 detects the state of the speaker 2 and generates second information associated with the voice uttered by the speaker 2 whose state is detected. Second search information based on the second information associated with the voice uttered by the detected speaker 2 is obtained. Alternatively, the result output processing unit 1120 identifies the speaker 2 by the information generating unit 1110 detecting the position of the speaker 2 and outputs the second information associated with the voice uttered by the identified speaker 2. When generated, second search information based on the second information associated with the voice uttered by the identified speaker 2 is obtained.
In addition, when the information generation unit 1110 identifies the speaker 2 who is looking at the second screen, and the result output processing unit 1120 generates the second information associated with the voice uttered by the identified speaker 2 The second search information is obtained based on the second information associated with the voice uttered by the identified speaker 2.

  Further, the result output processing unit 1120 selects the second search information to be selected from the second search results according to the state of the speaker 2. In addition, the result output processing unit 1120 outputs the first search information in the order in which it is obtained, and outputs the second search information in the order according to the priority of output.

  Next, an embodiment of processing in the display control apparatus 1100 will be described.

(Procedure 1) The information generation part 1110 detects the voice which the some speaker 2 uttered, and produces | generates the information matched with the detected voice.

(Procedure 2) The information generation part 1110 makes the search means 11 search based on the information produced | generated in the said procedure 1, ie, the 1st information matched with the audio | voice which the said several speaker uttered. For example, the information generation unit 1110 uses, as the first information, information (keywords) extracted as the utterance content uttered by the speaker 2.

(Procedure 3) The result output processing unit 1120 obtains the first search result searched by the search unit 11 based on the first information associated with the voices uttered by the plurality of speakers. For example, the result output processing unit 1120 obtains the first search result by a search based on information (keywords) included in the utterance content uttered by the speaker 2.

(Procedure 4) The result output processing unit 1120 generates first search information corresponding to the first search result, and outputs the generated first search information to the display unit 200. For example, the result output processing unit 1120 generates image information (first search information) indicating information obtained as the first search result, and outputs the generated image information to the display unit 200.

(Procedure 5) The result output processing unit 1120 uses the second information retrieved by the retrieval unit 11 based on the second information associated with the voice uttered by any one of the plurality of speakers 2. Get search results for. For example, for example, the information generation unit 1110 uses, as the second information, information (keywords) extracted as the utterance content uttered by any speaker 2 (specific speaker 2) of at least a plurality of speakers 2. To do. The result output processing unit 1120 obtains a second search result searched based on information (keyword) included in the utterance content uttered by the specific speaker 2.

(Procedure 6) The result output processing unit 1120 generates second search information corresponding to the second search result, and outputs the generated second search information to the display 300. For example, when the specific speaker 2 is one person, the result output processing unit 1120 generates image information (second search information) indicating information obtained as the second search result, and the generated image information is displayed on the display 300. Output. Alternatively, the result output processing unit 1120 performs a logical operation process based on a plurality of pieces of information obtained as the second search result when there are a plurality of specific speakers 2. Then, the result output processing unit 1120 generates image information (second search information) based on the search result of the arithmetic processing, and outputs the generated image information to the display 300.
For example, in the above arithmetic processing, the result output processing unit 1120 performs logical operation processing based on a logical product (AND) or logical sum (OR) based on a plurality of search results, or according to the importance level of the speaker 2 The priority processing obtained by the weighting calculation is performed.

  By performing the display control process according to the above procedure, the display control apparatus 1100 can display the obtained result on the display unit 200 and the display 300 as shown in FIG.

[Second Embodiment]
Next, with reference to FIG. 5, the display of the display part 200 and the display 300 is shown.
FIG. 5 is a diagram illustrating an example of display on two displays. In this FIG. 5, the state which planarly viewed the top plate | board surface of the desk 3 with which the display part 200 was provided is shown.
In addition to the areas ZA, ZB, ZC, and ZD shown in FIG. 3, the display surface of the display section 200 shown in FIG. . The region ZE is arranged at the center of the display surface of the display unit 200 when the top surface of the desk 3 is viewed in plan. In this area ZE, the information “E” displayed on the display 300 is reduced and displayed according to the size of the area ZE.
As described above, the result output processing unit 1120 (FIG. 4) is provided with the region ZE on the display surface of the display unit 200 and displays the information “E” also on the region ZE on the display surface of the display unit 200. . Accordingly, each speaker 2 can check the displayed information “E” without facing the direction of the display 300. In short, if the display is as shown in FIG. 3, the speaker 2 located on the display 300 side with respect to the desk 3 has the desk 3 on the front side of the speaker 2 and the display 300 on the back side. Such a speaker 2 cannot confirm the display on the display 300 without turning around, but the information “E” can be confirmed without turning around by displaying as shown in FIG. 5.

[Third Embodiment]
Next, with reference to FIG. 1 to FIG. 4, a description will be given of a case where information corresponding to the content of a statement made by a specific speaker 2 is displayed on the display 300.

In this embodiment, (procedure 1-1) is performed instead of (procedure 1) described above, and (procedure 2) to (procedure 6) are performed.
(Procedure 1-1) The information generation unit 1110 detects voices uttered by a plurality of speakers 2, associates the detected speech with each speaker, and the speaker 2 having a high signal level of the detected speech. Is identified. The information generation unit 1110 generates information associated with the detected voice as the voice of the speaker 2 having a high signal level of the detected voice.

A detailed description of the configuration for identifying the speaker 2 having a high voice signal level in the above (procedure 1-1) will be described later.
In addition, the information generation part 1110 is good also as specifying the speaker 2 in said (procedure 1-1) according to the state of the speaker 2. FIG. When the speaker 2 is specified according to the state of the speaker 2, the posture of the speaker 2, the direction in which the face is directed, the voice You may implement based on quality, the tone of a voice, the position of the speaker 2, etc. Due to the directivity of the sound collection characteristic of the sound collection unit 101, the information generation unit 1110 determines the speaker 2 specified according to the state of the speaker 2, and the loudness of the sound produced by the detected speaker 2 This can be implemented based on the signal level.

[Fourth Embodiment]
Next, with reference to FIG. 1 to FIG. 4, a description will be given of a case where information corresponding to the content of a statement made by a specific speaker 2 is displayed on the display 300.

In this embodiment, (procedure 1-2) is performed instead of (procedure 1) described above, and (procedure 2) to (procedure 6) are performed.
(Procedure 1-2) The information generation part 1110 detects the voice which the some speaker 2 uttered, and detects the position of each speaker from the detected voice. The information generation unit 1110 associates the detected voice with each speaker and identifies the speaker 2 according to the detected position of each speaker. The information generation unit 1110 generates information associated with the voice of the specified speaker 2.
Here, the information generation unit 1110 identifies the speaker 2 who is speaking at a position close to the display 300, for example. Thereby, the information generation part 1110 can identify the speaker 2 who is speaking with reference to the information displayed on the display 300.
A detailed description of the configuration for identifying the position of each speaker from the voice in (Procedure 1-2) will be described later.

[Fifth Embodiment]
Next, with reference to FIG. 1 to FIG. 4, a description will be given of a case where information corresponding to the content of a statement made by a specific speaker 2 is displayed on the display 300.

In this embodiment, (procedure 1-3) is performed instead of (procedure 1) described above, and (procedure 2) to (procedure 6) are performed.
(Procedure 1-3) The information generation unit 1110 detects voices uttered by a plurality of speakers 2, detects the speech frequency of each speaker 2 from the detected speech, and stores the detected speech frequency information in a storage area. Let me. The information generation unit 1110 associates the detected speech with each speaker, and identifies the speaker 2 having a high speech frequency according to the speech frequency information of each detected speaker 2. The information generation unit 1110 generates information associated with the voice of the specified speaker 2.
Here, the information generation part 1110 can identify the speaker 2 who is actively speaking, for example, by detecting the amount of speech of the speaker 2 according to the speech frequency information.
A detailed description of the configuration for identifying a speaker with a high speech frequency from the voice in (Procedure 1-3) will be described later.

[Sixth Embodiment]
Next, with reference to FIG. 1 to FIG. 4, a description will be given of a case where information corresponding to the content of a statement made by a specific speaker 2 is displayed on the display 300.

In the present embodiment, following (Procedure 1) to (Procedure 5), the following (Procedure 6-1) is performed.
(Procedure 6-1) The result output processing unit 1120 generates second search information corresponding to the second search result, and outputs the generated second search information to the display 300. For example, when the speaker 2 (specific speaker 2) in one utterance can be specified, the result output processing unit 1120 uses image information (second search information) indicating information obtained as the second search result. The generated image information is output to the display 300. Alternatively, the result output processing unit 1120 performs a logical operation process based on a plurality of pieces of information obtained as the second search result when a plurality of speaking speakers 2 (specific speakers 2) are specified. Then, the result output processing unit 1120 generates image information (second search information) based on the search result of the arithmetic processing, and outputs the generated image information to the display 300.
For example, in the above arithmetic processing, the result output processing unit 1120 performs logical operation processing based on a logical product (AND) or logical sum (OR) based on a plurality of search results, or according to the importance level of the speaker 2 The priority processing obtained by the weighting calculation is performed.

By performing the display control process according to the above procedure, the display control apparatus 1100 can display the obtained result on the display unit 200 and the display 300 as shown in FIG.
Moreover, in this embodiment, the result output process part 1120 utters the same statement content among the some speaker 2 by performing the logical operation process by the AND (AND) based on several search results. It can be detected that a certain speaker 2 is present. Thereby, it can be determined that a desired result is obtained as a search result.
Furthermore, when different search results are obtained, the result output processing unit 1120 detects the number of people who can determine that the same statement content is being spoken, so that the larger the number, the more the topic of the same statement content is spoken. It can be determined that the estimated accuracy is high. In such a case, the result output processing unit 1120 generates the second search information according to the search result corresponding to the utterance content with a large number of persons who are determined to utter the same utterance content. As a result, the result output processing unit 1120 can cause the display 300 to display the message content with a large number of people who have determined that the same message content is being spoken.

  In addition, the detailed description of the structure which specifies the content of the speech which the specific speaker 2 said in the said (procedure 6-1) is mentioned later.

[Seventh Embodiment]
Next, a seventh embodiment of the present invention will be described with reference to FIG.
FIG. 6 is a diagram illustrating an overview of a display system including the display control device according to the present embodiment.
The display system 1000B shown in FIG. 6 includes a display control device 1100B (not shown). Unless otherwise specified, the display system 1000 in the above description is replaced with the display system 1000B, and the display control device 1100 is replaced with the display control device 1100B.
The display system 1000B includes an imaging unit 150. The imaging unit 150 is provided in the display unit 200 of the display system 1000B. The imaging unit 150 shown in this figure shows only a portion provided for one speaker 2.
The imaging unit 150 includes a plurality of imaging devices including an imaging element, and outputs images for detecting the state of the speaker 2. The imaging unit 150 of the present embodiment includes four imaging units 150, for example, an imaging unit 150A, an imaging unit 150B, an imaging unit 150C, and an imaging unit 150D, and the imaging direction is the direction of each speaker 2, respectively. It is provided for. Here, the imaging unit 150A images, for example, the direction opposite to the direction DR1 (FIG. 1). Similarly, the imaging unit 150B images, for example, the reverse direction of the direction DR2 (FIG. 1), the imaging unit 150C images, for example, the reverse direction of the direction DR3 (FIG. 1), and the imaging unit 150D, for example, The reverse direction of the direction DR4 (FIG. 1) is imaged.

Details of the configuration of the display system 1000B will be described with reference to FIG.
FIG. 7 is a block diagram showing the configuration of the display system in the present embodiment.
7 shows a display control apparatus 1100B, a sound collection unit 101, an imaging unit 150, a display unit 200, a display 300, and a search unit 11 according to the display system 1000B.
The imaging unit 150 (imaging unit 150A, imaging unit 150B, imaging unit 150C, imaging unit 150D) images the situation of the speaker 2 as described above.
The display control device 1100B obtains information based on the utterance content of the speaker 2 from the voice uttered by the speaker 2 collected by the sound collection unit 101 based on the result captured by the imaging unit 150. Displayed on the display unit 200 and the display 300.

The display control apparatus 1100B includes an information generation unit 1110B and a result output processing unit 1120.
The information generation unit 1110B detects the state of the speaker 2 based on the image for detecting the state of the speaker 2 output from the imaging unit 150. The state of the speaker 2 by the information generation unit 1110B can be detected by a known image processing method or a combination of known image processing methods. The information generation unit 1110 </ b> B may identify the speaker 2 based on the state of the speaker 2 based on the image obtained by the imaging unit 150. When the information generation unit 1110B specifies the speaker 2 according to the state of the speaker 2, the posture and face of the speaker 2 are directed instead of the loudness (signal level) of the speaker 2 It may be carried out based on the direction of the speaker, the position of the speaker 2, and the like. Based on the image captured by the imaging unit 150, the information generation unit 1110B identifies the speaker 2 according to the state of the speaker 2.
For example, the information generation unit 1110 </ b> B detects the speaker 2 by detecting the posture of the speaker 2 and the direction in which the speaker 2 faces the face based on the image obtained by the imaging unit 150. Detect the screen that is. The information generation unit 1110B determines that the speaker 2 is looking at the image displayed on the display unit 200 if the speaker 2 is looking down and looking at the direction of the imaging unit 150. In addition, the information generation unit 1110B looks at the image displayed on the display 300 if the speaker 2 turns away from the direction of the imaging unit 150 and faces in the direction toward the display 300. Is determined.
As described above, the information generation unit 1110B may identify the speaker 2 who is looking at the second screen, and generate the second information associated with the voice uttered by the identified speaker 2.

  In addition, the information generation unit 1110B further detects information (for example, text) from the voice information of the voice uttered by the speaker 2 identified as viewing the second screen. Second information (for example, word information) extracted from (information) is generated.

  In addition, the information generation unit 1110B generates second information according to the state of the speaker 2. For example, the information generation unit 1110B may identify the state of the speaker 2 based on the detected posture of the speaker 2.

In the present embodiment, (procedure 1-4) is performed instead of (procedure 1) described above, and (procedure 2) to (procedure 6) are performed.
(Procedure 1-4) The information generation unit 1110B detects voices uttered by a plurality of speakers 2, and associates the detected voices with the respective speakers. In addition, the information generation unit 1110B detects the states (postures) of a plurality of speakers 2, and identifies the speaker 2 according to the detected states (postures) of the speakers 2. For example, the information generation unit 1110B specifies the speaker 2 who is looking at the display 300 from the detected state (posture) of the speaker 2, and stores information associated with the detected speech as the voice of the specified speaker 2. Generate.

  As described above, the information generation unit 1110B specifies the speaker 2 according to the state of the speaker 2 as described above (procedure 1-4), and thereby the size (signal) of the speaker 2 uttered. Instead of the level), it is specified based on the posture of the speaker 2, the direction in which the face is directed, the position of the speaker 2, and the like.

[Eighth Embodiment]
Next, an eighth embodiment of the present invention will be described with reference to FIG.
FIG. 8 is a diagram illustrating an overview of a display system including the display control device according to the present embodiment.
The display system 1000C shown in FIG. 8 includes a display control device 1100C. Unless otherwise specified, the display system 1000 in the above description is replaced with the display system 1000C, and the display control device 1100 is replaced with the display control device 1100C.
The display system 1000C is configured to include a plurality of display units 200.
A sound collection unit 101, a plurality of display units 200, and a display 300 are connected to the display control device 1100C. The display control device 1100C causes the display 300 to display the display content displayed on any of the display units 200. The display control device 1100C displays information corresponding to the utterance of the nearest speaker 2 on each of the plurality of display units 200. The display contents to be displayed on each display unit 200 are displayed on the display surface of the display unit 200 in information displayed in an area closer to the nearest speaker 2 and in an area away from the speaker 2. The information may be different.
For example, the display control apparatus 1100C displays information corresponding to the utterance of the nearest speaker 2 on each of the plurality of display units 200. The display content displayed on each display unit 200 is displayed in a region closer to the nearest speaker 2 on the display surface of the display unit 200.
FIG. 9 is a diagram showing an overview of a display system provided with a plurality of display units in the present embodiment.
The display system 1000 </ b> C illustrated in FIG. 9 includes a display control device 1100 that displays information on a plurality of display units 200. The information displayed on the plurality of display units 200 is the same as the information displayed on the display unit 200 described above.

[Ninth Embodiment]
Next, a display unit 200A having a shape different from that of the display unit 200 will be described with reference to FIG.
FIG. 10 is a diagram showing an overview of a display system provided with a display unit in the present embodiment.
The display system 1000D shown in FIG. 10 includes a display control device 1100.
The display system 1000D is configured to include a display unit 200A. Unless otherwise specified, the display unit 200 in the above description is replaced with the display unit 200A.
The sound collection unit 101, the display unit 200A, and the display 300 are connected to the display control device 1100D.
The display unit 200A has a flat display surface. For example, a flat panel display device such as an LCD can be applied to the display unit 200A.

  Next, in the tenth embodiment to the eleventh embodiment, an information display system (conference system) which is an embodiment of the present invention will be described. In the following description, details of the process for identifying the speaker will be described.

[Tenth embodiment]
Hereinafter, with reference to the drawings, audio signal processing applied as one embodiment of the present invention will be described by exemplifying a case where it is applied to a conference system.
FIG. 11 is a diagram illustrating an outline of the conference system 1 including the search unit control device 1101 according to the present embodiment.
The conference system 1 includes a search means control device 1101.
The search means control device 1101 includes a sound collection unit 101. Further, the search means control device 1101 is provided on the desk 3, for example, together with the sound collection unit 101 and the display unit 200A. The search means control device 1101 is connected to the search means 11 via the network 15.
The search means 11 searches for information related to the input character information. The search means 11 of this embodiment is a search engine that searches for information related to character information input via a network 15 such as the Internet or a LAN. Further, the search means 11 outputs the search result of information related to the input character information to the search means control device 1101 via the network 15 as a search result.

The display unit 200A displays an image corresponding to the input image information. The display unit 200 </ b> A of the present embodiment is embedded in, for example, the top plate surface of the desk 3 so that the displayed image can be viewed by a person (speaker) around the desk 3.
The sound collection unit 101 includes, for example, a microphone that collects sound, and collects sound emitted by a plurality of speakers 2. The sound collection unit 101 of the present embodiment includes, for example, four sound collection units 101, a sound collection unit 101A, a sound collection unit 101B, a sound collection unit 101C, and a sound collection unit 101D. The sound emitted by is picked up. Here, the sound collection unit 101A collects sound emitted in the direction DR1, for example. Similarly, the sound collection unit 101B emits, for example, a sound emitted in the direction DR2, the sound collection unit 101C, for example, a sound emitted in the direction DR3, and the sound collection unit 101D, for example, emits a sound in the direction DR4. Pick up each voice. For example, as shown in FIG. 12, the sound collection unit 101 is a speaker 2A, 2B, 2C seated at a predetermined position of the desk 3, for example, a position corresponding to the sound collection units 101A, 101B, 101C, 101D. The voice uttered by four 2D speakers 2 is collected.

FIG. 12 is a configuration diagram showing the configuration of the conference system 1 used by the speaker 2.
In this case, the sound collection unit 101A collects the sound mainly emitted by the speaker 2A. Similarly, the sound collecting unit 101B collects the sounds emitted mainly by the speaker 2B, the sound collecting unit 101C mainly by the speaker 2C, and the sound collecting unit 101D mainly collects the sounds emitted by the speaker 2D. To do. Here, for example, the conference system 1 detects which speaker 2 has collected the sound by detecting which sound collecting unit 101 out of the sound collecting units 101A, 101B, 101C, and 101D. Can be determined.
In this case, the display unit 200A displays images to the speakers 2A, 2B, 2C, and 2D so that the images displayed on the speakers 2A, 2B, 2C, and 2D are visually recognized.

Next, the details of the configuration of the conference system 1 will be described with reference to FIG.
FIG. 13 is a block diagram showing a configuration of the conference system 1 of the present embodiment.
The search means control device 1101 of this embodiment includes a sound collection unit 101, a generation unit 102, a conversion unit 103, an identification unit 104, a storage unit 105, a selection unit 106, an output unit 107, and a result selection unit. 108, a setting unit 109, and a result output unit 110.

  As described above, the sound collection unit 101 (the sound collection unit 101A, the sound collection unit 101B, the sound collection unit 101C, and the sound collection unit 101D) collects sounds emitted by the plurality of speakers 2.

  The production | generation part 102 produces | generates the information of the direction in which the audio | voice was emitted based on audio | voice information. Here, the direction in which the sound is generated is, for example, the direction indicated by the directions DR1 to DR4 in FIG. 11 as described above. For example, the generation unit 102 according to the present embodiment determines, based on the audio information, which sound collection unit 101 out of the plurality of sound collection units 101, and the sound is emitted. Generate direction information. That is, the generation unit 102 generates information on the direction in which the sound is emitted based on information indicating the volume of the sound collected for each sound collection unit 101.

  The conversion unit 103 converts voice information uttered by the speaker 2 into character information. For example, the conversion unit 103 of the present embodiment recognizes the input voice information by a known voice recognition unit, and converts the voice information into character information.

  Based on the voice information, the identification unit 104 identifies the speaker 2 that uttered the voice of the voice information from among the plurality of speakers 2, and character information obtained by converting the identified speaker 2 and the voice information; Are output in association. For example, the identification unit 104 of the present embodiment recognizes the input voice information by a known voice recognition unit, and extracts the voice characteristics of the speaker 2. The identifying unit 104 identifies voice information for each extracted voice feature of the speaker 2, determines the number of the speaker 2, and identifies information (for example, the speaker 2) to associate with the voice information. A unique identification ID) is assigned to each determined speaker 2. For example, the identification unit 104 assigns 2A as the identification ID for the speaker 2A, and assigns 2B as the identification ID for the speaker 2B. The identification unit 104 similarly assigns identification IDs to the speaker 2C and the speaker 2D. And the identification part 104 assign | provides identification ID provided for every speaker 2 for every input audio | voice information. Then, the identification unit 104 assigns the identification ID assigned to the input voice information to the character information obtained by converting the voice information by the conversion unit 103.

For example, when the voice of “Lion” is uttered by the speaker 2A, the identification unit 104 determines that the input voice information of “Lion” matches the characteristics of the voice information of the speaker 2A. In this case, “2A” as the identification ID of the speaker 2A is assigned to “Lion” as voice information. Then, the identification unit 104 gives the identification ID “2A” to “lion” as character information converted by the conversion unit 103 from “lion” as voice information. That is, the identification unit 104 associates the speaker 2A with “lion” as character information, and outputs the character information.
In this way, the identification unit 104 outputs the identified speaker 2 and the character information obtained by converting the voice information in association with each other.

  Further, the identification unit 104 generates frequency information indicating the frequency at which the speaker 2 utters voice for each speaker 2 identified by the identification unit 104, and stores the generated frequency information and the speaker 2 in association with each other. Store in the unit 105. In this case, the storage unit 105 stores the frequency and the speaker 2 in association with each speaker 2 identified by the identification unit 104.

The selection unit 106 selects specific character information from the character information based on the importance level of the speaker. The importance level of the speaker can be set variously.
For example, the importance level of the speech information includes the importance level of the voice information, and the selection unit 106 of the present embodiment, for example, selects the importance level of the voice information from the character information converted by the conversion unit 103. Based on, select specific character information.

  Specifically, the importance of the voice information is determined based on the volume of the voice uttered by the speaker 2, and the voice information includes volume information. Based on the contained volume information, specific character information is selected from the character information. For example, in this case, the selection unit 106 selects voice information emitted at a higher volume from the voice information, and selects character information obtained by converting the selected voice information as specific character information.

  Specifically, the importance of the voice information is determined based on the direction in which the voice is emitted, and the selection unit 106 is based on the information on the direction in which the voice is generated, which is generated by the generation unit 102. The specific character information is selected from the character information. For example, the selection unit 106 selects voice information uttered by the speaker 2 seated in a predetermined direction among the directions of the desk 3 (for example, directions DR1 to DR4), and the selected voice information The character information converted from is selected as specific character information. Here, the predetermined direction is, for example, a direction in which voice is emitted by the speaker 2 who leads the conference among the speakers 2. The predetermined direction is not limited to one direction and may be a plurality of directions. Further, the predetermined direction is not limited to the direction in which the voice is generated by the single speaker 2, and may include the direction in which the voice is generated by the plurality of speakers 2.

  Specifically, the importance of the voice information is determined based on the position where the voice is emitted, the voice information includes position information, and the selection unit 106 is included in the voice information. Based on the position information, specific character information is selected from the character information. For example, the selection unit 106 includes voice information uttered by the speaker 2 seated at a specific position among the seat positions of the desk 3 (for example, the seat positions corresponding to the sound collection units 101A to 101D). And the character information obtained by converting the selected voice information is selected as the specific character information.

  In addition, for example, the importance level of the speaker includes the importance level of the speaker 2, and the selection unit 106 of the present embodiment, for example, based on the importance level of the speaker 2, The specific speaker 2 is selected, and the specific character information is selected from the character information associated with the selected specific speaker 2. That is, the selection unit 106 selects, as the specific character information, the character information obtained by converting the voice information uttered by the specific speaker 2 from the character information converted by the conversion unit 103.

  Specifically, the importance level of the speaker 2 is determined based on the frequency at which the speaker 2 uttered voice, and the selection unit 106 selects a specific level based on the frequency stored in the storage unit 105. Speaker 2 is selected. As described above, the storage unit 105 stores the frequency and the speaker 2 in association with each speaker 2 identified by the identification unit 104. For example, the selection unit 106 of the present embodiment selects and selects the speaker 2 having a high utterance frequency as the specific speaker 2 based on the utterance frequency stored for each speaker 2 in the storage unit 105. Specific character information is selected from character information associated with a specific speaker 2.

The output unit 107 outputs the specific character information selected by the selection unit 106 to the search unit 11 that searches for information related to the input character information. For example, the output unit 107 of the present embodiment can communicate with the search unit 11 connected via the network 15, and the specific character information selected by the selection unit 106 is sent to the search unit 11. Send.
The output unit 107 determines whether or not to output the character information to the search unit 11 based on whether or not the character information includes a predetermined word / phrase. Here, the predetermined phrase is, for example, a moving verb such as “That's right” or “He”, or a questioning word such as “when” or “where”. For example, the output unit 107 determines that the character information is not output to the search unit 11 when the character information includes “sane” as a predetermined word / phrase. Further, the output unit 107 determines to output the character information to the search unit 11 when the character information does not include a predetermined word / phrase.

  The setting unit 109 sets a feature amount based on the audio information. The setting unit 109 according to the present embodiment acquires, for example, voice information collected by the sound collection unit 101, identifies information (for example, a voiceprint) indicating a feature amount included in the acquired voice information, and speaks. 2 attributes such as gender and age are estimated. Then, the setting unit 109 sets the estimated attribute as a feature amount used by the result selection unit 108 for selection.

  The result selection unit 108 selects information indicating the search result output by the search unit 11 based on the feature amount indicating the feature of the voice of the specific speaker 2. Here, the feature amount indicating the voice feature of the specific speaker 2 is, for example, a feature amount indicating attributes of the speaker 2 such as the gender and age of the speaker 2. Further, the information indicating the search result output by the search unit 11 may be image information of a picture of a lion as an animal if the character information output to the search unit 11 is, for example, “lion” as described above. It is text information indicating the name of the zoo that keeps the lion, image information of a map indicating the lion's habitat, and the like. The result selection unit 108 according to the present embodiment is, for example, information indicating the search result output by the search unit 11 according to the feature amount set by the setting unit 109 (for example, attributes such as the gender and age of the speaker 2). From this, information that matches the attribute of the speaker 2 is selected. For example, when the character information output to the search means 11 is “Lion”, for example, the result selection unit 108 determines that the attribute of the estimated speaker 2 is “Child”, among the information indicating the search result. From, for example, image information of an illustration of “lion” drawn for children is selected.

  The result output unit 110 outputs the image information selected by the result selection unit 108 to the display unit 200A. In this way, the search means control device 1101 of the conference system 1 controls the search means 11.

Next, with reference to FIG. 14, the operation of the search means control device 1101 provided in the conference system 1 will be described.
FIG. 14 is a flowchart showing the operation of the search means control device 1101 provided in the conference system 1 of the present embodiment.
First, the sound collection unit 101 of the search means control device 1101 acquires (sound collection) voice information uttered by the speaker 2 (step S10). The sound collection unit 101 of the present embodiment collects, for example, sound information emitted by the speaker 2A by the sound collection unit 101A. In this case, the direction in which the voice is emitted by the speaker 2A is the direction DR1 shown in FIG. That is, the sound collection unit 101A collects the sound uttered in the direction DR1 by the speaker 2A.

  Next, the conversion part 103 of the search means control apparatus 1101 converts the voice information uttered by the speaker 2 into character information (step S20). The conversion unit 103 of the present embodiment converts voice information into character information by voice recognition, for example.

  Next, the identification unit 104 of the search means control device 1101 identifies the speaker 2 that uttered the voice information of the plurality of speakers 2 based on the voice information, and identifies the identified speaker 2 and The character information obtained by converting the voice information is output in association with the character information (step S30). That is, the identification unit 104 identifies the speaker 2. Specifically, the voice information acquired in step S10 is voice-recognized by a known voice recognition means, and the voice characteristics of the speaker 2 are extracted. Here, the voice features include voiceprint features and the like. The identifying unit 104 identifies voice information for each extracted voice feature of the speaker 2, determines the number of the speaker 2, and identifies information (for example, the speaker 2) to associate with the voice information. A unique identification ID) is assigned to each determined speaker 2. And the identification part 104 assign | provides identification ID provided for every speaker 2 for every input audio | voice information. Then, the identification unit 104 assigns the identification ID assigned to the input voice information to the character information obtained by converting the voice information by the conversion unit 103.

  Next, the identification unit 104 of the search means control device 1101 stores the frequency of utterance in the storage unit 105 in association with the speaker 2 for each identified speaker 2 (step S40). For example, the identification unit 104 counts the number of utterances for each speaker 2 and stores the counted number of utterances in the storage unit 105 in association with the speaker 2.

  Next, the generation unit 102 of the search means control device 1101 generates information on the direction in which the voice is emitted based on the voice information (step S50). For example, the generation unit 102 of the present embodiment generates information indicating the direction DR1 as the direction in which sound is emitted, based on the sound information collected by the sound collection unit 101A.

Next, the selection unit 106 of the search means control device 1101 selects specific character information from the character information based on the importance level of the speaker (step S60). In step S60, the selection unit 106 of the present embodiment selects specific character information, for example, according to the procedure shown in FIG.
FIG. 15 is a flowchart illustrating an example of the operation of the selection unit 106 according to the present embodiment.
First, the selection unit 106 determines whether the utterance frequency exceeds a predetermined threshold (step S601). For example, the selection unit 106 of the present embodiment reads information on the frequency (for example, the number of utterances) of the frequency of the utterer 2 uttering voice stored in the storage unit 105 for each utterer 2. Then, the selection unit 106 determines for each speaker 2 whether or not the read frequency information exceeds a predetermined threshold value (for example, 5 times per minute). If the selection unit 106 determines that the read frequency information has exceeded a predetermined threshold for at least one speaker 2 (step S601: YES), the selection unit 106 advances the process to step S602.
And the selection part 106 is the character information by which the voice uttered by the speaker 2 (namely, the speaker 2 with high utterance frequency) linked | related with the information of the frequency exceeding the predetermined threshold value is converted. Is selected as specific character information, and the process of step S60 is terminated (step S602).
On the other hand, if the selection unit 106 determines that the read frequency information does not exceed the predetermined threshold value for each speaker 2 (step S601: NO), the selection unit 106 proceeds to step S611.

Next, the selection unit 106 determines whether or not the volume of the voice uttered by the speaker 2 exceeds a predetermined threshold value (step S611). For example, the selection unit 106 according to the present embodiment acquires volume information included in the audio information from the sound collection unit 101, and stores whether the acquired volume information exceeds a predetermined threshold. The determination is made for each sound unit 101. If the selection unit 106 determines that the acquired volume information exceeds a predetermined threshold (step S611: YES), the selection unit 106 advances the process to step S612.
Then, the selection unit 106 selects, as specific character information, character information in which voice information exceeding a predetermined threshold is converted, and ends the process of step S60 (step S612).
On the other hand, when the selection unit 106 determines that the volume information acquired from each sound collection unit 101 does not exceed the predetermined threshold (step S611: NO), the selection unit 106 advances the process to step S621.

Next, the selection unit 106 determines whether or not the direction of the voice uttered by the speaker 2 is a predetermined direction (step S621). For example, the selection unit 106 according to the present embodiment acquires information on the direction in which the sound generated by the generation unit 102 is emitted, and determines whether the acquired information on the direction indicates a predetermined direction. If the selection unit 106 determines that the acquired direction information indicates a predetermined direction (step S621: YES), the selection unit 106 advances the process to step S622.
Then, the selection unit 106 selects, as specific character information, character information in which sound information indicating a predetermined sound direction is converted, and ends the process of step S60 (step S622).
On the other hand, if the selection unit 106 determines that the acquired direction information does not indicate the predetermined direction (step S621: NO), the selection unit 106 advances the process to step S631.

Next, the selection unit 106 determines whether or not the position where the voice is uttered by the speaker 2 is a predetermined position (step S621). For example, the selection unit 106 of the present embodiment acquires voice information from the sound collection unit 101, and the position where the sound is emitted is determined based on the information of the position where the sound included in the acquired sound information is emitted. It is determined whether or not the position is a predetermined position (step S631). For example, the selection unit 106 is uttered by the speaker 2 based on which sound collection unit 101 of the plurality of sound collection units 101 is the acquired sound information. It is determined whether or not the position is a predetermined position. If the selection unit 106 determines that the position information included in the acquired audio information indicates a predetermined position (step S631: YES), the selection unit 106 advances the process to step S632.
Then, the selection unit 106 selects character information in which voice information indicating a predetermined position is converted as specific character information, and ends the process of step S60 (step S632).
On the other hand, when the selection unit 106 determines that the position information included in the acquired voice information does not indicate the predetermined position (step S631: NO), the selection unit 106 selects specific character information from the character information. The process of step S60 is terminated without making a selection.
In this manner, the selection unit 106 selects specific character information from the character information based on the importance level of the speaker.

With reference to FIG. 14 again, the operation of the search means control device 1101 provided in the conference system 1 will be described.
If the specific character information can be selected in each step of step S60 described above (step S65: YES), search means control apparatus 1101 advances the process to step S70. On the other hand, if the specific character information cannot be selected in each step of step S60 described above (step S65: NO), search means control apparatus 1101 returns the process to step S10.

  Next, the output unit 107 of the search unit control device 1101 outputs the specific character information selected by the selection unit 106 to the search unit 11 that searches for information related to the input character information (step S70). For example, the output unit 107 of the present embodiment transmits specific character information as a search character string to the search unit 11 connected via the network 15. As described above, the search unit control device 1101 controls the search unit 11 by causing the search unit 11 to search for information related to specific character information.

  Next, the setting unit 109 of the search means control device 1101 sets a feature quantity indicating the characteristics of the voice of the specific speaker 2 based on the voice information acquired from the sound collection unit 101 (step S80). The setting unit 109 according to the present embodiment sets, for example, attributes such as the gender and age of the speaker 2 as feature amounts based on the audio information acquired from the sound collection unit 101.

  Next, the result selection unit 108 of the search unit control device 1101 selects information indicating the search result output by the search unit 11 based on the feature amount indicating the feature of the voice of the specific speaker 2 (step S90). . The result selection unit 108 of the present embodiment, for example, from among a plurality of search results output by the search unit 11 based on the feature amount (for example, gender and age) set by the setting unit 109 in step S80. Select specific information.

Next, the result output unit 110 of the search means control device 1101 outputs the specific information selected in step S90 to the display unit 200A, displays an image of the output information on the display unit 200A, and ends the process. (Step S100).
In this way, the search means control device 1101 controls the search means 11 based on the voice information uttered by the speaker 2 to display specific information on the display unit 200A.

  As described above, the search unit control device 1101 of this embodiment includes the conversion unit 103, the selection unit 106, the output unit 107, and the result output unit 110. The conversion unit 103 converts voice information of a voice uttered by the speaker 2 into character information. The selection unit 106 selects specific character information from the character information based on the importance level of the speaker. The output unit 107 outputs the specific character information selected by the selection unit 106 to the search unit 11 that searches for information related to the input character information. In addition, the result output unit 110 causes the display device that displays the search results to output the search results by the search means. Thereby, the search means control device 1101 of the present embodiment automatically converts the voice information uttered by the speaker 2 into character information and causes the search means 11 to search, and based on the importance level of the speaker, Character information to be searched by the search means 11 can be selected. Thereby, since the search means control apparatus 1101 of this embodiment can convert voice information into character information automatically, for example, the character information based on the voice information uttered by the speaker 2 is converted into the speaker. It is possible to eliminate the need for two persons or those around to input to the search means 11. Furthermore, since the search means control apparatus 1101 can select the character information to be searched by the search means 11 from the character information based on the voice information uttered by the speaker 2, for example, a speech with high importance It is possible to make the search means 11 search by narrowing down the content.

  For example, when a conference is performed using the conference system 1 including the search unit control device 1101 of the present embodiment, the search unit 11 may search for information related to the utterance content according to the utterance content of the speaker 2. it can. Thus, since the search means control apparatus 1101 can automatically provide the speaker 2 with information related to the utterance content of the speaker 2, the topic of the conference can be reduced while reducing the flow of the conference. Can be expanded and deepened. Furthermore, since the search means control device 1101 can make the search means 11 search for utterance contents with high importance, for example, it is possible to reduce provision of information unnecessary for the speaker 2 to the speaker 2. be able to. That is, the search means control apparatus 1101 of this embodiment can provide the speaker 2 with information for smoothly proceeding with the conference.

  In addition, the selection unit 106 included in the search unit control device 1101 according to the present embodiment selects specific character information from character information based on the importance of voice information. Here, the importance of the voice information is included in the importance of the speaker. Thereby, the search means control apparatus 1101 of this embodiment can make the search means 11 search from the speech information generated by the speaker 2 to the utterance contents having high importance. That is, the search means control device 1101 can reduce the search for information unnecessary for the speaker 2.

  In addition, the selection unit 106 included in the search unit control device 1101 of the present embodiment selects specific character information from the character information based on the volume information included in the sound information. Here, the importance of the voice information is determined based on the volume of the voice uttered by the speaker 2. Thereby, the search means control apparatus 1101 of this embodiment can make the search means 11 search from the speech information generated by the speaker 2 to the utterance contents having high importance. For example, the selection unit 106 determines that the content spoken by the speaker 2 at a volume that is difficult for other speakers 2 to hear is low in importance, and the volume at which the speaker 2 can be heard by other speakers 2. It can be determined that the content uttered in is highly important. For example, the search means control device 1101 determines that the content that the speaker 2A is talking to the speaker 2B sitting next to the speaker is low in importance, and the converted character information of the voice information Information is not output to the search means 11. That is, the search means control device 1101 can reduce the search for information unnecessary for the speaker 2.

  In addition, the search unit control device 1101 of the present embodiment includes a generation unit 102 that generates information on the direction in which the voice is emitted based on the voice information, and the selection unit 106 adds the information on the generated direction. Based on the character information, specific character information is selected. Here, the importance of the voice information is determined based on the direction in which the voice is emitted. Thereby, the search means control apparatus 1101 of this embodiment can make the search means 11 search from the speech information generated by the speaker 2 to the utterance contents having high importance. For example, when there is a speaker 2 (for example, a speaker 2A and a speaker 2B) on the side to be explained in a conference and a speaker 2 (for example, a speaker 2C and a speaker 2D) on the side to be explained, the selection unit 106 can determine that the voice information in the direction in which the voice is uttered by the speaker 2A and the speaker 2B is high in importance. As a result, the search means control device 1101 can output, for example, character information obtained by converting the voice information of the speaker 2 on the side of the explanation to the search means 11, so that the explanation materials are sequentially added according to the progress of the explanation. Can be displayed. That is, the search means control apparatus 1101 can provide the speaker 2 with information for smoothly proceeding with the conference.

  In addition, the search unit control device 1101 of the present embodiment includes a plurality of sound collection units 101 that collect the sound uttered by the speaker 2, and the generation unit 102 collects the sound collected for each sound collection unit 101. Based on the information indicating the sound volume, information on the direction in which the sound is emitted is generated. Thereby, the search means control apparatus 1101 of this embodiment does not need to generate | occur | produce the information of the direction in which the audio | voice was emitted based on the audio | voice information collected by the single sound collection part 101, for example, Information of the direction in which the voice is emitted can be generated easily.

  In addition, the selection unit 106 of the search unit control device 1101 of the present embodiment selects specific character information from character information based on position information included in the audio information. Here, the importance of the voice information is determined based on the position where the voice is emitted, and the voice information includes position information. Thereby, the search means control apparatus 1101 of this embodiment can make the search means 11 search based on the audio | voice information uttered by the speaker 2 seated in the specific position, for example. For example, the search means control device 1101 is based on the audio information uttered by the speaker 2 (for example, the speaker 2A) seated in the seat of the facilitator (for example, the chairman) of the conference as a specific position, The search means 11 can search. As a result, the search means control device 1101 can output, for example, character information obtained by converting the voice information of the facilitating speaker 2A to the search means 11, so that related materials are sequentially added according to the progress of the conference. Can be displayed. That is, the search means control apparatus 1101 can provide the speaker 2 with information for smoothly proceeding with the conference.

  Further, the search means control device 1101 of this embodiment includes an identification unit 104. Based on the voice information, the identification unit 104 identifies the speaker 2 that uttered the voice of the voice information from among the plurality of speakers 2, and character information obtained by converting the identified speaker 2 and the voice information; Are output in association. Further, the selection unit 106 of the search means control device 1101 of the present embodiment selects a specific speaker 2 from the speakers 2 based on the importance of the speaker 2, and selects the selected specific speaker 2 Select specific character information from the character information associated with. Here, the importance level of the speaker 2 is included in the importance level of the speaker. Thereby, the search means control device 1101 of the present embodiment is uttered by the speaker 2A even if the speaker 2A moves the position (for example, the seat) when the speaker 2A is the facilitator. Based on the voice information, the search unit 11 can search. That is, the search means control apparatus 1101 can provide the speaker 2 with information for smoothly proceeding with the conference.

  Further, the search means control device 1101 of the present embodiment includes a storage unit 105 in which the frequency and the speaker 2 are stored in association with each speaker 2 identified by the identification unit 104, and the selection unit 106. Selects a specific speaker 2 based on the frequency stored in the storage unit 105. Here, the importance level of the speaker 2 is determined based on the frequency at which the speaker 2 utters voice. Thereby, the search means control apparatus 1101 of this embodiment can make the search means 11 search based on the audio | voice information uttered by the speaker 2 (for example, speaker 2A) with high utterance frequency, for example. Further, the search means control apparatus 1101 of the present embodiment can cause the search means 11 to search based on voice information uttered by a speaker 2 (for example, a speaker 2D) with a low utterance frequency, for example. For example, the search means control device 1101 provides the speaker 2 with information for deepening the topic of the conference by causing the search means 11 to perform a search based on voice information uttered by the speaker 2A with a high utterance frequency. be able to. Further, for example, the search unit 11 is made to search based on voice information uttered by a speaker 2D having a high utterance frequency, and information for changing or broadening the topic of the conference is provided to the speaker 2. Can do. That is, the search means control apparatus 1101 can provide the speaker 2 with information for smoothly proceeding with the conference.

  In addition, the search unit control apparatus 1101 of the present embodiment includes a result selection unit 108 that selects information indicating the search result output from the search unit 11 based on the feature amount indicating the feature of the voice of the specific speaker 2. ing. Thereby, the search means control apparatus 1101 of this embodiment can select the information which shows a search result with attributes, such as sex and age, as the importance degree of the speaker 2, for example. For example, the search means control apparatus 1101 of this embodiment can select information indicating a search result based on voice information issued by a child or an adult, or a man or a woman. For example, the search means control device 1101 is based on voice information uttered by a child as the speaker 2 when he / she wants to pay attention to a child's speech in a conference where an adult and a child as the speaker 2 are mixed. The information indicating the search result can be selected. That is, the search means control apparatus 1101 can provide the speaker 2 with information for smoothly proceeding with the conference.

  In addition, the search unit control device 1101 of this embodiment includes a setting unit 109 that sets a feature amount based on audio information. Thereby, the search means control apparatus 1101 of this embodiment automatically sets, for example, attributes such as gender and age as the importance level of the speaker 2 according to the timing when the speaker 2 speaks (for example, in real time). can do. That is, the search means control device 1101 can set the importance level of the speaker 2 without requiring the speaker 2 to set. That is, the search means control device 1101 can reduce the amount of work necessary for smoothly proceeding with the conference.

  Further, the output unit 107 of the search unit control apparatus 1101 of this embodiment determines whether or not to output the character information to the search unit 11 based on whether or not the character information includes a predetermined word / phrase. . Here, as described above, the predetermined phrase is, for example, a moving verb such as “That's right” or “He”, or an interrogative word such as “when” or “where”. Thereby, the search means control apparatus 1101 of this embodiment is controlled so that it is not output to the search means 11 when the speech information includes a moving verb or a question word that is uttered by the speaker 2 during the meeting. be able to. That is, the search means control device 1101 can reduce the search for information unnecessary for the speaker 2.

[Eleventh embodiment]
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. Note that the description of the same configuration and operation as those of the first embodiment described above will be omitted.
FIG. 16 is a block diagram showing the configuration of the search result output device 130 in this embodiment.
The search result output device 130 includes the search means 11 and the search means control device 1101 described above.
The search unit 11 stores character information and related information related to the character information in association with each other in advance, and outputs the related information related to the input character information as information indicating the search result. The search means 11 of the present embodiment, for example, includes a database 111 in which character information and related information related to character information are stored in association with each other, and related information related to input character information. And a search unit 112 that outputs information indicating the search result. That is, the search result output device 130 of this embodiment includes the search means 11 described in the first embodiment inside the search result output device 130.

  As a result, the search result output device 130 of the present embodiment does not require the network 15 described in the first embodiment, and therefore can be used even in a place where the network 15 is not installed. In addition, since the search result output device 130 according to the present embodiment does not cause a delay in communication through the network 15, the search result output device 130 searches for the time from when the speaker 2 speaks until the search result is provided via the network 15. It can be shortened compared to

In the description of each embodiment described above, the search unit control device 1101 and the search result output device 130 display information on the display unit 200A embedded in the top plate portion of the desk 3, but the present invention is not limited thereto. Absent. For example, the display unit 200A may be installed on a wall, or the hemispherical display unit 200 may be installed on the desk 3 as shown in FIG.
FIG. 17 is a configuration diagram illustrating an example of the conference system 1 that outputs to the display unit 200 configured in a hemispherical shape.
The conference system 1 includes a display unit 200 configured on a desk 3 in a hemispherical shape. The search means control device 1101 and the search result output device 130 cause the display unit 200 to display an image indicating the search result. In this case, the image displayed on the display unit 200 is visually recognized only by the specific speaker 2. For example, the search means control device 1101 and the search result output device 130 are based on voice information uttered by a specific speaker 2 seated in a predetermined direction (for example, the speaker 2A in FIG. 17). Specific character information can be selected from the character information. Thereby, for example, the search means control device 1101 and the search result output device 130 request the speaker 2A to search and display specific information (for example, conference material created by the speaker 2A itself). When speaking, specific information can be displayed so that only the speaker 2A can visually recognize it.

  In each of the above-described embodiments, the search unit control device 1101 and the search result output device 130 have been described with the configuration including the four sound collection units 101 as an example, but the invention is not limited thereto. The search means control device 1101 and the search result output device 130 only need to be able to detect the sound around the sound collection unit 101. For example, the search means control device 1101 and the search result output device 130 may include a single or multiple sound collection units 101 in the lower part of the desk 3, for example. Good. Thereby, the search means control device 1101 and the search result output device 130 can include, for example, the sound collecting units 101 having the position and the number corresponding to the size of the desk 3.

  As mentioned above, although embodiment of this invention has been explained in full detail with reference to drawings, a concrete structure is not restricted to this embodiment and can be suitably changed in the range which does not deviate from the meaning of this invention. .

  Note that the generation unit 102, the conversion unit 103, the identification unit 104, the selection unit 106, the output unit 107, the result selection unit 108, the setting unit 109, or the result output unit 110 in the above embodiment (hereinafter collectively referred to as control) Each unit included in the control unit CONT may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

  Next, in the twelfth to sixteenth embodiments, an information display device (search control device) that is an embodiment of the present invention will be described. In the following description, details of a process for specifying the content of a utterance made by a speaker will be described.

Hereinafter, a search control device and a search device (output device) according to an embodiment of the present invention will be described with reference to the drawings.
[Twelfth embodiment]
FIG. 18 is a first diagram illustrating an example of a system configuration of the search device (information display device 1200) according to the present embodiment.
In FIG. 18, a search device (information display device 1200) is connected to the search means 11 via a network 15 (for example, the Internet). The search device (information display device 1200) is connected to the sound collection unit (sound collection microphone) 101. Thereby, voice information (for example, voice signal of conversation) of the user (for example, two users (U1, U2)) is input from the sound collection unit 101 to the search device (information display device 1200).

The sound collection unit 101 collects the voice of the user's conversation and converts it into a voice signal (voice information). The sound collection unit 101 outputs a voice signal (voice information) obtained by converting the user's voice to the search device (information display device 1200) as first information.
The search unit 11 searches for information (second information) such as an image, sound, and text related to the keyword based on a keyword (search word) or a search expression based on the keyword. For example, the search unit 11 may search for information (second information) as a search result via the network 15, or information as a search result from information stored in a database provided in the own device. You may search (2nd information).

FIG. 19 is a block diagram of the search device (information display device 1200) according to this embodiment.
In FIG. 19, the search device (information display device 1200) (output device) includes a control unit 1102 (search processing control unit) and a display unit 200. In the present embodiment, the control unit 1102 functions as a search control device (display control device).
The control unit 1102 controls each unit of the search device (information display device 1200). Based on the voice information (first information) input from the sound collection unit 101, the control unit 1102 causes the search unit 11 to search via the network 15, and information such as an image (second information) as a search result. To get. Further, the control unit 1102 causes the display unit 200 to display the search result acquired from the search unit 11.
The control unit 1102 includes a voice recognition unit 1111 and a search control unit 1112.

  The speech recognition unit 1111 (extraction unit) extracts words included in the speech information (first information) input from the sound collection unit 101. That is, the speech recognition unit 1111 recognizes the input speech information, converts it into text information (sentence), and extracts words included in the text information. The voice recognition unit 1111 supplies the extracted word to the search control unit 1112.

The search control unit 1112 (control unit) causes the search unit 11 to search for information such as an image output as a search result based on the words extracted by the speech recognition unit 1111. That is, the search control unit 1112 causes the search unit 11 to search for the second information based on the search condition based on the word extracted by the voice recognition unit 1111. Further, the search control unit 1112 generates a search expression based on keywords (for example, nouns, proper nouns, etc.) based on the words extracted by the speech recognition unit 1111, and information such as images based on the generated search expressions. To the search means 11. For example, the search control unit 1112 generates a search expression in which keywords of the words included in the first information are sequentially added as a search condition, and searches for information such as an image based on the generated search expression. To search. That is, the search control unit 1112 supplies the generated search formula to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15. Note that the above-described keyword (search word) may be a character string (for example, an adjective + noun) or a sentence combining words.
Here, the search control unit 1112 has the following two search modes.

  The topic provision mode (first search mode) is a search mode in which search conditions are expanded (expanded), and is a search mode having a wider search condition than a conversation tracking mode described later. Topic offer mode, for example, information that provides a topic in the user's conversation information such as images searched for the content that the user is currently interested in, using the words included in the conversation by the user such as a conference as a keyword As a search mode. This topic providing mode is an effective search mode for brainstorming in a conference. In the topic providing mode, the search control unit 1112 combines the keywords included in the newly input voice information (first information) with a logical sum (OR: OR) and adds it to the search formula.

  For example, in the topic providing mode, the search control unit 1112 generates a search formula as shown in the following formula (1).

Search formula = KEY1 or KEY2 (1)
Here, KEY1 and KEY2 are symbols indicating keywords.

  When the speech recognition unit 1111 newly extracts a keyword (KEY3) in the state of the search formula of the formula (1), the search control unit 1112 adds the keyword (KEY3) to the search shown in the following formula (2) Generate an expression.

  Search formula = KEY1 or KEY2 or KEY3 (2)

  The conversation tracking mode (second search mode) is a search mode in which search conditions are narrowed (reduced), and is a search mode with narrower search conditions than the topic provision mode. The conversation tracking mode is a search mode in which information such as images searched by narrowing down the content that the user wants to know is used as a keyword for words included in conversations by users such as meetings, for example. This conversation tracking mode is a search mode that is effective in the work of gathering opinions and information in a meeting. In the conversation tracking mode, the search control unit 1112 combines the keywords included in the newly input voice information (first information) by AND (AND) and adds it to the search formula.

  For example, in the conversation tracking mode, the search control unit 1112 generates a search formula as shown in the following formula (3).

Search formula = KEY1 and KEY2 (3)
Here, KEY1 and KEY2 are symbols indicating keywords.

  When the voice recognition unit 1111 newly extracts a keyword (KEY3) in the state of the search formula of the formula (3), the search control unit 1112 adds the keyword (KEY3) to the search shown in the following formula (4). Generate an expression.

  Search formula = KEY1 and KEY2 and KEY3 (4)

In addition, when the word extracted by the speech recognition unit 1111 includes a predetermined specified word (predetermined word), the search control unit 1112 determines the topic providing mode and the conversation tracking mode according to the specified word. And switch. For example, when the search control unit 1112 is interested in information such as an image (image group) displayed on the display unit 200, the search control unit 1112 shifts from the topic providing mode to the conversation tracking mode. That is, the search control unit 1112 shifts from the topic providing mode to the conversation tracking mode when the word extracted by the speech recognition unit 1111 includes, for example, pronouns such as “what” and “this”.
Further, for example, when the user is satisfied with the search result, the search control unit 1112 shifts from the conversation tracking mode to the topic providing mode. That is, the search control unit 1112 includes a specified word that is found to be satisfied in the word extracted by the speech recognition unit 1111, for example, a noun or pronoun such as “OK”, “around”, “it”, and the like. In this case, the conversation tracking mode is changed to the topic providing mode.

  Note that the search control unit 1112 may acquire a lot of unnecessary search results because there is too much information such as images as search results in the topic providing mode. Therefore, in the present embodiment, the search control unit 1112 excludes the keyword from the search formula when the number of search results is equal to or more than a first predetermined number in the topic providing mode, and searches the search unit. 11 is searched again. In this case, for example, the search control unit 1112 excludes keywords from the search formula in the oldest order (the oldest input order) extracted by the voice recognition unit 1111. Here, excluding the above keywords from the search expression means to delete (exclude) some keywords from the keywords included in the search expression.

  In addition, the search control unit 1112 may not obtain a required search result in the conversation tracking mode. Therefore, in the present embodiment, the search control unit 1112 excludes the keyword from the search formula when the number of search results is equal to or less than a predetermined second specified number in the conversation tracking mode, and searches the search unit. 11 is searched again. In this case, for example, the search control unit 1112 excludes keywords from the search formula in the oldest order (the oldest input order) extracted by the voice recognition unit 1111.

  As described above, the search control unit 1112 selectively switches between the topic providing mode and the conversation tracking mode based on the words extracted by the voice recognition unit 1111 and generates a search expression. The search control unit 1112 supplies the generated search expression to the search unit 11 and acquires information such as an image as a search result from the search unit 11 via the network 15. The search control unit 1112 supplies the search result acquired from the search unit 11 to the display unit 200 and causes the display unit 200 to display the search result.

  The display unit 200 is a display device such as a liquid crystal display. The display unit 200 displays information (second information) such as an image supplied from the search control unit 1112.

Next, the operation of the search device (information display device 1200) in the present embodiment will be described.
FIG. 20 is a flowchart illustrating an example of processing in the topic provision mode in the present embodiment.
In this figure, first, the control unit 1102 determines whether or not there is an audio input from the sound collection unit 101 (step S101). That is, the control unit 1102 determines whether audio information is input from the sound collection unit 101 to the audio recognition unit 1111. When it is determined that voice information has been input (step S101: YES), the control unit 1102 advances the process to step S102. When it is determined that no voice information is input (step S101: NO), the control unit 1102 returns the process to step S101 and repeats the process of step S101 until the voice information is input.

  Next, the control unit 1102 performs speech recognition and morphological analysis (step S102). That is, the voice recognition unit 1111 of the control unit 1102 recognizes the input voice information, converts it into text information (sentence), and extracts words included in the text information. The voice recognition unit 1111 supplies the extracted word to the search control unit 1112.

  Next, the control unit 1102 executes search word (keyword) addition and search processing (step S103). That is, the search control unit 1112 of the control unit 1102 combines the keywords included in the newly input voice information (first information) with a logical sum (OR), and adds them to the search formula. The search control unit 1112 supplies the generated search expression to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15.

  Next, the control unit 1102 displays the search result on the display unit 200 (step S104). That is, the search control unit 1112 of the control unit 1102 supplies information such as an image acquired as a search result to the display unit 200 for display. Accordingly, the display unit 200 displays information such as an image that is a search result supplied from the search control unit 1112.

  Next, the control unit 1102 determines whether or not a specified word is included in the input voice information (step S105). That is, the search control unit 1112 of the control unit 1102 determines whether or not a prescribed word (predetermined word) is included in the words extracted by the speech recognition unit 1111. Here, as described above, the specified word is a pronoun such as “what” or “this”. When it is determined that the specified word (predetermined word) is included (step S105: YES), the search control unit 1112 shifts the processing from the topic providing mode to the conversation tracking mode (step S111 in FIG. 21). If the search control unit 1112 determines that the specified word (predetermined word) is included (step S105: NO), the search control unit 1112 advances the process to step S106.

  Next, in step S106, the control unit 1102 determines whether or not the search result is equal to or greater than a specified number (first specified number). That is, the search control unit 1112 of the control unit 1102 determines whether or not the number of search results (second information) acquired from the search unit 11 is equal to or greater than a specified number (first specified number). If the search control unit 1112 determines that the number of search results is greater than or equal to the specified number (first specified number) (step S106: YES), the search control unit 1112 advances the process to step S107. In addition, when the search control unit 1112 determines that the number of search results is less than the specified number (first specified number) (step S106: NO), the search control unit 1112 returns the process to step S101.

  Next, in step S107, the control unit 1102 deletes the word contained in the old sentence (voice information), and returns the process to step S101. That is, the search control unit 1112 of the control unit 1102 excludes keywords from the search formula in the oldest order (the oldest input order) extracted by the speech recognition unit 1111 and returns the process to step S101.

  As described above, in the topic providing mode according to the present embodiment, the control unit 1102 combines the keywords included in the newly input voice information (first information) with a logical sum (OR) and adds the keywords to the search formula. Then, the search unit 11 is made to search for information such as an image as a search result. Then, the control unit 1102 displays the search result on the display unit 200, and excludes the keyword from the search formula when the number of search results is equal to or more than a predetermined first predetermined number, and searches the search unit. 11 is searched again.

FIG. 21 is a flowchart showing an example of processing in the conversation tracking mode in the present embodiment.
In this figure, first, the control unit 1102 determines whether or not there is an audio input from the sound collection unit 101 (step S111). That is, the control unit 1102 determines whether audio information is input from the sound collection unit 101 to the audio recognition unit 1111. When it is determined that voice information has been input (step S111: YES), the control unit 1102 advances the process to step S112. When it is determined that the voice information is not input (step S111: NO), the control unit 1102 returns the process to step S111 and repeats the process of step S111 until the voice information is input.

  Next, the control unit 1102 performs voice recognition and morphological analysis (step S112). That is, the voice recognition unit 1111 of the control unit 1102 recognizes the input voice information, converts it into text information (sentence), and extracts words included in the text information. The voice recognition unit 1111 supplies the extracted word to the search control unit 1112.

  Next, the control unit 1102 executes search word (keyword) addition and search processing (step S113). That is, the search control unit 1112 of the control unit 1102 combines the keywords included in the newly input voice information (first information) by logical product (AND) and adds them to the search formula. The search control unit 1112 supplies the generated search expression to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15.

  Next, the control unit 1102 displays the search result on the display unit 200 (step S114). That is, the search control unit 1112 of the control unit 1102 supplies information such as an image acquired as a search result to the display unit 200 for display. Accordingly, the display unit 200 displays information such as an image that is a search result supplied from the search control unit 1112.

  Next, the control unit 1102 determines whether or not a specified word is included in the input voice information (step S115). That is, the search control unit 1112 of the control unit 1102 determines whether or not a prescribed word (predetermined word) is included in the words extracted by the speech recognition unit 1111. Here, the specified word is a noun or pronoun such as “OK”, “around”, “it”, etc., as described above. When it is determined that the specified word (predetermined word) is included (step S115: YES), the search control unit 1112 shifts the processing from the conversation tracking mode to the topic providing mode (step S101 in FIG. 20). If the search control unit 1112 determines that the specified word (predetermined word) is included (step S115: NO), the search control unit 1112 advances the process to step S116.

  Next, in step S116, the control unit 1102 determines whether or not the search result is equal to or less than a specified number (second specified number). That is, the search control unit 1112 of the control unit 1102 determines whether or not the number of search results acquired from the search unit 11 is equal to or less than a specified number (second specified number). If the search control unit 1112 determines that the number of search results is equal to or less than the specified number (first specified number) (step S116: YES), the search control unit 1112 advances the process to step S117. If the search control unit 1112 determines that the number of search results exceeds the specified number (second specified number) (step S116: NO), the search control unit 1112 returns the process to step S111. In the present embodiment, the first specified number and the second specified number described above have a relationship of (first specified number> second specified number), and the first specified number and the second specified number The specified number is determined in advance so as to maintain an appropriate number of search results.

  Next, in step S117, the control unit 1102 deletes the word included in the old sentence (speech information), and returns the process to step S111. That is, the search control unit 1112 of the control unit 1102 excludes keywords from the search formula in the oldest order (the oldest input order) extracted by the speech recognition unit 1111 and returns the process to step S111.

  As described above, in the conversation tracking mode according to the present embodiment, the control unit 1102 combines the keywords included in the newly input voice information (first information) with the logical product (AND) and adds them to the search expression. Then, the search unit 11 is made to search for information such as an image as a search result. Then, the control unit 1102 causes the display unit 200 to display the search results, and excludes the keyword from the search formula when the number of search results is equal to or less than a predetermined second predetermined number, and searches the search unit. 11 is searched again.

  As described above, the control unit 1102 (search control device) in the present embodiment includes the voice recognition unit 1111 and the search control unit 1112. The voice recognition unit 1111 extracts words included in the input voice information (first information). The search control unit 1112 causes the search unit 11 to search for information such as an image output as a search result based on a search condition based on the word extracted by the speech recognition unit 1111. The search control unit 1112 has a topic providing mode in which the search condition is widened and a conversation tracking mode in which the search condition is narrowed.

  Accordingly, the search device (information display device 1200) in the present embodiment displays, on the display unit 200, a search result obtained by searching for the content that the user is currently interested in, for example, from the user's conversation in the topic providing mode. indicate. Then, in the conversation tracking mode, the search device (information display device 1200) in the present embodiment displays the searched search result on the display unit 200 so as to narrow down the content that the user wants to know. Therefore, the control unit 1102 and the search device (information display device 1200) according to the present embodiment can provide information narrowed down to contents that the user wants to know while providing a wide range of information to the user. For example, the control unit 1102 and the search device (information display device 1200) in the present embodiment are not fixed search conditions intentionally selected by the user, but by search conditions extracted from the user's natural conversation. While providing a wide variety of topics, it is possible to provide information that delves into topics that the user is interested in. Therefore, the control unit 1102 and the search device (information display device 1200) in the present embodiment can appropriately provide useful information to the user.

Further, in the present embodiment, the search control unit 1112, when the word extracted by the voice recognition unit 1111 includes a predetermined word (prescribed word), the topic is determined according to the predetermined word. Switch between provision mode and conversation tracking mode.
As a result, the two search modes, the topic providing mode and the conversation tracking mode, can be appropriately switched. Therefore, the control unit 1102 and the search device (information display device 1200) in this embodiment can provide information useful to the user. Can be provided appropriately.

In the present embodiment, the search control unit 1112 generates a search expression in which keywords (search words) of the words extracted by the speech recognition unit 1111 are sequentially added as a search condition, and is based on the generated search expression. Thus, the search means 11 is made to search for information such as images.
Thereby, it becomes possible to widen the search condition and narrow the search condition by sequentially inputting the voice information. Therefore, the control unit 1102 and the search device (information display device 1200) in the present embodiment can make the search unit 11 appropriately search for the topic providing mode and the conversation tracking mode.

In this embodiment, the search control unit 1112 combines keywords by logical sum (OR) in the topic providing mode and adds them to the search formula, and combines keywords by logical product (AND) in the conversation tracking mode. To add to the search expression.
As a result, the control unit 1102 and the search device (information display device 1200) in the present embodiment can appropriately generate search formulas in two search modes by simple means.

In this embodiment, the search control unit 1112 excludes keywords from the search formula in the oldest order extracted by the speech recognition unit 1111.
Thereby, since the control part 1102 and the search device (information display device 1200) in this embodiment delete an old keyword from a search formula, it can make the search means 11 search with an appropriate search condition.

In the present embodiment, the search control unit 1112 excludes the keyword from the search formula when the number of search results is equal to or more than a first predetermined number in the topic providing mode.
Thereby, the number of search results is reduced. For this reason, the control unit 1102 and the search device (information display device 1200) in the present embodiment are in a state of providing unnecessary search results to the user due to excessive information such as images as search results. Can be reduced.

In the present embodiment, the search control unit 1112 excludes the keyword from the search formula when the number of search results is equal to or less than a predetermined second specified number in the conversation tracking mode.
This increases the number of search results. Therefore, the control unit 1102 and the search device (information display device 1200) in the present embodiment can reduce the situation where the user cannot provide the user with the necessary search results due to excessive narrowing down.

In the present embodiment, the voice recognition unit 1111 recognizes input voice information and converts it into text information, and extracts a word included in the text information.
Thereby, the control unit 1102 and the search device (information display device 1200) in the present embodiment extract words from the audio information input from the sound collection unit 101, and information (such as images) according to the search conditions based on the extracted words ( Second information) can be retrieved. Therefore, the control unit 1102 and the search device (information display device 1200) according to the present embodiment search for and provide appropriate information (second information) from natural conversations that the user is not conscious of. Can do.

Further, the search device (information display device 1200) (output device) in the present embodiment includes a control unit 1102 and a display unit 200 that outputs information such as an image searched by the control unit 1102.
Thereby, the search device (information display device 1200) in this embodiment can appropriately provide information such as the searched image to the user.

In the above-described embodiment, an example in which the search control unit 1112 switches between the topic providing mode and the conversation tracking mode depending on whether or not the specified word (predetermined word) is included has been described. It is also possible to switch between these two modes.
Next, a modified example for switching between the two modes will be described.

<First Modification>
In the first modification, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode according to a predetermined time. Here, the predetermined time is, for example, two-thirds (2/3) of the scheduled conference period, a time ten minutes before the scheduled end time, a predetermined elapsed time from the start of the conference, or the like.

FIG. 22 is a flowchart showing a first modification of the topic providing mode in the present embodiment.
The topic providing mode process in this figure is the same as the topic providing mode process shown in FIG. 20 except that the process in step S105a is different. Here, the process of step S105a will be described.

In step S105a, the search control unit 1112 of the control unit 1102 determines whether a predetermined time has elapsed. Here, as described above, the predetermined time is, for example, two-thirds (2/3) of the scheduled conference period. When it is determined that the predetermined time has elapsed (step S105a: YES), the search control unit 1112 shifts the processing from the topic providing mode to the conversation tracking mode. If the search control unit 1112 determines that the predetermined time has not elapsed (step S105a: NO), the search control unit 1112 advances the process to step S106.
Although not shown, the conversation tracking mode may be switched from the conversation tracking mode to the topic providing mode according to a predetermined time in the conversation tracking mode as in FIG.

As described above, the search control unit 1112 switches between the topic provision mode and the conversation tracking mode according to a predetermined time.
As a result, the two search modes, the topic providing mode and the conversation tracking mode, can be appropriately switched. Therefore, the control unit 1102 and the search device (information display device 1200) in this embodiment can provide information useful to the user. Can be provided appropriately. For example, in the conference, the topic providing mode and the conversation tracking mode are switched according to a predetermined time, so that the control unit 1102 and the search device (information display device 1200) in the present embodiment are useful in accordance with the progress of the conference. Information can be provided.

<Second Modification>
In the second modification, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode when the keyword extracted by the speech recognition unit 1111 includes a keyword included in the search expression. That is, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode when the same keyword (search word) appears in the voice information (for example, voice information in conversation).

FIG. 23 is a flowchart showing a first modification of the topic providing mode in the present embodiment.
The topic providing mode process in this figure is the same as the topic providing mode process shown in FIG. 20 except that the process in step S105b is different. Here, the process of step S105b will be described.

  In step S105b, the control unit 1102 determines whether the same search word (keyword) is included in the input voice information. That is, the search control unit 1112 of the control unit 1102 determines whether or not a keyword included in the search formula is included in the words extracted by the speech recognition unit 1111. When it is determined that the extracted word includes a keyword included in the search expression (step S105b: YES), the search control unit 1112 shifts the processing from the topic providing mode to the conversation tracking mode. If the search control unit 1112 determines that the extracted word does not include the keyword included in the search expression (step S105b: NO), the search control unit 1112 advances the process to step S106.

As described above, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode when the keyword extracted by the speech recognition unit 1111 includes a keyword included in the search expression.
As a result, the two search modes, the topic providing mode and the conversation tracking mode, can be appropriately switched. Therefore, the control unit 1102 and the search device (information display device 1200) in this embodiment can provide information useful to the user. Can be provided appropriately.

<Third Modification>
In the third modification, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode when a specific speaker is speaking.
For example, the voice recognition unit 1111 identifies the speaker based on the voice information input from the sound collection unit 101, associates the information indicating the speaker with the extracted word, and supplies the information to the search control unit 1112. Note that the voice recognition unit 1111 may identify the speaker based on the feature amount in voice information such as a voiceprint, or may identify the speaker from the direction of voice based on the voice information input from the plurality of sound pickup units 101. May be.
For example, when the speaker is a person who brings together a meeting like a chairperson in the meeting, the search control unit 1112 determines that the meeting has shifted to a summary, and switches from the topic providing mode to the conversation tracking mode. The search control unit 1112 switches the topic providing mode from the conversation tracking mode when, for example, the speaker is a person in a position to give a reference opinion or a person in a position to give various opinions.

In this way, the search control unit 1112 switches between the topic providing mode and the conversation tracking mode when a specific speaker is speaking.
As a result, the two search modes, the topic providing mode and the conversation tracking mode, can be appropriately switched. Therefore, the control unit 1102 and the search device (information display device 1200) in this embodiment can provide information useful to the user. Can be provided appropriately.

  In this embodiment, when switching between the topic providing mode and the conversation tracking mode, for example, the search control unit 1112 temporarily deletes the search expression and uses the most recent keyword or the most recent predetermined number of keywords. A new search expression may be generated. For example, when the topic providing mode is switched to the conversation tracking mode, the search control unit 1112 may generate a search expression by combining the most recent predetermined number of keywords with a logical product (AND). In addition, for example, when the conversation tracking mode is switched to the topic providing mode, the search control unit 1112 may generate a search expression by combining a predetermined number of keywords with a logical sum (OR). Good.

Further, the search control unit 1112 may change the operator of only the most recent predetermined number of keywords when switching between the topic providing mode and the conversation tracking mode. For example, when the topic providing mode is switched to the conversation tracking mode, the search control unit 1112 combines the most recent predetermined number of keywords by logical product (AND), and the rest remains the logical sum (OR). The search expression may be generated by. Further, for example, when the conversation tracking mode is switched to the topic providing mode, the search control unit 1112 combines the most recent predetermined number of keywords with a logical sum (OR), and the rest is logically ANDed. The search expression may be generated as it is.
The search control unit 1112 may maintain the search formula as it is when switching between the topic providing mode and the conversation tracking mode, and generate the search formula by changing the operator from the newly added keyword.

Next, a search control device and a search device (output device) according to a second embodiment will be described with reference to the drawings.
[Thirteenth embodiment]
The system configuration of the search device (information display device 1200) in the thirteenth embodiment is the same as the system configuration of the search device (information display device 1200) in the twelfth embodiment shown in FIG.
FIG. 24 is a block diagram of a search device (information display device 1200) according to the thirteenth embodiment.
In FIG. 24, a search device (information display device 1200) (output device) includes a control unit 1102a and a display unit 200. In the present embodiment, the control unit 1102a functions as a search control device.
In this figure, the same components as those in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted.

The control unit 1102a controls each unit of the search device (information display device 1200). Based on the voice information (first information) input from the sound collection unit 101, the control unit 1102a causes the search unit 11 to search via the network 15, and acquires information such as an image as a search result. In addition, the control unit 1102 a causes the display unit 200 to display information such as an image acquired from the search unit 11. In the present embodiment, when the input voice information (sentence) is a negative sentence, the control unit 1102a determines that the user does not want a word included in the sentence, and starts from the previous keyword. Search again without the words contained in the negative sentence.
In addition, the control unit 1102 a includes a voice recognition unit 1111, a search control unit 1112, and a negative sentence detection unit 1113.

  The negative sentence detection unit 1113 (detection unit) detects a negative sentence included in the voice information input from the sound collection unit 101. Here, the negative sentence is a sentence including a word indicating negative. Also, the words indicating negative are “not”, “not”, “no”, “no”, and the like. For example, the negative sentence detection unit 1113 acquires from the speech recognition unit 1111 the text information that the speech recognition unit 1111 has recognized the speech information and the words extracted by the speech recognition unit 1111. The negative sentence detection unit 1113 detects a negative sentence based on the text information acquired from the voice recognition unit 1111 and the word. For example, the negative sentence detection unit 1113 extracts a sentence based on text information, and detects a negative sentence based on whether or not a word indicating negative is included in the extracted sentence. The negative sentence detection unit 1113 supplies the detected negative sentence and the words included in the negative sentence to the search control unit 1112.

When a negative sentence is detected by the negative sentence detection unit 1113, the search control unit 1112 excludes a keyword included in the negative sentence from the search formula. For example, the search control unit 1112 determines whether there is a negative sentence supplied from the negative sentence detection unit 1113. Furthermore, when it is determined that there is a negative sentence, the search control unit 1112 determines whether or not the word supplied from the negative sentence detection unit 1113 is in the search formula. Then, the search control unit 1112 deletes the word determined to be in the search formula from the search formula. In addition, when the search control unit 1112 determines that there is a negative sentence, the keyword included in the newly input voice information (first information) is ORed (OR) or according to two search modes. Combined by AND (AND) and added to the search expression.
Since other processing in the search control unit 1112 is the same as that in the twelfth embodiment, description thereof is omitted here.

Next, the operation of the search device (information display device 1200) in the present embodiment will be described.
FIG. 25 is a flowchart illustrating an example of processing in the topic provision mode in the present embodiment.
In this figure, in this embodiment, the point that the control unit 1102a performs a determination process using a negative sentence is different from the process in the topic provision mode in the twelfth embodiment shown in FIG.
In this figure, first, the control unit 1102a determines whether or not there is an audio input from the sound collection unit 101 (step S201). That is, the control unit 1102a determines whether or not voice information is input from the sound collection unit 101 to the voice recognition unit 1111. When it is determined that the audio information has been input (step S201: YES), the control unit 1102a advances the process to step S202. When it is determined that the voice information is not input (step S201: NO), the control unit 1102a returns the process to step S201 and repeats the process of step S201 until the voice information is input.

  Next, the control unit 1102a performs voice recognition and morphological analysis (step S202). That is, the voice recognition unit 1111 of the control unit 1102a recognizes input voice information, converts it into text information (sentence), and extracts words included in the text information. The voice recognition unit 1111 supplies the extracted word to the search control unit 1112. The voice recognition unit 1111 supplies the converted text information (sentence) and the extracted word to the negative sentence detection unit 1113.

  Next, the control unit 1102a determines whether there is a negative sentence (step S203). First, the negative sentence detection part 1113 of the control part 1102a detects a negative sentence based on the text information acquired from the speech recognition part 1111 and the word. The negative sentence detection unit 1113 supplies the detected negative sentence and the words included in the negative sentence to the search control unit 1112. The search control unit 1112 determines whether there is a negative sentence supplied from the negative sentence detection unit 1113. When it is determined that there is a negative sentence (step S203: YES), the search control unit 1112 advances the process to step S204. If the search control unit 1112 determines that there is no negative statement (step S203: NO), the search control unit 1112 advances the process to step S205.

  Next, in step S204, the control unit 1102a executes search word (keyword) deletion and search processing. That is, the search control unit 1112 of the control unit 1102a deletes the word determined to be in the search formula from the search formula. The search control unit 1112 supplies the generated search expression to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15.

  In step S205, the control unit 1102a executes search word (keyword) addition and search processing. That is, the search control unit 1112 of the control unit 1102a combines the keywords included in the newly input voice information (first information) with a logical sum (OR) and adds it to the search formula. The search control unit 1112 supplies the generated search expression to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15.

The processing from the next step S206 to step S209 is the same as the processing from step S104 to step S107 shown in FIG. 20, and a description thereof will be omitted here.
Although not shown, also in the conversation tracking mode process, the control unit 1102a executes a determination process using a negative sentence as in Steps S203 and S204 of FIG.

As described above, the control unit 1102a according to the present embodiment detects a negative sentence that is a negative sentence that is a sentence including a word indicating negative and is included in the voice information (first information). Part 1113 is provided. When a negative sentence is detected by the negative sentence detection unit 1113, the search control unit 1112 excludes a keyword included in the negative sentence from the search formula.
Thereby, the control unit 1102a and the search device (information display device 1200) in the present embodiment can remove the keyword (search word) from the search formula. For example, the control unit 1102a and the search device (information display device 1200) in the present embodiment can remove an incorrect keyword (search word) from the previous search formula when a recognition error occurs by the speech recognition unit 1111. it can. Therefore, since the control unit 1102a and the search device (information display device 1200) in the present embodiment can perform a search appropriate for the user, it is possible to appropriately provide useful information to the user.

Next, a search control device and a search device (output device) according to a fourteenth embodiment will be described with reference to the drawings.
[Fourteenth embodiment]
FIG. 26 is a second diagram illustrating an example of the configuration of the search device (information display device 1200) according to the present embodiment.
In FIG. 26, the search device (information display device 1200) is connected to the search means 11 and the synonym search means 12 via the network 15 (for example, the Internet). The search device (information display device 1200) is connected to the sound collection unit 101. Thereby, voice information (for example, voice signal of conversation) of the user (for example, two users (U1, U2)) is input from the sound collection unit 101 to the search device (information display device 1200).
In this figure, the same components as those in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted. Note that the present embodiment is different from the twelfth and thirteenth embodiments in that it is connected to the synonym search means 12.

  The synonym search unit 12 searches for synonyms that are related words related to the input word based on the word input via the network 15, and outputs the searched synonym via the network 15. The synonyms are, for example, “Apple”, “Apple” and the like for the word “apple”. For example, the synonym search unit 12 may search for a synonym via the network 15 or may search for a synonym from information stored in a database (for example, a synonym dictionary) provided in its own device.

FIG. 27 is a block diagram of a search device (information display device 1200) according to the fourteenth embodiment.
In FIG. 27, the search device (information display device 1200) (output device) includes a control unit 1102b and a display unit 200. In the present embodiment, the control unit 1102b functions as a search control device.
In this figure, the same components as those in FIG. 24 are denoted by the same reference numerals, and the description thereof is omitted.

The control unit 1102b controls each unit of the search device (information display device 1200). Based on the audio information (first information) input from the sound collection unit 101, the control unit 1102b causes the search unit 11 to search via the network 15, and acquires information such as an image as a search result. In addition, the control unit 1102 b causes the display unit 200 to display information such as an image acquired from the search unit 11. Further, in the present embodiment, the control unit 1102b acquires a synonym related to the word included in the speech information (first information) from the synonym search means 12, and based on the acquired synonym, an image or the like is obtained as a search result. Get information.
In addition, the control unit 1102 b includes a voice recognition unit 1111, a search control unit 1112, a negative sentence detection unit 1113, and a synonym acquisition unit 1114.

  The synonym acquisition unit 1114 (acquisition unit) acquires a synonym (related word) related to the word based on the word extracted by the speech recognition unit 1111. That is, the synonym acquisition unit 1114 supplies the word supplied from the search control unit 1112 to the synonym search unit 12 via the network 15 and causes the synonym search unit 12 to search for a synonym. Then, the synonym acquisition unit 1114 supplies the synonym supplied from the synonym search unit 12 via the network 15 to the search control unit 1112.

The search control unit 1112 acquires a synonym from the synonym acquisition unit 1114 based on the word extracted by the voice recognition unit 1111, and based on a search condition based on the acquired synonym and the word extracted by the voice recognition unit 1111. The search means 11 is made to search for information such as images. For example, the search control unit 1112 first causes the search unit 11 to search for information such as an image based on a search condition based on the word extracted by the speech recognition unit 1111. Then, the search control unit 1112 is extracted by the synonym and speech recognition unit 1111 when the number of search results based on the words extracted by the speech recognition unit 1111 is equal to or less than a predetermined third specified number. Based on the search condition based on the word, the search means 11 is made to search for information such as an image.
Since other processing in the search control unit 1112 is the same as that in the thirteenth embodiment, description thereof is omitted here.

Next, the operation of the search device (information display device 1200) in the present embodiment will be described.
FIG. 28 is a flowchart illustrating an example of processing in the topic provision mode in the present embodiment.
In this figure, the present embodiment is different from the topic providing mode process in the second embodiment shown in FIG. 25 in that the control unit 1102b performs a search process based on a synonym.

In this figure, the processing from step S201 to step S207 is the same as that in the thirteenth embodiment shown in FIG. Here, the processing after step S208a will be described.
In step S208a, the control unit 1102b determines whether or not the search result is equal to or greater than a specified number (first specified number). That is, the search control unit 1112 of the control unit 1102b determines whether or not the number of search results acquired from the search unit 11 is equal to or greater than a specified number (first specified number). If the search control unit 1112 determines that the number of search results is equal to or greater than the specified number (first specified number) (step S208a: YES), the search control unit 1112 advances the process to step S209. If the search control unit 1112 determines that the number of search results is less than the specified number (first specified number) (step S208a: NO), the search control unit 1112 advances the process to step S210.

  Since the process in step S209 is the same as that in the thirteenth embodiment shown in FIG. 25, the description thereof is omitted here.

  In step S210, the control unit 1102b determines whether or not the search result is equal to or less than a specified number (third specified number). That is, the search control unit 1112 of the control unit 1102b determines whether or not the number of search results acquired from the search unit 11 is equal to or less than a specified number (third specified number). If the search control unit 1112 determines that the number of search results is equal to or greater than the specified number (third specified number) (step S210: YES), the search control unit 1112 advances the process to step S211. If the search control unit 1112 determines that the number of search results exceeds the specified number (third specified number) (step S210: NO), the search control unit 1112 returns the process to step S201.

Next, in step S211, the control unit 1102b searches for synonyms and adds the synonyms as keywords. That is, the search control unit 1112 acquires a synonym from the synonym acquisition unit 1114 based on the keyword (search word) extracted by the voice recognition unit 1111. The search control unit 1112 adds the acquired synonym to the search expression to generate a new search expression, and returns the process to step S201. As a result, the search control unit 1112 causes the search unit 11 to search for information such as an image based on the newly generated search expression.
In addition, when adding the acquired synonym to a search formula in step S211, the search control part 1112 couple | bonds a keyword and the corresponding synonym by a logical sum (OR) and adds it to a search formula, for example.

  Although not shown, also in the conversation tracking mode process, the control unit 1102b executes a process of adding a synonym to the search expression, similarly to steps S210 and S211 of FIG. In addition, in the process of the conversation tracking mode, when the acquired synonym is added to the search expression, the search control unit 1112 combines the keyword and the corresponding synonym by a logical sum (OR), for example, and adds it to the search expression. .

As described above, the control unit 1102b according to the present embodiment includes the synonym acquisition unit 1114 that acquires a synonym (related word) related to a word based on the word extracted by the speech recognition unit 1111. The search control unit 1112 acquires a synonym from the synonym acquisition unit 1114 based on the word extracted by the speech recognition unit 1111, and based on a search condition based on the acquired synonym and the word extracted by the speech recognition unit 1111. Thus, the search means 11 is made to search for information such as images.
Accordingly, information such as an image can be searched based on a search condition including a synonym, so that the control unit 1102b and the search device (information display device 1200) in the present embodiment can perform flexible search. Therefore, the control unit 1102b and the search device (information display device 1200) in the present embodiment can perform appropriate search for the user, and thus can appropriately provide useful information to the user.

In the present embodiment, the search control unit 1112 has the number of search results based on the words (search words) extracted by the speech recognition unit 1111 equal to or less than a predetermined value (third specified number). In this case, the search unit 11 is made to search for information such as an image based on a search condition based on a synonym and a word (search word).
Accordingly, the control unit 1102b and the search device (information display device 1200) in the present embodiment can flexibly adjust the number of search results. Therefore, the control unit 1102b and the search device (information display device 1200) in the present embodiment can appropriately provide useful information to the user.

Note that according to the present embodiment, the control unit 1102b (detection control device) includes the voice recognition unit 1111 and the search control unit 1112. The voice recognition unit 1111 extracts words included in the input voice information. The search control unit 1112 causes the search unit 11 to search for information such as an image output as a search result based on the word extracted by the voice recognition unit 1111. Then, when the number of search results based on the word is equal to or less than a predetermined number, the control unit 1112 is configured to obtain information such as an image based on a synonym (related word) related to the word. To the search means 11.
Accordingly, the control unit 1102b and the search device (information display device 1200) in the present embodiment can flexibly adjust the number of search results. Therefore, the control unit 1102b and the search device (information display device 1200) in the present embodiment can appropriately provide useful information to the user.

Next, a search control device and a search device (output device) according to a fifteenth embodiment will be described with reference to the drawings.
[Fifteenth embodiment]
The system configuration of the search device (information display device 1200) in the fifteenth embodiment is the same as the system configuration of the search device (information display device 1200) in the fourteenth embodiment shown in FIG.
The configuration of the search device (information display device 1200) is the same as the system configuration of the search device (information display device 1200) in the fourteenth embodiment shown in FIG.

In the present embodiment, the search control unit 1112 searches for a keyword excluded from the search formula when the word extracted by the speech recognition unit 1111 includes a word indicating a predetermined revival (resurrection word). Add to Here, the word indicating the predetermined resurrection is, for example, a word such as “Sakaki no Yatsu” or “A little before”. For example, the search control unit 1112 determines whether the word extracted by the voice recognition unit 1111 includes the above-described word indicating resurrection. Further, when the search control unit 1112 determines that the above-described word indicating resurrection is included, the search control unit 1112 adds a keyword excluded from the search formula in the past to the search formula. In this case, for example, the search control unit 1112 reinstates the newly added keywords by adding them to the search formula. When restoring the excluded keyword, the search control unit 1112 combines the restored keyword with a logical sum (OR) or a logical product (AND) and adds it to the search formula according to two search modes. .
Since other processing in the search control unit 1112 is the same as that in the fourteenth embodiment, description thereof is omitted here.

Next, the operation of the search device (information display device 1200) in the present embodiment will be described.
FIG. 29 is a flowchart illustrating an example of processing in the topic provision mode in the present embodiment.
In this figure, the present embodiment is different from the processing in the topic providing mode in the fourteenth embodiment shown in FIG. 28 in that the control unit 1102b performs a process of determining whether or not a word includes a resurrection word.
In this figure, first, the control unit 1102b determines whether or not there is an audio input from the sound collection unit 101 (step S301). That is, the control unit 1102b determines whether or not voice information is input from the sound collection unit 101 to the voice recognition unit 1111. When it is determined that the voice information has been input (step S301: YES), the control unit 1102b advances the process to step S302. When it is determined that the voice information is not input (step S301: NO), the control unit 1102b returns the process to step S301, and repeats the process of step S301 until the voice information is input.

  Next, the control unit 1102b performs voice recognition and morphological analysis (step S302). That is, the voice recognition unit 1111 of the control unit 1102b recognizes the input voice information, converts it into text information (sentence), and extracts words included in the text information. The voice recognition unit 1111 supplies the extracted word to the search control unit 1112. The voice recognition unit 1111 supplies the converted text information (sentence) and the extracted word to the negative sentence detection unit 1113.

  Next, the control unit 1102b determines whether or not the word includes a resurrection word (step S303). First, the search control unit 1112 of the control unit 1102b determines whether or not the word acquired from the speech recognition unit 1111 includes a resurrection word that is a word indicating the above-described resurrection. When it is determined that the resurrection word is included (step S303: YES), the search control unit 1112 advances the process to step S304. If the search control unit 1112 determines that the resurrection word is not included (step S303: NO), the search control unit 1112 advances the process to step S305.

  Next, in step S304, the control unit 1102b executes search word (keyword) restoration and search processing. That is, the search control unit 1112 of the control unit 1102b restores the search formulas in order from the newest one of the keywords excluded from the search formulas in the past. The search control unit 1112 supplies the generated search expression to the search unit 11 via the network 15, and acquires information such as an image as a search result from the search unit 11 via the network 15. Thereafter, the search control unit 1112 advances the process to step S308.

The processing from the next step S305 to step S313 is the same as the processing from step S206 to step S211 shown in FIG. 28, and the description is omitted here.
Although not shown, also in the conversation tracking mode process, the control unit 1102b executes a process for determining whether or not a word includes a resurrection word, as in steps S303 and S304 in FIG.

As described above, the search control unit 1112 of the control unit 1102b in the present embodiment uses the search formula when the word extracted by the speech recognition unit 1111 includes a word indicating a predetermined resurrection. Add the excluded keyword to the search expression.
As a result, keywords (search words) that have been accidentally deleted (excluded) from the search formula can be restored, and the control unit 1102b and the search device (information display device 1200) in the present embodiment appropriately perform search processing. be able to. Since the control unit 1102b and the search device (information display device 1200) in the present embodiment can perform an appropriate search for the user, it is possible to appropriately provide useful information to the user.

Next, a search control device and a search device (output device) according to a sixteenth embodiment will be described with reference to the drawings.
[Sixteenth Embodiment]
FIG. 30 is a block diagram of a search device (information display device 1200) according to the sixteenth embodiment.
In FIG. 30, a search device (information display device 1200) (output device) includes a control unit 1102b, a display unit 200, a search unit 11a, and a synonym search unit 12a. In the present embodiment, the control unit 1102b, the search unit 11a, and the synonym search unit 12a function as the search control device 130B.
In this figure, the same components as those in FIG. 27 are denoted by the same reference numerals, and the description thereof is omitted.

The search control device 130B and the search device (information display device 1200) in the present embodiment are different from the fifteenth embodiment in that they include a search unit 11a and a synonym search unit 12a.
The search unit 11a (search means) has the same function as the search means 11 in the fifteenth embodiment, and based on a keyword (search word) or a search expression based on the keyword, an image, sound, and text related to the keyword Search for information such as For example, the search unit 11a may search for information that becomes a search result via a network, or may search for information that becomes a search result from information stored in a database (not shown) provided therein. Good.

  The synonym search unit 12a has the same function as the synonym search unit 12 in the fifteenth embodiment, and based on the words supplied from the synonym acquisition unit 1114, synonyms that are related words related to the supplied words. The retrieved synonym is output to the synonym acquisition unit 1114. The synonym search unit 12a may search for synonyms via, for example, a network, or may search for synonyms from information stored in a database (not shown) (for example, a synonym dictionary) provided therein.

  Note that the search control device 130B and the search device (information display device 1200) in the present embodiment are the same as the operations in the fifteenth embodiment, and are therefore omitted here.

As described above, the search control device 130B in the present embodiment includes the search unit 11a (search means) that searches for information such as images.
As a result, the search control device 130B according to the present embodiment can appropriately provide useful information to the user even in an environment (offline) where connection to the network is not possible.

The present invention is not limited to the above embodiments, and can be modified without departing from the spirit of the present invention.
For example, in each of the above-described embodiments, the form in which audio information is used as the first information has been described, but the present invention is not limited to this. For example, the embodiments described above can be applied to communication using text information such as chat or email. In this case, since the voice recognition function is unnecessary, each of the above embodiments can be realized with a simple configuration.

  Further, in each of the embodiments described above, the form of searching for information such as an image has been described as an example of a search result. Further, in each of the embodiments described above, the search device (information display device 1200) has been described as including the display unit 200 as an output unit. However, the output unit may be an output device such as a speaker that outputs sound information. Other forms are also possible. In addition, the search device (information display device 1200) may include an interface that outputs search results as an output unit, and may output to an externally connected display unit 200 or a speaker via the interface.

  Further, in each of the above embodiments, a mode has been described in which a plurality of users such as a conference use the search device (information display device 1200). However, the search device (information display device 1200) is used by one user. It may be a form. As described above, the search device (information display device 1200) can appropriately provide useful information even when one person examines or summarizes something.

Further, in each of the above embodiments, the form using logical sum and logical product as the logical operators for generating the search expressions has been described, but other logical operators may be used.
Moreover, the combination with each embodiment or each modification in each said embodiment is not limited above. For example, the fourteenth embodiment has been described in conjunction with the thirteenth embodiment, but may be implemented in combination with the twelfth embodiment. Similarly, the fifteenth embodiment has been described in combination with the fourteenth embodiment, but may be implemented in combination with the twelfth or thirteenth embodiment. Similarly, the sixteenth embodiment has been described in combination with the fourteenth or fifteenth embodiment, but may be implemented in combination with the twelfth or thirteenth embodiment.

In addition, each part with which the control part 1102 (1102a, 1102b) and the search control apparatus 130B in each said embodiment are provided may be implement | achieved by exclusive hardware, and the control part 1102 (1102a, 1102b). ) And the search control device 130B are configured by a memory and a CPU (central processing unit), and a program for realizing the functions of the control unit 1102 (1102a, 1102b) and the search control device 130B is loaded into the memory. The function may be realized by executing as described above.
In addition, the control unit 1102 (1102a, 1102b), the search control device 130B, and the search device (information display device 1200) described above have a computer system therein. The processing steps (processing procedures) of the control unit 1102 (1102a, 1102b), the search control device 130B, and the search device (information display device 1200) described above are stored in a computer-readable recording medium in the form of a program. The above processing may be performed by the computer reading and executing this program.

  In addition, a program for realizing the function of each unit included in the control unit 1102 (1102a, 1102b), the search control device 130B, and the search device (information display device 1200) is recorded on a computer-readable recording medium. The program recorded on the computer may be read into the computer system and executed, whereby processing by each unit included in the control unit 1102 (1102a, 1102b), the search control device 130B, and the search device (information display device 1200) may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  Next, in the seventeenth embodiment to the twenty-first embodiment, an information display device which is an embodiment of the present invention will be described. In the following description, details of the display unit whose display content is controlled by the display control device will be described.

[Seventeenth embodiment]
Embodiments of the present invention will be described below with reference to the drawings. FIG. 31 shows an information display device 1200 according to an embodiment of the present invention. FIG. 32 is an explanatory diagram of operation mode determination by the display control device 1100. FIG. 33 is an explanatory diagram of operation mode display by the display unit 200.

  The information display device 1200 includes a display operation mode (hereinafter referred to as a touch mode) in which an operation is performed on the display by touching an image displayed on the display unit 200, and an external device 1500 connected to the information display device 1200. It is configured to be able to be operated by switching between two operation modes of a movable operation mode (hereinafter referred to as a mouse mode) that can be operated.

The information display device 1200 includes a display input unit 1300 and a display control device 1100.
The display input unit 1300 includes a housing 1311 having a substantially rectangular parallelepiped shape and a display unit 200 disposed on the upper surface of the housing 1311. An image projecting unit 1330, a touch operation detecting unit 1340, and a movement amount detecting unit 1350 are disposed inside the housing 1311.
The display control device 1100 performs overall control of the information display device 1200 and is connected from the display input unit 1300 via the output unit 65.

The display unit 200 includes a plate-like display plate 21 disposed so as to cover substantially the entire top surface of the housing 1311 and a screen 22 formed on the inner surface (the surface facing the inside of the housing 1311). It is configured.
The display board 21 is formed to a predetermined thickness by a material that can transmit visible light and infrared light, such as transparent acrylic or polycarbonate, and can guide light using total reflection in the plate surface direction. Yes.
The screen 22 is a light scattering layer formed on the inner surface of the display panel 21 so as to diffusely reflect the irradiated visible light.
Then, the display unit 200 scatters image light projected from an image projection unit 1330 described later on the screen 22 and displays the image light so as to be visible from the outer surface side of the display plate 21.

  Although not shown in detail, the image projection unit 1330 includes a projection light source, an image translucent display unit, and an optical system. The image projection unit 1330 projects the image light toward the display plate 21 (screen 22). It is disposed at a predetermined position inside the body 1311. The image projection unit 1330 projects image light of image information input from a display control device 1100 (image processing device 62), which will be described later, onto the screen 22.

The touch operation detection unit 1340 includes a detection light source 41 and a detection light receiving device 42.
The detection light source 41 is a light source that emits infrared light (detection infrared light) as detection light, and irradiates the inside of the display plate 21 with detection infrared light on the side end surface of the display plate 21 in the display unit 200. It is arranged to do. The detection infrared light irradiated inside from the side end surface of the display plate 21 by the detection light source 41 repeats total reflection on the inner and outer surfaces of the display plate 21 and reaches the opposite side end.
Although not shown in detail, the detection light receiving device 42 is a so-called infrared camera that includes an imaging optical system, a light receiving element, and the like, and can capture detection infrared light having a wavelength emitted from the detection light source 41 as an image. The detection light receiving device 42 is disposed at a predetermined position inside the housing 1311 so that the entire area of the screen 22 can be photographed.

The touch operation detection unit 1340 configured as described above detects the detection infrared light scattered by the contact of the hand (finger or palm, etc.) that is the operation subject to the outer surface of the display plate 21 in the display unit 200. The device 42 takes an image. That is, when a hand or the like comes into contact with the outer surface of the display plate 21, the reflected infrared light path in the display plate 21 is disturbed at the contact portion, and the detected infrared light leaks to the inner surface side of the display plate 21. . The detection light receiving device 42 images the detection infrared light leaked to the inner surface side of the display plate 21.
Then, the touch operation detection unit 1340 outputs imaging information including contact information of the hand to the outer surface of the display plate 21 by the detection light receiving device 42 to the display control device 1100 via the output unit 65.

  In this embodiment, the movement amount detection unit 1350 includes an infrared light source 51 that irradiates a surface (installation surface) on which the information display device 1200 is installed, and an infrared light installation surface that is irradiated by the infrared light source 51. It is comprised by the light-receiving part 52 which light-receives reflected light. The movement amount detection unit 1350 detects the movement of the light receiving unit 52 (that is, the movement of the information display device 1200) from the change in the infrared reflected light from the installation surface received by the light receiving unit 52, and the detection information (movement information). ) To the display control device 1100 (CPU 61).

  The display control device 1100 includes a CPU 61, an image processing device 62, a memory 63, a storage 66, an output cable 64 connected to the external device 1500, and the like. The display control apparatus 1100 drives the CPU 61 according to the program stored in the memory 63, processes the image information stored in the storage 66 by the image processing apparatus 62, outputs the processed image information to the image projection unit 1330, and passes through the image projection unit 1330. Display on the display unit 200.

  Further, the display control device 1100 analyzes the imaging including the touch operation information to the outer surface of the display board 21 input from the touch operation detection unit 1340 (detection light receiving device 42), and operates the operation mode (touch mode or mouse mode). ) And the respective controls.

The image processing device 62 analyzes the imaging of the display board 21 input from the touch operation detection unit 1340 (detection light receiving device 42), and creates determination basic information for operation mode determination.
The basic determination information for determining the operation mode in the present embodiment is the position, contact area, number, etc., of the hand touching the outer surface of the display plate 21 in the display unit 200.
The image processing device 62 outputs the created basic determination information to the CPU 61.

The CPU 61 determines the operation mode (touch mode or mouse mode) based on the determination basic information input from the image processing device 62 and controls each of them.
For example, the CPU 61 determines the operation mode by comparing the area of the contact area where the hand contacts the outer surface of the display panel 21 in the display unit 200 with a predetermined criterion (reference area). If the contact area does not exceed the reference area, the touch mode is determined. If the contact area exceeds the reference area, the mouse mode is determined.

  That is, as shown in FIGS. 32 (a) and 32 (b), when the display board 21 is operated with a fingertip, the area of the contact area TA (S) is small even in total, and the touch mode is determined. On the other hand, when the entire information display device 1200 is gripped for movement operation, the palm touches the display board 21 as shown in FIG. 32 (c), and the area of the contact area TA (L) increases. The mouse mode is determined.

  Further, the operation mode may be determined based on the number of contact areas. For example, as shown in FIG. 32 (d), the mouse mode is determined when the contact area TA is four or more. In addition, you may determine the criteria which considered both the contact area and the number of contact areas.

  Further, when the CPU 61 determines that the mouse mode is determined by the operation mode determination, the CPU 61 may additionally determine the direction of the operation by identifying the direction of the operating hand. That is, when a user puts his / her hand on the display unit 200 (display board 21) to move the information display device 1200, as shown in FIG. 32 (d), a plurality of small contact areas by the fingertips: TA (S) and palm A wide contact area by TA: TA (L) is detected. Accordingly, a plurality of small contact areas: TA (S) side is identified as the front side: F, and a wide contact area: TA (L) side is identified as the front (rear) side: R, and the movement coordinates are set. Control based on the movement coordinates.

Further, as shown in FIG. 33, the CPU 61 displays the operation mode to be applied (touch mode or mouse mode) on the periphery of the display unit 200 (display board 21) that is not hidden by the operating hand. As a result, the applicable operation mode can be understood by the operator, and erroneous operation can be prevented. In FIG. 33, the mouse mode is displayed by the mouse symbol: S and the character notation: L.
In addition, it is good also as a structure which turns off the display in the display part 200 as control which makes an operator recognize the operation mode to apply. In the mouse mode, the display unit 200 is covered with a hand so that it is difficult to visually recognize the display unit 200, so that there is little need for display on the display unit 200, thereby saving power.

After determining the operation mode as described above, the CPU 61 controls each of the touch mode and the mouse mode according to the determination result.
In the touch mode, from the imaging of the detection light receiving device 42 input from the touch operation detection unit 1340, the contact and movement of the hand on the outer surface of the display board 21 are analyzed, and the contact and movement are displayed on the display board 21. It is recognized and controlled as a click on an image (for example, a thumbnail) or an enlargement / reduction and movement operation of a display image.
In the mouse mode, a signal based on the movement information of the information display device 1200 input from the movement amount detection unit 1350 is output to the external device 1500 connected via the output cable 64, and the external device 1500 is operated.

  The display control device 1100 may be connected to the display input unit 1300 by wire or wireless. The display control device 1100 is disposed inside the display input unit 1300 (inside the housing 1311). May be. Further, the display control apparatus 1100 may be configured by a personal computer.

  In the information display device 1200 configured as described above, the CPU 61 in the display control device 1100 determines the touch mode and the mouse mode, and touches the image displayed on the display unit 200 in the touch mode. In the mouse mode, the external device 1500 can be operated by moving the information display device 1200. Thereby, it has a function of both a display apparatus and a mouse | mouth.

[Eighteenth Embodiment]
Next, an eighteenth embodiment of the present invention shown in FIG. 34 will be described.
FIG. 34 is a conceptual configuration diagram of an information display device 1200B according to the eighteenth embodiment of the present invention.
Note that the basic configuration of the information display apparatus 1200B in the eighteenth embodiment is the same as that of the information display apparatus 1200 in the seventeenth embodiment described above, and the same components are denoted by the same reference numerals and description thereof is omitted.
An information display device 1200B illustrated in FIG. 34 includes a housing 1311, a display unit 200 disposed on the top of the housing 1311, an image projection unit 1330, a touch operation detection unit 1340, a movement amount detection unit 1350, A pressure sensor 1370 and a display control device 1100 are provided. A display 300 is connected to the information display device 1200B.

The display plate 21 in the display unit 200 is formed in a hollow hemispherical shape with a predetermined plate thickness, and a screen 22 is formed over the entire inner peripheral surface thereof.
The image projection unit 1330 includes a projection display unit 31 and a projection light receiving optical system 32.
The projection display unit 31 includes an image translucent display unit using an LCD and a projection light source (not shown).

The projection light receiving optical system 32 includes an objective lens 32A that is a fish-eye lens having an angle of view of approximately 180 °, and the entire area of the display unit 200 is disposed within the angle of view.
The projection light receiving optical system 32 is disposed substantially at the center of the hemispherical display unit 200 so as to project image light toward the display plate 21 (screen 22). The projection light receiving optical system 32 projects image light of image information input from the display control device 1100 (image processing device 62) onto the screen 22. The projection light receiving optical system 32 also serves as the incident optical system of the touch operation detection unit 1340.

The touch operation detection unit 1340 includes a detection light source 41 and a detection light receiving device 42.
The detection light source 41 is arranged on the side end surface of the display plate 21 in the display unit 200 so as to irradiate the inside of the display plate 21 with detection infrared light. The detection infrared light irradiated inside from the side end surface of the display plate 21 by the detection light source 41 repeats total reflection on the inner and outer surfaces of the display plate 21 and reaches the opposite side end.

The detection light receiving device 42 includes a dichroic mirror 42M and an infrared light receiving sensor 42S such as a CCD.
The dichroic mirror 42M is a reflecting mirror having a frequency characteristic that reflects only infrared light having a specific wavelength (detected infrared light emitted from the detection light source 41) and transmits visible light. The dichroic mirror 42M is interposed in the optical path between the projection display unit 31 and the projection light receiving optical system 32 in the projection light receiving optical system 32 at a predetermined angle with respect to the optical path.
The infrared light receiving sensor 42S is disposed so as to be able to detect detection infrared light derived from the detection light source 41 that is incident from the projection light receiving optical system 32 and reflected by the dichroic mirror 42M.
Since such a detection light receiving device 42 shares the image projecting unit 1330 and the projection light receiving optical system 32, the entire configuration can be made compact, and image distortion correction is unnecessary or easy.

  The movement amount detection unit 1350 includes a ball 53 rotatably provided at the bottom of the housing 1311 and a sensor (not shown) that detects rotation of the ball 53 in two directions orthogonal to each other, and includes a housing 1311 (that is, an information display device). 1200B) is a so-called trackball that detects and outputs rotation of the ball 53 with a sensor.

The pressure sensor 1370 is interposed between the housing 1311 and the display plate 21 in the display unit 200. The pressure sensor 1370 detects the pressing force acting on the display board 21 and outputs the detection information to the display control device 1100.
The configuration for detecting the pressing force acting on the display board 21 is not limited to the pressure sensor 1370, and a switch that is turned ON / OFF by a predetermined pressing force may be used. Moreover, the arrangement | positioning position may intervene between an installation surface as the back surface of the housing | casing 1311, for example.

  The display 300 displays an image or the like instead of the display unit 200 because the display unit 200 is covered with a hand in the mouse mode and is difficult to view. The operability is improved by representing the mouse mode operation with a pointer on the screen.

The display control device 1100 determines the operation mode (touch mode or mouse mode) based on the detection information of the pressing force acting on the display unit 200 (display board 21) input from the pressure sensor 1370, and controls each of them. Do.
The operation mode is determined by comparing the pressing force applied to the display board 21 input from the pressure sensor 1370 with a determination reference (threshold value), and when the pressing force does not exceed a predetermined reference pressure, the touch mode is determined. When the pressing force exceeds the reference pressure, the mouse mode is determined.

  That is, when the display board 21 is operated with a fingertip, the applied pressure is small, so the touch mode is set. When the entire information display device 1200 is moved and operated, at least the weight of the hand is applied. Since the pressing force increases, the mouse mode is determined.

  After determining the operation mode as described above, the display control device 1100 controls each of the touch mode and the mouse mode according to the determination result. This control is the same as in the seventeenth embodiment described above, but in the mouse mode, an image or the like is displayed on the display 300 instead of the display unit 200, and the mouse mode operation is performed on the screen. Display with pointer.

[Nineteenth Embodiment]
Next, a nineteenth embodiment of the present invention shown in FIG. 35 is described.
FIG. 35 is a conceptual structural diagram of an information display apparatus 1200C in the nineteenth embodiment of the present invention.
The basic configuration of the information display device 1200C in the nineteenth embodiment is the same as that of the information display device 1200B in the eighteenth embodiment described above, and the same components are denoted by the same reference numerals and description thereof is omitted.

The information display device 1200C according to the nineteenth embodiment differs from the information display device 1200B according to the eighteenth embodiment in the shape of the housing 1311 and the configuration of the pressure sensor 1370.
The information display device 1200C includes a palm rest 1312 for placing a hand (palm) 190 in the touch mode.
In other words, the palm rest 1312 is formed so as to protrude from one side of the housing 1311 and whose upper surface is inclined downward toward the outside.

A pressure sensor 1370 is provided on the top surface of the palm rest 1312.
The pressure sensor 1370 detects the pressure of the hand 190 placed on the palm rest 1312 and outputs the detection information to the display control device 1100.
When the pressure detection information of the hand 190 placed on the palm rest 1312 is input from the pressure sensor 1370, the display control device 1100 determines that the mouse mode is set.
The detection of the hand 190 placed on the palm rest 1312 is not limited to the pressure sensor, and may be a press switch, an infrared detection switch, or the like.

[20th embodiment]
Next, a twentieth embodiment of the present invention shown in FIG. 36 will be described.
FIG. 36 is a conceptual configuration diagram of an information display device 1200D according to the twentieth embodiment of the present invention.
Note that the basic configuration of the information display device 1200D in the twentieth embodiment is the same as that of the information display device 1200B in the eighteenth embodiment described above, and the same components are denoted by the same reference numerals and description thereof is omitted.
In addition to the information display device 1200B in the eighteenth embodiment described above, the information display device 1200D according to the twentieth embodiment includes a movement restriction mechanism 1390 that restricts the movement of the information display device 1200D in the touch mode.

The movement restricting mechanism 1390 includes a slide pad 91 provided at the bottom of the housing 1311 and an actuator 92 that moves and drives the slide pad 91.
The slide pad 91 is made of a material having a low friction coefficient and high wear resistance (for example, polyacetal resin), and is in a retracted state retracted into the housing 1311 and an operating state projecting a predetermined amount from the bottom surface of the housing 1311. It is provided so that it can move between.
The actuator 92 is, for example, an electrically driven solenoid or the like. The actuator 92 is driven and controlled by the display control device 1100, and the slide pad 91 is moved from the retracted state retracted into the housing 1311 to the operating state protruding from the bottom surface. To do.

The display control device 1100 puts the slide pad 91 in the retracted state when the touch mode is determined by the operation mode determination, and drives the actuator 92 to move the slide pad 91 to the housing 1311 when the mouse mode is determined. Project from the bottom.
Accordingly, in the touch mode, the entire bottom surface of the housing 1311 is in contact with the installation surface and can be operated with stability, and in the mouse mode, only the slide pad 91 is in contact with the installation surface and can be smoothly moved. It becomes.

[Twenty-first embodiment]
Next, a twenty-first embodiment of the present invention shown in FIG. 37 is described.
FIG. 37 is a conceptual structural diagram of an information display device 1200E according to the twenty-first embodiment of the present invention.
The basic configuration of the information display device 1200E in the twenty-first embodiment is the same as that of the information display device 1200 in the seventeenth embodiment described above, and the same components are denoted by the same reference numerals and description thereof is omitted.

An information display device 1200E according to the twenty-first embodiment includes an audio generation unit 1380 in addition to the information display device 1200 according to the seventeenth embodiment described above.
The sound generation unit 1380 is connected to the CPU 61 and can play music stored in the storage 66 from the information display device 1200E.
In the twenty-first embodiment, in the mouse mode, the music in the sound generation unit 1380 is changed by the instruction from the CPU 61 based on the movement information of the information display device 1200 input from the movement amount detection unit 1350, or the sound of the sound is displayed. Changes in strength are made.

As described above, this embodiment has the following effects.
(1) The information display device 1200 according to the present invention includes a touch mode in which an image displayed on the display unit 200 is touched to operate the display, and various operations are performed by moving the entire information display device 1200. It can be operated in two operation modes, mouse mode, where operation is possible.
(2) Since the information display device 1200 is switched between the operation modes of the touch mode and the mouse mode based on the determination based on the contact state with the display unit 200, the operation mode can be easily switched.

  The present invention is not limited to the above-described embodiment, and various modifications and changes as described below are possible, and these are also within the scope of the present invention. For example, the shape of the display unit 200, the configuration of the touch operation detection unit 1340 and the movement amount detection unit 1350, and the combination of configurations in each of the above embodiments are not limited to each embodiment, and can be changed as appropriate.

  In addition, although embodiment and a deformation | transformation form can also be used in combination as appropriate, detailed description is abbreviate | omitted. Further, the present invention is not limited to the embodiment described above.

In the description of each embodiment described above, the display control device 1100, the search unit control device 1101, and the search result output device 130 display information on the display units 200 and 200A embedded in the top plate portion of the desk 3. However, it is not limited to this. For example, the display unit 200 may be installed on a wall, or the hemispherical display unit 200 may be installed on the desk 3 as shown in FIG.
FIG. 1 is a configuration diagram illustrating an example of a display system 1000 that outputs to a display unit 200 configured in a hemispherical shape.
The display control apparatus 1100 displays an image indicating the search result on the display unit 200. In this case, the image displayed on the display unit 200 is visually recognized only by the specific speaker 2. For example, the display control device 1100, the search means control device 1101, and the search result output device 130 are uttered by a specific speaker 2 (for example, the speaker 2A in FIGS. 2 and 7) sitting in a predetermined direction. Based on the received voice information, specific character information can be selected from the character information. Thereby, for example, the display control device 1100, the search means control device 1101, and the search result output device 130 cause the speaker 2 </ b> A to search for and display specific information (for example, conference material created by the speaker 2 </ b> A itself). When the content requested to the user is spoken, specific information can be displayed so that only the speaker 2A can visually recognize it.

  In each of the above-described embodiments, the display control device 1100, the search unit control device 1101, and the search result output device 130 have been described by taking the configuration including the four sound collection units 101 as an example. Absent. The display control device 1100, the search means control device 1101, and the search result output device 130 are only required to be able to detect the sound around the sound collection unit 101. For example, a single or a plurality of sound collection units 101 are provided below the desk 3. May be provided. Thereby, the display control device 1100, the search means control device 1101, and the search result output device 130 can include, for example, the sound collecting units 101 having the positions and the number corresponding to the size of the desk 3.

  As mentioned above, although embodiment of this invention has been explained in full detail with reference to drawings, a concrete structure is not restricted to this embodiment and can be suitably changed in the range which does not deviate from the meaning of this invention. .

  Note that the information generation unit 1110, result output processing unit 1120, generation unit 102, conversion unit 103, identification unit 104, selection unit 106, output unit 107, result selection unit 108, setting unit 109, or result output in the above embodiment The unit 110 (hereinafter collectively referred to as a control unit CONT) or each unit included in the control unit CONT may be realized by dedicated hardware, or realized by a memory and a microprocessor. It may be allowed.

  Each unit included in the control unit CONT includes a memory and a CPU (central processing unit), and the function is realized by loading a program for realizing the function of each unit included in the control unit CONT into the memory and executing the program. It may be realized.

  Further, by recording a program for realizing the function of each unit included in the control unit CONT on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, the control unit You may perform the process by each part with which CONT is provided. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

1000 ... display system 1100 ... display control device, 1110 ... information generation unit, 1120 ... result output processing unit,
1200 ... information display device,
2 ... Speaker, 11 ... Search means,
101 ... Sound collecting unit, 200 ... Display unit (display device), 300 ... Display (external device)

Claims (27)

An information generating unit that generates information associated with the voice uttered by the speaker from the voice uttered by the speaker;
Output first search information corresponding to a first search result searched based on first information associated with voices uttered by a plurality of the speakers, and at least one of the plurality of speakers And a result output processing unit that outputs second search information corresponding to the second search result searched based on the second information associated with the voice uttered by the speaker. . The information generator is
Generating the first information based on the utterance content of the speaker extracted from the voices uttered by the plurality of speakers, and extracting from the voice uttered by at least one of the plurality of speakers The display control apparatus according to claim 1, wherein the second information is generated based on the content of any one of the speakers. The result output processing unit
The first search information to be displayed on a first screen and the second search information to be displayed on a second screen are output as information to be displayed on a plurality of screens. Display control device. The information generator is
Detecting the state of the speaker, and generating second information associated with the voice uttered by the speaker whose state is detected;
The result output processing unit
The display control apparatus according to claim 3, wherein the second search information is obtained based on second information associated with a voice uttered by a speaker whose state is detected. The information generator is
Identifying the speaker by detecting the position of the speaker, and generating second information associated with the voice uttered by the identified speaker;
The result output processing unit
The display control apparatus according to claim 4, wherein the second search information is obtained based on second information associated with a voice uttered by the identified speaker. The information generator is
Identifying the speaker looking at the second screen and generating second information associated with the speech uttered by the identified speaker;
The display control device according to claim 3, wherein the display control device is a display control device. The information generator is
Generating the second information extracted from the speech information spoken by the speaker and the information identified from the speech information spoken by the speaker identified as viewing the second screen. The display control apparatus according to claim 6. The information generator is
The display control device according to any one of claims 4 to 7, wherein the second information is generated according to a state of the speaker. The result output processing unit
The display control according to any one of claims 4 to 8, wherein the second search information to be selected from among the second search results is selected according to the state of the speaker. apparatus. The information generator is
The display control apparatus according to claim 4, wherein the state of the speaker is identified by the detected posture of the speaker. The information generator is
The display control apparatus according to any one of claims 4 to 10, wherein the state of the speaker is identified by the detected speech frequency (aggressiveness) of the speaker. The information generator is
The first information and the second information are different from each other so that the first information serving as a search condition for searching for the first search information and the second information serving as a search condition for searching for the second search information are different from each other. The display control apparatus according to any one of claims 1 to 11, wherein: The result output processing unit
Outputting the first search information in the order obtained;
The display control apparatus according to any one of claims 1 to 12, wherein the display information is output in an order corresponding to a priority of outputting the second search information. The information generator is
An extraction unit that extracts words included in the speech uttered by the plurality of speakers;
A search processing control unit that controls a search unit to execute a search based on the first information that defines a search condition based on the word extracted by the extraction unit, and obtains the first search result;
With
The search processing control unit
The display according to any one of claims 1 to 13, further comprising: a first search mode for widening the search condition; and a second search mode for narrowing the search condition. Control device. The information generator is
A converter that converts voice information of the voice uttered by the speaker into character information;
A selection unit that selects specific character information from the character information based on the importance of the speaker;
An output unit that outputs the specific character information selected by the selection unit as the first information to a search unit that searches for information related to the input character information;
With
The result output processing unit
The display control apparatus according to any one of claims 1 to 14, wherein the first search information corresponding to the first search result by the search means is output. An operation detection unit for detecting an operation performed on the display control device;
With
The operation detection unit is
Determining whether the operation is a first operation related to driving the display unit displaying the first search information or a second operation related to driving an external device;
The result output processing unit
When the operation is determined as the first operation, the display unit is driven according to the content of the operation, and when the operation is determined as the second operation, the drive is performed according to the content of the operation. The display control apparatus according to any one of claims 1 to 15, wherein a signal is output to the external apparatus. A display control device according to claim 1,
The display control device includes:
The display system, wherein the first search information is displayed on a display unit, and the second search information is displayed on an external device. A display control device according to claim 1;
And a display unit that displays the first search information output from the display control device. The display unit
The display device according to claim 18, wherein the display surface includes a part of a spherical surface or a flat surface. An image projection unit that projects an image on the display unit;
With
The display control device includes:
A signal output unit that outputs at least the second search information to be displayed on an external device;
An operation detection unit for detecting an operation performed on the display control device;
With
The display control device includes:
It is determined whether the operation is a first operation related to driving of the display unit or a second operation related to driving of the external device, and when the operation is determined to be the first operation, the operation is determined according to the content of the operation. The display unit is driven, and when the operation is determined to be the second operation, a drive signal corresponding to the content of the operation is output from the signal output unit to the external device. The display device according to claim 19. A display control device according to claim 1;
A terminal display for displaying the first search information;
With
The result output processing unit
The terminal device, wherein the first search information is displayed on the terminal display unit. A terminal device controlled by the display control device according to claim 1,
A terminal display for displaying the first search information;
A terminal control unit that obtains the first search information from the display control device and displays the first search information on the terminal display unit. Generating information associated with the voice uttered by the speaker from the voice uttered by the speaker;
Outputting first search information according to a first search result searched based on first information associated with voices uttered by a plurality of the speakers;
A step of outputting second search information according to a second search result searched based on second information associated with voice uttered by at least one of the plurality of speakers;
A display control method comprising: In the computer provided with the display control device,
Generating information associated with the voice uttered by the speaker from the voice uttered by the speaker;
Outputting first search information corresponding to a first search result searched based on first information associated with voices uttered by a plurality of the speakers;
The step of further outputting the second search information corresponding to the second search result searched based on the second information associated with the voice uttered by at least any one of the plurality of speakers is executed. Program to let you. A computer provided in a display device including a display control device and a display unit that displays information output by the display control device,
Generating information associated with the voice uttered by the speaker from the voice uttered by the speaker;
Outputting first search information corresponding to a first search result searched based on first information associated with voices uttered by a plurality of the speakers;
A step of further outputting second search information corresponding to a second search result searched based on second information associated with a voice uttered by at least one of the plurality of speakers;
Displaying either the first search information or the second search information on the display unit;
A program for running In a computer provided in a terminal device provided with a terminal display unit,
Generating information associated with the voice uttered by the speaker from the voice uttered by the speaker;
Displaying, on the terminal display unit, first search information corresponding to a first search result searched based on first information associated with voices uttered by a plurality of the speakers;
Outputting second search information according to a second search result searched based on second information associated with a voice uttered by at least one of the plurality of speakers;
A program for running The display content to be displayed on the terminal display unit provided in a computer provided in the terminal device controlled by the display control device according to claim 1,
The terminal display unit displaying the first search information;
Obtaining the first search information from the display control device and displaying it on the terminal display unit;
A program for running

Images