JP2005184583A - Information processing apparatus and communication method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing apparatus and a communication method, which share among users factors that disturb progress of communication. <P>SOLUTION: User terminals 100a, 100b realize interaction environment, and display a user A log display area 22a, a user B log display area 22b, a common language log display area 22c, a user A using language input column 25a, and the common language input column 25c. The user A inputs phrases to be searched in the user A using language input column 25a or in the common language input column 25c. The inputted phrases and their translation are displayed in the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c. The information processing device sends and receives operation information to a user interface so as display contents of the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c, become the same in all user terminals 100a, 100b. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information processing apparatus and a communication method that realize an interactive environment by exchanging video and audio via a network.

  In recent years, with the spread of network equipment, communication between remote locations has become more familiar. For example, this is a video conference system.

  A conventional video conference system is configured to mutually transmit and receive video and audio acquired at each site and reproduce them. However, in a system that simply exchanges the state of users with each other, the information to be transmitted is limited, and the target information may not be sufficiently transmitted to the other party.

  As a technique for solving such a problem, for example, there is a technique disclosed in Patent Document 1 shown below. A system configuration according to the conventional technology is shown in FIG.

  As shown in FIG. 1, the conventional system configuration has a server / client type system configuration, and includes client terminals 910 a and 910 b connected to a network 20, a server 920, and a content server 930. The client terminals 910a and 910b are terminals used by each user, and have a camera and a microphone for acquiring video and audio. The server 920 transmits video and audio sent from the client terminals 910a and 910b to the other client terminals 910b and 910a, respectively. As a result, video and audio are relayed live between the client terminals 910a and 910b. The content server 930 stores multimedia content used for meetings and education, and distributes it based on requests from the client terminals 910a and 910b. The user uses the multimedia contents together with the video and audio sent from the other client terminal to proceed with the conference or receive education.

This prior art is intended to correctly convey information held by one user to the other user. Therefore, in the conventional technique, the target multimedia content is registered in advance in the content server 930 and is distributed to each of the client terminals 910a and 910b as necessary.
JP 2003-9105 A

  However, the purpose of the conference using the video conference system is not limited to the transmission of information. In many cases, for example, brainstorming-type conferences aim to jointly reach some purpose. In such a case, in order to facilitate communication in the process of joint action, it is important to extract a factor that hinders this and solve it.

  For example, in a video conference across countries, a conference is conducted using a language that all participating users can understand. If one user is not fluent in the other user's language, a common language such as English is used. It is necessary to proceed with the conversation. In such a case, the meaning of the words and sentences may become unclear, and problems may occur in the progress of communication, making it impossible to smoothly proceed with the conference. Also, in presentations and educational sites, problems may occur in the progress of communication due to the lack of information that listeners should have as knowledge, such as the meaning of technical terms and current news.

  In order to solve such a problem, it is necessary to let other users know information that is unclear for a certain user, that is, a factor that hinders the progress of communication. If this factor is understood, the user can proceed with meetings, education, presentations, etc. based on this factor.

  However, it is impossible to solve such a problem with the prior art that only distributes information registered in advance as described above.

  Therefore, an object of the present invention is to provide an information processing apparatus and a communication method that can share a factor that hinders the progress of communication among users.

  In order to achieve the above object, the present invention has a user interface common to another information processing apparatus connected via a network and a video image with the other information processing apparatus. And an information processing apparatus that realizes an interactive environment between users by exchanging voices, wherein search means for searching for information based on information input to the user interface, and search results by the search means are displayed. Interface creation means for updating the user interface; transmission means for sending the input information and the search result to the other information processing apparatus; and input information and input from the other information processing apparatus to the user interface. Receiving means for receiving a search result obtained based on information, wherein the interface creating means is the other information processing apparatus. Configured to update the user interface based on the input information and the search results are al received. By providing a user interface that is shared with other information processing apparatuses, it is possible to share contents input to this user interface among users. Therefore, by inputting factors that inhibit the progress of communication into this user interface, for example, it is possible to share what is the factor that inhibits the progress among users, and as a result, how to solve this factor It is possible to easily guide whether or not In addition, as a factor that hinders communication, there is often a lack of knowledge of one user. Therefore, by providing a user interface for searching for information that is unclear to one user and notifying the other user of operations on this user interface, a factor that hinders the progress of communication, that is, It is possible to easily share information that is unclear to other users. Further, by adopting a configuration in which unclear information can be searched, unclear information can be shared and clarified at the same time.

  Further, the information processing apparatus according to claim 1 is preferably configured such that the user interface lists and displays the history of the input information or the history of the search result as described in claim 2. By displaying the retrieved histories in a list, each user can easily reconfirm what information has been retrieved while the communication is in progress.

  Further, the information processing apparatus according to claim 1 has storage means for storing information, for example, according to claim 3, and the search means searches for information from the storage means based on the input information. It may be configured as follows. That is, the information source of the information to be searched may be provided inside the information processing apparatus.

  Further, the information processing apparatus according to claim 3 may be an electronic dictionary that searches for a translation corresponding to the input information, for example, as described in claim 4. As communication through networks becomes widespread, opportunities for communication between users with different native languages such as cross-country communication and cross-cultural communication increase. In such a case, using an electronic dictionary is a great help for communication. Also, sharing the user interface for utilizing this electronic dictionary among users is a great help in understanding the other person's thoughts and vocabulary unclear for the other party.

  The information processing apparatus according to claim 1 includes request means for requesting another information processing apparatus via the network to perform a search based on the input information, for example, as in claim 5. May be. That is, the information source of the information to be searched may be provided in another information processing apparatus via the network.

  Further, the information processing apparatus according to claim 1 includes request means for requesting a search based on the input information from a search engine connected via the network, as in claim 6. May be. That is, the information source of the information to be searched may be an information site on the network. In this case, the information processing apparatus is configured to request a search from a search engine that searches for information sites on the network.

  In addition, the information processing apparatus according to claim 1 may be connected to the other information processing apparatus via a peer-to-peer network as described in claim 7, for example. By connecting information processing apparatuses via a peer-to-peer network, there is no need for a server or the like, so communication via the network can be realized with a simple system configuration. However, the present invention does not exclude a system configuration via a server, that is, a server / client system configuration.

  According to another aspect of the present invention, there is provided a communication method for communicating with another user by exchanging video and audio with another information processing apparatus via a network. A first step of creating a user interface common to other information processing apparatuses, a second step of searching for information based on information input to the user interface, and a search result of the second step are displayed. A third step of updating the user interface, a fourth step of transmitting the input information and the search result to the other information processing device, and an input from the other information processing device to the user interface A fifth step of receiving information and a search result obtained based on the input information, and receiving in the fifth step And based on the input information and the search results made and a sixth step of updating the user interface. By creating a user interface and transmitting / receiving an operation for this to / from another information processing apparatus, it is possible to share the contents input to this user interface among users. Therefore, by inputting factors that inhibit the progress of communication into this user interface, for example, it is possible to share what is the factor that inhibits the progress among users, and as a result, how to solve this factor It is possible to easily guide whether or not Further, as described above, one factor that hinders communication is a lack of knowledge of one user. Therefore, by providing a user interface for searching for information that is unclear to one user and notifying the other user of operations on this user interface, a factor that hinders the progress of communication, that is, It is possible to easily share information that is unclear to other users. Further, by adopting a configuration in which unclear information can be searched, unclear information can be shared and clarified at the same time.

  According to the present invention, a factor that hinders the progress of communication can be shared among users, and as a result, the user can easily guide how to solve this factor.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  First, Embodiment 1 according to the present invention will be described in detail with reference to the drawings. FIG. 2 is a diagram showing a schematic configuration of the dialogue support system 1 according to the present embodiment. Note that the communication method according to the present invention is realized by using the dialogue support system 1.

  As shown in FIG. 2, the dialogue support system 1 includes user terminals 100 a and 100 b connected via a network 20. As the network 20, for example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or the like can be applied. Each of the user terminals 100a and 100b is mounted on the network 20 as a peer on a peer-to-peer framework represented by JXTA (registered trademark) or the like. That is, the user terminals 100a and 100b can form a peer-to-peer network with each other. The user terminals 100a and 100b use this function to exchange various data on the network 20, thereby realizing real-time interactive communication without using a server or the like.

  The user terminal 100a includes a PC main body 101 on which a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive and the like are mounted, a display 102, a keyboard 103, a mouse 104, and the like. And a console. An information processing apparatus such as a personal computer (hereinafter referred to as a PC) can be applied to the user terminal 100a. Since the user terminal 100b has the same configuration, the following description will focus on the user terminal 100a.

  The user terminal 100a includes a camera 105, a microphone 106, and a speaker 107 in addition to the configuration described above. The camera 105 has a configuration for acquiring a video of a user who participates in the dialogue. The microphone 106 has a configuration for acquiring audio from a user who participates in the dialogue. The video and audio acquired by the camera 105 and the microphone 106 are input to the PC main body 101. The PC main body 101 transmits the input video and audio to another user terminal (here, the user terminal 100b). In other words, the video and audio acquired by the camera 105 and the microphone 106 are shared with other user terminals.

  Further, the PC main body 101 reproduces the video and audio received from the other user terminal 100 b using the display 102 and the speaker 107. In this way, since video and audio are relayed live between the user terminals 100a and 100b, a user of the user terminal 100a (referred to as user A) and a user of the user terminal 100b (referred to as user B). Dialogue is possible via the network 20. Note that the user terminals 100 a and 100 b may reproduce the video acquired by the camera 105 provided on the user terminal 100 a and 100 b on the display 102 of the user terminal.

  An example of the screen displayed on the display 102 is shown in FIG. In the following description, the language used by the user A (for example, the native language, hereinafter referred to as the language used) is Japanese, the language used by the user B is Chinese, and the user A and the user B commonly use the language. The language to be used (hereinafter referred to as a common language) is English. In addition, it is assumed that the user terminals 100a and 100b incorporate electronic dictionaries of daytime (including Chinese and Japanese) and Japanese-English (including English and Japanese). The electronic dictionary functions as search means.

  As shown in FIG. 3, on the display 102, the user A video display area 21a for displaying the video of the user A acquired by the camera 105 of the user terminal 100a and the user acquired by the camera 105 of the user terminal 100b are displayed. A user B video display area 21b for displaying the B video is displayed. User A interacts with user B based on the video displayed in user B video display area 21 b and the audio output from speaker 107.

  In addition, the display 102 of the user terminal 100a has a user A log display area 22a, a user B log display area 22b, and a common language log display area 22c as user interfaces that can be operated by the user to search for various types of information. Then, the user A language input field 25a and the common language input field 25c are displayed.

  The user A use language input field 25a is a field for inputting a phrase (hereinafter referred to as a search word) that the user A wants to check in the user A's use language (Japanese). The user A log display area 22a is an area in which a search word input by the user A and a Japanese translation for the search word input by the user B are displayed. The common language input field 25c is a field for inputting a phrase (hereinafter referred to as a search word) that the user A wants to check in a common language (English). The common language log display area 22c is an area in which an English translation for the search word input by the user A and an English translation for the search word input by the user B are displayed. The user B log display area 22b is an area in which a search word input by the user B and a translated word in Chinese for the search word input by the user A are displayed. In this description, a search term input by the user A / B during communication and a translation for the search term input by the user A / B are referred to as a log (history).

  In the above configuration, when the user A wants to look up an English or Chinese translation for a certain Japanese phrase, he / she enters the target search word in the user A language input field 25a and clicks the “search” button 24a. The “search” button 24a is a user interface that can be operated by the user to input a search execution instruction.

  When the “search” button 24a is clicked, the search term input in the user A use language input field 25a is displayed in the user A log display area 22a. The user A log display area 22a displays the search word log input in the user A language input field 25a in a list format, for example. The newly added search term is displayed at the top or bottom of the list, for example. However, the list is not limited to this, and the list may be displayed in alphabetical order, alphabetical order, or phonetic order.

  When the “search” button 24a is clicked, an English translation related to the input search word is displayed in the common language log display area 22c. The common language log display area 22c displays a log of translated words in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  Further, when the “search” button 24a is clicked, a translated word in Chinese related to the input search word is displayed in the user B log display area 22b. The user B log display area 22b displays the translated word log in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  On the other hand, the display 102 of the user terminal 100b displays a user A video display area 21a, a user B video display area 21b, and various user interfaces similar to the above. However, the user A use language input field 25a and the “search” button 24a are a user B use language input field for the user B to input a search word in the user B's use language (Chinese) and a search word input thereto. It is replaced with a “search” button for instructing execution based on the search. These display positions are on the lower side of the user B log display area 22b. Therefore, the “dictionary switching” button 27 (which will be described later) arranged below the user B log display area 22b in the user terminal 100a is displayed below the user A log display area 22a in the user terminal 100b. Be placed.

  When user B wants to look up an English or Japanese translation for a certain Chinese phrase, he / she enters the target search word in the user B language input field and clicks the “search” button. When the “Search” button is clicked, the search word input in the user B use language input field is displayed in the user B log display area 22b. The user B log display area 22b displays a log of input search terms in a list format, for example. The newly added search term is displayed at the top or bottom of the list, for example. However, the list is not limited to this, and the list may be displayed in alphabetical order, alphabetical order, or phonetic order.

  When the “search” button is clicked, a translated word in English related to the input search word is displayed in the common language log display area 22c. The common language log display area 22c displays a log of translated words in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  Further, when the “search” button is clicked, a translated word in Japanese relating to the input search word is displayed in the user A log display area 22a. The user A log display area 22a displays a log of translated words in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  In addition, when the user A (or B) wants to look up a Japanese or Chinese translation of a certain English word, he / she inputs the target search word in the common language input field 25c and clicks the “search” button 24c. Click. The “search” button 24c is a user interface that can be operated by the user to input a search execution instruction.

  When the “search” button 24c is clicked, the search word input in the common language input field 25c is displayed in the common language log display area 22c. The common language log display area 22c displays the search word log input in the common language input field 25c in a list format, for example. The newly added search term is displayed at the top or bottom of the list, for example. However, the list is not limited to this, and the list may be displayed in alphabetical order, alphabetical order, or phonetic order.

  When the “search” button 24c is clicked, a translated word in Japanese related to the input search word is displayed in the user A log display area 22a. The user A log display area 22a displays a log of translated words in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  Further, when the “search” button 24c is clicked, a translated word in Chinese related to the input search word is displayed in the user B log display area 22b. The user B log display area 22b displays the translated word log in a list format, for example. The newly added translation may be displayed at the top or bottom of the list, for example, but is not limited thereto.

  The log of the search word and the translated word is displayed on the same list in the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c.

  Of the user interfaces described above, the display contents of at least the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c are the same in all the user terminals 100a and 100b. In other words, each user terminal has a user interface shared with other user terminals. Thereby, the other user B can know which vocabulary is unclear for one user A, and the user B can inform the user A which vocabulary is unclear for himself. That is, a factor that hinders smooth communication can be shared with the other party. As a result, this factor can be extracted and solved, and smooth communication can proceed.

  Another configuration of the user interface displayed on the display 102 will be described below. The lists displayed in the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c have a horizontal correspondence. That is, the translated word corresponding to the search word is displayed next to the search word. Thereby, the users A and B can easily recognize the correspondence between the search word and the translated word. However, there are cases where a plurality of translated words correspond to one search word. In such a case, each column of the list may be selectable, and only the translation or search word corresponding to the selected search word or translation may be listed in another display area.

  The user interface displayed on the display 102 includes scroll bars 23a, 23b, and 23c for scrolling the lists displayed in the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c. And scroll buttons 26a and 26c. When the user wants to refer to a log outside the display area, the user operates these bars and buttons using the mouse 104 or the keyboard 103 to move the list display area.

  Further, a “dictionary switching” button 27 is also displayed on the display 102. The “dictionary switching” button 27 is a user interface that can be operated by the user to switch the electronic dictionary. In the above description, the search means is an electronic dictionary of daytime (including Chinese and Japanese) and Japanese-English (including English and Japanese). However, the present invention is not limited to this. For example, an electronic dictionary of another language or an electronic dictionary of technical terms Etc. can be applied. Therefore, by providing a function for switching the search means as described above, the dialogue support system 1 that can cope with various situations can be realized.

  Next, functional blocks of the user terminals 100a and 100b will be described with reference to FIG. Since the functional blocks of the user terminals 100a and 100b have the same configuration, they will be described as the user terminal 100 in the following description.

  As shown in FIG. 4, the user terminal 100 includes a video / audio acquisition unit 11, an input unit 12, a search unit 13, a storage unit 14, an interface creation unit 15, a transmission unit 16, a reception unit 17, and a display unit 18. .

  The video / audio acquisition unit 11 is a function realized by including the camera 105 and the microphone 106, and acquires the video and audio of the user A (B). The video / audio acquisition unit 11 inputs the acquired video to the display unit 18 and inputs the acquired video and audio to the transmission unit 16.

  The input unit 12 is a function realized by including the keyboard 103 and the mouse 104, and inputs various operations on the user interface shown in FIG. The input unit 12 inputs the search term input in the user A (B) language input field 25a to the search unit 13, the interface creation unit 15, and the transmission unit 16, respectively. Further, the input unit 12 inputs information of other input operations to the interface creation unit 15. The other operation is, for example, scrolling of a list.

  The search means 13 is a function realized by the CPU of the PC main body 101 and searches for the content stored in the storage means 14 based on the input search word. The storage unit 14 is a function realized by including the hard disk drive of the PC main body 101, and stores data serving as contents such as an electronic dictionary. The electronic dictionary described above is a function realized by including the search means 13 and the storage means 14. The search means 13 inputs the translation obtained as a search result to the interface creation means 15 and also to the transmission means 16.

  The transmission means 16 is a function realized by including the network adapter of the PC main body 101 and transmits various data to other user terminals 100 via the network 20. The various data includes video and audio input from the video / audio acquisition unit 11, a search word input from the input unit 12, other operation information, a translation input from the search unit 13, and the like.

  The receiving means 17 is a function realized by including the network adapter of the PC main body 101 and the like, and receives various data (including video and audio, search words and translated words) transmitted from other user terminals 100 via the network 20. Receive. The receiving unit 17 inputs the received video and audio to the display unit 18. In addition, the transmission unit 17 inputs the received search word, translation, and other operation information to the interface creation unit 15.

  The interface creation unit 15 is a function realized by the CPU of the PC main body 101, and creates and updates the user interface shown in FIG. 3 based on the input search word, translated word, and other operation information. The interface creation unit 15 inputs the created and updated user interface to the display unit 18.

  The display means 18 is a function realized by including the display 102 and the speaker 107. The display unit 18 displays the video acquired by the video / audio acquisition unit 11 in the user A video display area 21a (see FIG. 3). The display unit 18 displays the video received by the receiving unit 17 in the user B video display area 21b (see FIG. 3), and outputs the audio received by the receiving unit 17 from the speaker 107 as audio. Further, the display means 18 displays various user interfaces created and updated by the interface creation means 15 as shown in FIG.

  Each means described above is realized by a predetermined program executed in the PC main body 101 in addition to the above-described configuration. This program is recorded on the hard disk of the PC main body 101, for example, and is read and executed as necessary.

  Next, the operation of the user terminal 100 will be described with reference to the drawings. 5 to 9 are flowcharts showing the operation of the user terminal 100.

  First, as shown in FIG. 5, when the predetermined program is started, the user terminal 100 searches for another user terminal 100 existing on the network 20 as a peer-to-peer peer (step S101). When another user terminal 100 exists (Yes in step S102), the user terminal 100 acquires an address on the network 20 of the other user terminal 100 (step S103). This address is, for example, an IP (Internet Protocol) address. Next, the user terminal 100 generates a predetermined event (step S104). Thereafter, the user terminal 100 returns to step S101 and searches for another user terminal 100 anew.

  The occurrence of the event is monitored by a resident process such as an event listener. When the occurrence of an event is detected, the user terminal 100 executes the operations shown in FIGS. These operations are also realized by a predetermined program executed by the PC main body 101.

  First, the operation shown in FIG. 6 will be described. When an event occurs as described above, the user terminal 100 acquires the video and audio of the user A using the camera 105 and the microphone 106 (step S111). Next, the user A video display area 21a (see FIG. 3) is created (step S112), and the acquired video of the user A is displayed (step S113). Further, the user terminal 100 transmits the video and audio acquired in step S111 to another user terminal 100 (step S114). This is done based on the address obtained above.

  Next, the operation shown in FIG. 7 will be described. When the event occurs as described above, the user terminal 100 receives the video and audio of the user B transmitted from the other user terminal 100 (step S121). Next, the user B video display area 21b (see FIG. 3) is created (step S122), and the received video of the user B is displayed thereon (step S123). Further, the user terminal 100 outputs the sound received in step S121 from the speaker 107 (step S124).

  The operations shown in FIGS. 6 and 7 are repeatedly executed until the above-described predetermined program end command is input. Thereby, video and audio are relayed live between the user terminals 100.

  Next, the operation shown in FIG. 8 will be described. When an event occurs as described above, the user terminal 100 creates each user interface described with reference to FIG. 3 (step S131) and displays it on the display 102 (step S132). Next, the user terminal 100 monitors whether there is an operation input using the keyboard 103 or the mouse 104 (step S133). When there is an operation input from the user (Yes in step S133), the user terminal 100 determines whether or not the input operation is a search word (step S134). If the result of the determination in step S134 is a search word (Yes in step S134), the user terminal 100 executes a search based on this search word (step S135). Next, the user terminal 100 updates the above-described user interface, in particular, the user A log display area 22a, the user B log display area 22b, and the common language log display area 22c based on the translated word obtained by the search and the search word ( Step S136). In addition, the user terminal 100 transmits the input search word and the obtained translated word to another user terminal 100. (Step S137). This is done based on the address obtained above. Then, it returns to step S133 and waits for the next operation. On the other hand, when the operation input from the user is not a search word (No in step S134), the user terminal 100 updates the above-described user interface based on the other input operation (step S138). Is transmitted to the other user terminal 100 (step S139). Then, it returns to step S133 and waits for the next operation.

  Next, the operation shown in FIG. 9 will be described. When an event occurs as described above, the user terminal 100 monitors whether or not a search word and a translation word or other operation information has been received from another user terminal 100 (step S141). When receiving the search word and the translation word or other operation information (Yes in step S141), the user terminal 100 determines whether or not the received information is the search word and the translation word (step S142). As a result of the determination in step S142, if it is a search word and a translated word (Yes in step S142), the user terminal 100 performs the above-described user interface based on the received search word and translated word, particularly the user A log display area 22a, the user B log. The display area 22b and the common language log display area 22c are updated (step S143). Then, it returns to step S141 and waits for the next reception. On the other hand, when the received information is not the search word and the translated word (No in step S142), the user terminal 100 updates the above-described user interface based on the other received operation information (step S144), and then returns to step S141. And wait for the next reception.

  The operations shown in FIGS. 8 and 9 are repeatedly executed until the above-described predetermined program end command is input. Thereby, a user interface that can be shared among the user terminals 100 is realized.

  With the configuration and operation as described above, according to the present embodiment, the user's video and audio can be relayed live between a plurality of user terminals. In addition, according to the present embodiment, the user can know which vocabulary was unclear for the other party, and can further notify which vocabulary is unclear for himself. That is, a factor that hinders smooth communication can be shared with the other party. As a result, this factor can be extracted and solved, and smooth communication can proceed.

  In the present embodiment, the electronic dictionary has been described as an example of the search function. However, the present invention is not limited to this, and various information sources capable of searching for information (also referred to as content) can be applied. . Further, in the present invention, the user interface for the purpose of search is given as an example, but the present invention is not limited to this. For example, there are various user interfaces such as a user interface for inputting a sentence and a user interface for drawing an image. The user interface can be applied. That is, by sharing the user interface that can be operated by the user with other user terminals on the network, it is possible to share the factors that hinder the progress of communication, which is the purpose of the present invention, between users.

  Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Further, the configuration not specifically mentioned is the same as that of the first embodiment.

  FIG. 10 is a diagram showing a schematic configuration of a dialogue support system 1A according to the present embodiment. As illustrated in FIG. 10, the dialogue support system 1 </ b> A includes user terminals 100 a and 100 b and a server 200 connected via a network 20. The network 20 is the same as that in the first embodiment. The user terminals 100a and 100b have the same configuration as in the first embodiment, but have some changes in the functional blocks to be realized.

  The server 200 is an information processing apparatus equipped with a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like, and a general server can be applied.

  Next, functional blocks of the user terminals 100a and 100b and functional blocks of the server 200 according to the present embodiment will be described with reference to the drawings. In addition, since the functional blocks of the user terminals 100a and 100b have the same configuration, these will be described as the user terminal 100A in the following description.

  FIG. 11 is a diagram showing functional blocks of the user terminal 100A. As shown in FIG. 11, the user terminal 100A has the same configuration as the functional block of the user terminal 100 according to the first embodiment, and the search means 13 and the storage means 14 are replaced with request means 13a.

  The requesting unit 13a is a function realized by including the CPU and network adapter of the PC main body 101, requests the server 200 connected via the network 20, and receives the result. The request unit 13 a inputs the translation received as the search result to the interface creation unit 15 and also to the transmission unit 16.

  FIG. 12 is a diagram illustrating functional blocks of the server 200. As illustrated in FIG. 12, the server 200 includes a search unit 13 and a storage unit 14. These are the same as the search means 13 and the storage means 14 shown in the first embodiment (see particularly FIG. 4). That is, in this embodiment, an information source such as an electronic dictionary is provided in another information processing apparatus (server 200) connected via the network 20, and the user terminal 100A requests a search for this. It is configured.

  Next, the operation of the user terminal 100A will be described with reference to the drawings. The operation of the user terminal 100A is the same as that of the user terminal 100 according to the first embodiment, but the operation shown in FIG. 8 is changed. Hereinafter, an operation that replaces the operation of FIG. 8 will be described with reference to FIG.

  As shown in FIG. 13, when an event occurs (see step S104 in FIG. 5), the user terminal 100 creates each user interface described with reference to FIG. 3 (step S231) and displays it on the display 102. (Step S232). Next, the user terminal 100A monitors whether or not there is an operation input using the keyboard 103 or the mouse 104 (step S233). When there is an operation input from the user (Yes in step S233), the user terminal 100A determines whether or not the input operation is a search word (step S234). If the result of the determination in step S234 is a search word (Yes in step S234), the user terminal 100A requests the server 200 to search based on this search word (step S235), and the search result responded to this Is waited for (step S236). Next, the user terminal 100A performs the above-described user interface, particularly the user A log display area 22a, the user B log display area 22b, and the common language log display area based on the translation word and the search word included in the search result received from the server 200. 22c is updated (step S237). In addition, the user terminal 100A transmits the input search word and the received translated word to the other user terminal 100A. (Step S238). This is performed based on the address acquired in step S103 of FIG. Thereafter, the process returns to step S233 and waits for the next operation. On the other hand, when the operation input from the user is not a search word (No in step S234), the user terminal 100A updates the above-described user interface based on the other input operation (step S239). Is transmitted to the other user terminal 100A (step S240). Thereafter, the process returns to step S233 and waits for the next operation.

  The operation shown in FIG. 13 is also repeatedly executed until the above-described predetermined program end command is input. Thereby, a user interface that can be shared between the user terminals 100A is realized.

  With the configuration and operation as described above, according to the present embodiment, the user's video and audio can be relayed live between a plurality of user terminals as in the first embodiment. Further, according to the present embodiment, as in the first embodiment, the user can know which vocabulary was ambiguous for the other party, and can further inform which vocabulary is ambiguous for himself. That is, a factor that hinders smooth communication can be shared with the other party. As a result, this factor can be extracted and solved, and smooth communication can proceed.

  In the present embodiment, the electronic dictionary has been described as an example. However, the present invention is not limited to this, and various information sources having a search function can be provided.

  Next, Example 3 of the present invention will be described in detail with reference to the drawings. In the following description, the same components as those in the first embodiment or the second embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Further, the configuration not specifically mentioned is the same as that of the first embodiment or the second embodiment.

  FIG. 14 is a diagram showing a schematic configuration of a dialogue support system 1B according to the present embodiment. As illustrated in FIG. 14, the dialogue support system 1 </ b> B includes user terminals 100 a and 100 b connected via a network 20, and a search engine 310 and an information site 320 connected thereto via the Internet 30. The network 20 is the same as that in the first embodiment. The user terminals 100a and 100b have the same configuration as in the first embodiment, but have some changes in the functional blocks to be realized. Further, the Internet 30 may be included in the network 20.

  The search engine 310 is a search unit that exists on the Internet 30 and searches the Internet 30 of the corresponding information site 320 based on the search conditions input from the user terminals 100a and 100b. The information site 320 is a storage means that stores various information, and stores current news, technical explanations, and other various information.

  As described above, the present invention can be configured such that a site existing on the Internet is used as an information source, and the search is performed by an engine that is also present on the Internet. Since other configurations are the same as those in the first and second embodiments, description thereof is omitted here. In addition, the functional blocks of the user terminals 100a and 100b are the same as those in the second embodiment, and a description thereof will be omitted here.

  Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same components as those in the first to third embodiments are denoted by the same reference numerals, and detailed description thereof is omitted. Further, the configuration not specifically mentioned is the same as that in any one of the first to third embodiments. However, in the following description, the description will be made with reference to the first embodiment. However, the present embodiment is not limited to the first embodiment, and can be similarly applied to other embodiments.

  The configuration of the dialogue support system according to the present embodiment is the same as that of the dialogue support system 1 (see FIG. 2) according to the first embodiment. The configuration of each user terminal 100a, 100b is the same as that of the first embodiment. However, the functional blocks of the user terminals 100a and 100b have some changes to the functional blocks realized in the same configuration as in the first embodiment.

  Here, functional blocks of the user terminals 100a and 100b according to the present embodiment will be described with reference to the drawings. In addition, since the functional blocks of the user terminals 100a and 100b have the same configuration, these will be described as the user terminal 100C in the following description.

  FIG. 15 is a diagram showing functional blocks of the user terminal 100C. As illustrated in FIG. 15, the user terminal 100 </ b> C has the same configuration as the functional block of the user terminal 100 according to the first embodiment, and a history storage unit 14 c for storing search results as a history is added.

  The history storage unit 14c is a function realized by including the hard disk drive or RAM of the PC main body 101, and stores the search word input to the search result 13 and the search result obtained for this search word. When a search word is input from the input unit 12, the search unit 13 searches the content stored in the storage unit 14 based on this, as in the first embodiment. Next, the search means 13 stores the obtained translated word together with the search word in the history storage means 14c. Thereby, the search history of the user A (B) is accumulated.

  The history stored in the history storage unit 14c can be used in various ways. For example, the interface creation unit 15 may be configured not to display, preferentially display, or display non-prioritized search words and translations stored as history in the history storage unit 14c. However, not limited to this, there are various utilization methods.

  Next, a fifth embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same components as those in the first to fifth embodiments are denoted by the same reference numerals, and detailed description thereof is omitted. Further, the configuration not specifically mentioned is the same as that in any one of the first to fifth embodiments. However, in the following description, the description will be made with reference to the first embodiment. However, the present embodiment is not limited to the first embodiment, and can be similarly applied to other embodiments.

  In this embodiment, the configurations of the dialogue support system and the user terminals 100a and 100b are the same as those in the first embodiment. FIG. 16 shows functional blocks of the user terminals 100a and 100b according to the present embodiment. In addition, since the functional blocks of the user terminals 100a and 100b have the same configuration, these will be described as the user terminal 100D in the following description.

  As shown in FIG. 16, the functional block of the user terminal 100D has the same configuration as that of the first embodiment, and an image processing means 19 is added.

  The image processing unit 19 is a unit for creating a user interface such as a pointer using the input specific video. This means can be realized by using a CPU or a graphic accelerator of the PC main body 101.

  The image processing unit 19 receives the video acquired by the video / audio acquisition unit 11, the video received by the reception unit 17 (including audio), and the user interface generated by the interface generation unit 15. The The image processing unit 19 cuts out a specific video such as a hand or a pen of the user A (or B) from the video input from the video / audio acquisition unit 11 or the reception unit 17. The image processing unit 19 combines the cut-out video with a screen composed of the video input from the video / audio acquisition unit 11 and the reception unit 17 and the user interface image input from the interface creation unit 15. Input to the display means 18. As a result, a screen as shown in FIG. 17 is displayed on the display 102.

  As shown in FIG. 17, on the display 102, the pointer 28a is displayed in the foreground on the same screen as that displayed on the display 102 in the first embodiment (see FIG. 3). In this embodiment, the pointer 28a is an image of the hand of the user A. The pointer 28a moves on the screen in conjunction with the movement of the hand by the user A. As a result, the user A can instruct the user B to use the pointer 28a on a point of interest on the screen.

  Note that the video processing as described above can be realized by using techniques such as chroma key technology and MPEG (Moving Picture Experts Group) -4.

  The above-mentioned Examples 1 to 5 are only examples for carrying out the present invention, and the present invention is not limited to these. Various modifications of these examples are within the scope of the present invention. Furthermore, it is obvious from the above description that various other embodiments are possible within the scope of the present invention.

It is a figure which shows the structure of the system by a prior art. It is a figure which shows schematic structure of the dialogue assistance system 1 by Example 1 of this invention. It is a figure which shows an example of the screen displayed on the display in Example 1 of this invention. It is a figure which shows the functional block of the user terminal 100 by Example 1 of this invention. It is a flowchart which shows operation | movement of the user terminal 100 in Example 1 of this invention (1). It is a flowchart which shows operation | movement of the user terminal 100 in Example 1 of this invention (2). It is a flowchart which shows the operation | movement of the user terminal 100 in Example 1 of this invention (3). It is a flowchart which shows operation | movement of the user terminal 100 in Example 1 of this invention (4). It is a flowchart which shows operation | movement of the user terminal 100 in Example 1 of this invention (5). It is a figure which shows schematic structure of the dialog assistance system 1A by Example 2 of this invention. It is a figure which shows the functional block of 100 A of user terminals by Example 2 of this invention. It is a figure which shows the functional block of the server 200 by Example 2 of this invention. It is a flowchart which shows operation | movement of 100 A of user terminals in Example 2 of this invention. It is a figure which shows schematic structure of the dialog assistance system 1B by Example 3 of this invention. It is a figure which shows the functional block of 100 C of user terminals by Example 4 of this invention. It is a figure which shows the functional block of user terminal 100D by Example 5 of this invention. It is a figure which shows an example of the screen displayed on the display in Example 5 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 1A, 1B Dialogue support system 11 Video / audio acquisition means 12 Input means 13 Search means 13a Request means 14 Storage means 14c History storage means 15 Interface creation means 16 Transmission means 17 Reception means 18 Display means 19 Image processing means 20 Network 21a User A video display area 21b User B video display area 22a User A log display area 22b User B log display area 22c Common language log display area 23a, 23b, 23c Scroll bar 24a, 24c Search button 25a User A use language input field 25b User B Language input field 25c Common language input field 26a, 26c Scroll button 27 Dictionary switching button 28a Pointer 30 Internet 100, 100A, 100a, 100b, 100C, 100D User terminal 01 PC main body 102 Display 103 Keyboard 104 Mouse 105 camera 106 microphone 107 speaker 200 server 310 search engine 320 information site

Claims (8)

An information processing apparatus that has a common user interface with another information processing apparatus connected via a network and realizes an interactive environment between users by exchanging video and audio with the other information processing apparatus. And
Search means for searching for information based on input information to the user interface;
Interface creation means for updating the user interface so as to display search results by the search means;
Transmitting means for transmitting the input information and the search result to the other information processing apparatus;
Receiving means for receiving input information from the other information processing apparatus to the user interface and a search result obtained based on the input information;
The information processing apparatus, wherein the interface creation unit updates the user interface based on input information and search results received from the other information processing apparatus.   The information processing apparatus according to claim 1, wherein the user interface displays a list of the history of the input information or the history of the search results. Having storage means for storing information;
The information processing apparatus according to claim 1, wherein the retrieval unit retrieves information from the storage unit based on the input information.   4. The information processing apparatus according to claim 3, wherein the search unit and the storage unit are electronic dictionaries that search for a translation corresponding to the input information.   The information processing apparatus according to claim 1, further comprising request means for requesting a search based on the input information to another information processing apparatus via the network.   The information processing apparatus according to claim 1, further comprising request means for requesting a search based on the input information to a search engine connected via the network.   The information processing apparatus according to claim 1, wherein the information processing apparatus is connected to the other information processing apparatus via a peer-to-peer network. A communication method for communicating with other users by exchanging video and audio with other information processing devices via a network,
A first step of creating a common user interface with the other information processing apparatus;
A second step of retrieving information based on information input to the user interface;
A third step of updating the user interface to display search results from the second step;
A fourth step of transmitting the input information and the search result to the other information processing apparatus;
A fifth step of receiving input information to the user interface from the other information processing apparatus and a search result obtained based on the input information;
And a sixth step of updating the user interface based on the input information and search results received in the fifth step.

Images