JP2012054640A - Conference terminal device, communication method, and communication program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide conference terminal devices, a communication method, and a communication program which enable sufficient reference to material information shared with the other bases.SOLUTION: Conference terminal devices 3 to 5 share material owned by the respective terminals and can display the material on display units 26. When a speech of a user is detected by a conference terminal device 5 being the user's terminal and a material B provided from a conference terminal device 4 is active, the user is probably speaking referring to the material B. Then, even if an update instruction of the material B is received from the conference terminal device 4, the material B is not to be updated during the speech by the user. As the material B is not to be updated while the user speaks by referring to the material B, the pre-updated material B can be referred sufficiently.

Description

  The present invention relates to a conference terminal device, a communication method, and a communication program that transmit and receive material information to and from conference terminal devices at other bases via a network.

  2. Description of the Related Art Conventionally, a conference terminal device that can perform a video conference between a plurality of sites is known. The conference terminal device transmits and receives audio information and video information to and from conference terminal devices at other bases via a network. The conference terminal device can share and discuss the same material information in each conference terminal device by transmitting the material information to be agenda to other bases. For example, a conference system that automatically detects a speaker and outputs a video used by the speaker to another point is known (for example, see Patent Document 1).

JP 2004-56279 A

  However, the document information provider terminal does not know which material information each site is paying attention to. Therefore, even though the shared material information is referenced at another base, the material information may be updated without permission, which is inconvenient.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a conference terminal device, a communication method, and a communication program that can sufficiently refer to material information shared with other bases.

  The conference terminal device according to the first aspect of the present invention transmits / receives material information to / from conference terminal devices at other bases via a network, displays the material information on a display means, and performs a conference And receiving means for receiving other site material information that is the material information of the other site transmitted from the conference terminal device of the other site, and displaying the other site material information received by the receiving unit. Display control means for displaying on the means, update determining means for determining whether or not the other site material information has been updated in the conference terminal device at the other site, and the other site material information being updated by the update determining unit. If determined, an update unit that updates the other site material information displayed on the display unit, an utterance detection unit that detects an utterance of a user of the conference terminal device at the local site, and The utterance of the user is detected by the attention object information detecting means for detecting the attention object information that is the attention object of the user and the utterance detection means for one or more of the material information displayed on the display means. In addition, when the other base material information is detected as the attention target information by the attention target information detection means, the update interruption for interrupting the update by the updating means for the other base material information detected as the attention target information Means.

  The conference terminal device according to the first aspect is for transmitting / receiving material information to / from conference terminal devices at other bases via a network, sharing the material information, and displaying the information on a display means for a conference. . Other base material information is displayed on the display means of the own base. When the other site material information is updated in the conference terminal device at the other site, the other site material information displayed on the display unit of the own site is usually updated. When the user's utterance is detected by the utterance detection means and the other base material information is detected as the attention target information by the attention target information detection means, it is determined that the user is referring to the other base material information. In such a case, the update interruption means interrupts the update by the update means for the other site material information detected as the attention object information. As a result, the user who has referred to the other site material information can continuously refer to the other site material information before being updated. Therefore, the user can speak by sufficiently referring to the information on other base materials.

  Moreover, in the first aspect, the attention target information detecting means converts the material information that is the operation target of the user among the plurality of material information displayed on the display means to the attention target information. May be detected. The material information that is the operation target at that time can be said to be the material information that the user pays attention to. By using such material information as attention target information, it is possible to interrupt the update of other base material information that the user is paying attention to.

  In the first aspect, the attention object information detection unit is configured to convert the material information displayed in the foreground among the plurality of material information displayed on the display unit to the attention object information. May be detected. The material information that is in an active state at that time can be said to be material information that the user is paying attention to. By using such material information as attention target information, it is possible to interrupt the update of other base material information that the user is paying attention to.

  Further, in the first aspect, the attention target information detection unit includes a gaze direction detection unit that detects a gaze direction of the user, and the material information ahead of the gaze direction detected by the gaze direction detection unit is You may make it detect as said attention object information. The material information ahead of the user's line of sight at that time can be said to be the material information that the user pays attention to. By using such material information as attention target information, it is possible to interrupt the update of other base material information that the user is paying attention to.

  Further, in the first aspect, when the user's utterance detected by the utterance detection unit is ended, or the attention target information detected by the attention target information detection unit is no longer the other base material information. In this case, a release means for releasing the interruption of the update by the update interruption means may be provided. If the other base material information that has been referred to by the user is no longer referred to, it is not necessary to interrupt the update of the other base material information. Therefore, the canceling means cancels the interruption of the update of the other site material information, so that the display of the other site material information at the own site can be updated again in accordance with the update of the other site material information at the other site. it can.

  The communication method according to the second aspect of the present invention includes a conference terminal device that transmits / receives material information to / from a conference terminal device at another site via a network, displays the material information on a display unit, and performs a conference. A communication method to be performed, the receiving step of receiving other site material information which is the material information of the other site transmitted from the conference terminal device of the other site, and the other site material information received in the receiving step Are displayed on the display means, an update determination step for determining whether or not the other site material information has been updated in the conference terminal device at the other site, and the other site material information in the update determination step. When it is determined that the information has been updated, an update step of updating the other site material information displayed on the display means, and the conference terminal device of the own site The utterance detection step for detecting the user's utterance by the utterance detection means for detecting the user's utterance, and the one or more pieces of the material information displayed on the display means are the attention target of the user. Attention target information detection step for detecting attention target information, and when the user's utterance is detected at the utterance detection step, and the other site material information is detected as the attention target information at the attention target information detection step And an update interruption step of interrupting the update by the update step with respect to the other site material information detected as the attention object information.

  The communication method according to the second aspect includes a conference terminal device that performs transmission / reception of material information with a conference terminal device at another base via a network, shares the material information, displays the information on a display unit, and performs a conference. Is what you do. Other base material information is displayed on the display means of the own base. When the other site material information is updated in the conference terminal device at the other site, the other site material information displayed on the display unit of the own site is usually updated. When the user's utterance is detected by the utterance detection means and the other site material information is detected as the target information, the user refers to the other site material information. In such a case, in the update interruption step, the update by the updating unit is interrupted for the other site material information detected as the attention object information. As a result, the user who has referred to the other site material information can continuously refer to the other site material information before being updated. Therefore, the user can speak by sufficiently referring to the information on other base materials.

  The communication program according to the third aspect of the present invention is a communication terminal device for transmitting / receiving material information to / from a conference terminal device at another site via a network, displaying the material information on a display means, and performing a conference. A communication program for controlling the operation, wherein the computer receives the other site material information which is the material information of the other site transmitted from the conference terminal device of the other site, and is received in the receiving step. In a display control step for displaying the other site material information on the display means, an update determination step for determining whether or not the other site material information has been updated in the conference terminal device at the other site, and an update determination step An update step of updating the other site material information displayed on the display means when it is determined that the other site material information has been updated. An utterance detection step of detecting an utterance of the user by an utterance detection unit that detects an utterance of the user of the conference terminal device at the local site, and one or more of the material information displayed on the display unit The attention target information detection step for detecting attention target information that is the target attention of the user, and the utterance of the user is detected in the utterance detection step, and the other site material information is detected in the attention target information detection step. When it is detected as the attention object information, an update interruption step of interrupting the update by the updating step is executed for the other site material information detected as the attention object information.

  A communication program according to the third aspect is provided for a conference terminal device that transmits / receives material information to / from conference terminal devices at other bases via a network, shares the material information, displays the information on a display unit, and performs a conference. What the computer does. Other base material information is displayed on the display means of the own base. When the other site material information is updated in the conference terminal device at the other site, the other site material information displayed on the display unit of the own site is usually updated. When the user's utterance is detected by the utterance detection means and the other site material information is detected as the target information, the user refers to the other site material information. In such a case, in the update interruption step, the update by the updating unit is interrupted for the other site material information detected as the attention object information. As a result, the user who has referred to the other site material information can continuously refer to the other site material information before being updated. Therefore, the user can speak by sufficiently referring to the information on other base materials.

It is a figure which shows the outline | summary of the conference system 1, and the electrical structure of the conference terminal device 5. FIG. It is a figure which shows an example of the display mode of the materials A and B in the conference system. It is a figure which shows the display mode in each base when the material B shown in FIG. 2 is updated to B '. It is a figure which shows the display mode in each base when the data B is updated to the data B 'during the speech of the user P. FIG. It is a conceptual diagram (at the time of update execution) of the update determination table 241. It is a conceptual diagram (at the time of an update interruption) of the update determination table 241. FIG. It is a conceptual diagram (at the time of an update interruption) of the update determination table 241. FIG. 5 is a conceptual diagram of a material management table 242. FIG. It is a main flowchart of a shared material update control process. It is a flowchart of an update execution determination process. It is a flowchart of an utterance determination process. It is a flowchart of an attention material detection process. It is a flowchart of the attention material detection process (1st modification) based on a gaze position. It is a flowchart of the attention material detection process (2nd modification) based on a pointer position.

  Hereinafter, a conference system 1 including a conference terminal device 5 which is an embodiment of a conference terminal device according to the present invention will be described with reference to the drawings. The drawings to be referred to are used for explaining technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like described in the drawings are not intended to be limited to these, but are merely illustrative examples.

  First, the configuration of the conference system 1 will be described with reference to FIG. The conference system 1 includes, for example, conference terminal devices 3, 4, and 5 installed at three bases, respectively. The conference terminal devices 3 to 5 are connected to other terminals via the network 2 to communicate various data such as video and audio, thereby enabling a remote conference. The conference terminal devices 3 to 5 have at least a function (IP telephone function) for executing a call between users at each base. For example, SIP (Session Initiation Protocol) is used as a call control protocol for realizing the IP telephone function. As the conference terminal devices 3 to 5, a personal computer, a portable terminal, or the like can be used in addition to a dedicated terminal.

  Identification information is assigned to each of the conference terminal devices 3 to 5. In SIP, the SIP address corresponds to identification information. For example, “001” is assigned to the conference terminal device 3, “002” is assigned to the conference terminal device 4, and “003” is assigned to the conference terminal device 5. A user designates a SIP address and calls a conference terminal device of a specific user at another site, thereby establishing a communication state in which image information, audio information, and the like can be transmitted and received between the conference terminals. In this embodiment, these SIP addresses are used as terminal IDs as identification information.

  Next, the outline of the material sharing and its updating method in the conference system 1 will be described with reference to FIGS. As shown in FIG. 2, in the conference system 1, materials independently owned by the conference terminal devices 3 to 5 can be shared by all bases and displayed on the display unit 26 of each terminal. For example, a case where the material A owned by the conference terminal device 3 and the material B owned by the conference terminal device 4 are shared by the conference terminal devices 3 to 5 will be described as an example. As shown in FIG. 2, the conference terminal device 3 transmits the image information of the material A to the conference terminal devices 4 and 5. The conference terminal device 4 transmits the image information of the material B to the conference terminal devices 3 and 5. In each of the display units 26 of the conference terminal devices 3 to 5, materials A and B are displayed in a window as shared materials.

  In each window of the materials A and B displayed on the display unit 26 of each base, one window selected by each user is activated. Other windows are inactive. For example, the active material B is displayed in the foreground with respect to the display surface of the display unit 26. The document A in the inactive state is displayed on the back side of the document B. Whether to enter the active state or the inactive state is determined by the operation of the user of each conference terminal device 3-5. In general, the user activates the material he / she wants to refer to and displays it in the foreground with respect to the display surface.

  Update of document A and document B is possible at the conference terminal device of each provider of documents A and B. The content updated by the provider is transmitted as an update command to each conference terminal device to which the materials A and B are provided. Then, it is reflected in the material displayed on the display unit 26. As shown in FIG. 3, for example, in the conference terminal device 4 that is the source of the document B, when the document B is updated to the document B ′, the conference terminal device 4 sends the document B to the conference terminal devices 3 and 5. Update instructions are sent. The update instruction is update instruction information including image information of the material B ′. The conference terminal devices 3 and 5 that have received the update command update the document B to the document B ′ based on the received update command. Therefore, in principle, when a material is updated at the conference terminal device that is the source of the material, the updated content is also reflected at the conference terminal device at the other base receiving the material.

  For example, FIG. 4 illustrates a state in which the user P of the conference terminal device 5 speaks about the material B displayed on the display unit 26 during the conference. The user P speaks while referring to the material B displayed as the operation target at that time. The users of the conference terminal devices 3 and 4 become listeners and listen to the remarks of the user P while watching the material B displayed on each display unit 26.

  Here, it is assumed that the user of the conference terminal device 4 that is the provider of the document B updates the document B to the document B ′. In the conference system 1 according to the present embodiment, when the user is speaking while referring to the shared material provided from another base, the user receives the provision even if the shared material is updated by the provider. It is not updated in the conference terminal device. For example, as shown in FIG. 4, the user P of the conference terminal device 5 receiving the material B is speaking while displaying the material B in the foreground. Therefore, even if the update instruction of the document B is transmitted from the conference terminal device 4 that is the provider of the document B to the conference terminal devices 3 and 5, the conference terminal devices 3 and 5 that receive the provision of the document B Do not update document B. Thereby, the user P can speak while fully referring to the document B before update. Furthermore, even the user of the conference terminal device 3 who is listening to the user P's remarks can sufficiently hear the remarks of the user P by sufficiently referring to the material B before the update.

  Hereinafter, the electrical configuration, control method, and the like of the conference terminal devices 3 to 5 according to the present embodiment will be described in detail.

  First, the electrical configuration of the conference terminal device 5 will be described with reference to FIG. Since the electrical configuration of the conference terminal devices 3 and 4 is the same as that of the conference terminal device 5, description thereof will be omitted. The conference terminal device 5 includes a CPU 21 that controls the conference terminal device 5. The CPU 21 is electrically connected to the ROM 22, RAM 23, hard disk drive (HDD) 24, input unit 25, display unit 26, camera 27, microphone 28, speaker 29, drive device 30, and communication unit 31. The ROM 22 stores a boot program, BIOS, OS, and the like. The RAM 23 stores a timer, a counter, and temporary data.

  Various programs such as a control program for the CPU 21 are stored in the HDD 24. The HDD 24 stores an update determination table 241 (see FIGS. 5 to 7) and a material management table 242 (see FIG. 8), which will be described later. The input unit 25 is a keyboard or a mouse that receives input from the user. The display unit 26 is a display that displays a desired image. The drive device 30 can read information stored in the storage medium 301. For example, when the conference terminal device 5 is set up, the control program stored in the storage medium 301 is read by the drive device 30 and stored in the HDD 24. The communication unit 31 performs timing control when communicating with other conference terminal devices 3 and 4 via the network 2.

  Next, the update determination table 241 stored in the conference terminal device 5 will be described with reference to FIGS. The update determination table 241 is used to determine whether or not to update the shared material at the local site when the shared material provided from another site is updated by the provider. The update determination table 241 stores the terminal ID, utterance information, attention material information, and update determination information of the conference terminal device 5 at its own base, and also includes update determination information received from the conference terminal devices 3 and 4 at other bases. Remembered.

  As the terminal ID, an SIP address is used, but any terminal ID can be used. For example, a Mac address or the like can be used.

  The utterance information is information indicating whether or not the user of the conference terminal device 5 at the local site is uttering. An utterance determination process, which will be described later, for determining whether or not the user is speaking is executed when a material update command is received from another site. In the speech determination process, it is determined whether or not the user is speaking based on the volume of the user's voice collected by the microphone 28. When the utterance is in progress, the information of “utterance” is stored in the update determination table 241. When not speaking, information of “non-uttering” is stored in the update determination table 241. In the update determination table 241, only the utterance information of the conference terminal device 5 at the local site is stored as the utterance information.

  The attention material information is information on whether or not the material that the user of the conference terminal device 5 at the local site is interested in is an updated material updated by the provider of the material. In the present embodiment, a material in an active state on the display unit 26 is set as a material of interest. The material in the active state is specified by a material management table 242 (see FIG. 8) described later. The attention material detection process for detecting that such attention material is an updated material is executed when a material update command is received from another base and a user's utterance is detected.

  If the user is uttering and receives an update command from another base for the material of interest, the material is subject to the update command. Therefore, information of “update material” is stored in the update determination table 241. If an update command has not been received, the data is not subject to the update command. Therefore, information of “non-updated material” is stored in the update determination table 241. This makes it possible to reliably detect the situation in which the user is speaking while referring to the material.

  Note that, in the update determination table 241, only the material of interest information of the conference terminal device 5 at the local site is stored as the material of interest. When the user of the conference terminal device 5 is not speaking, the target material detection process is not executed, so the target material information is not stored. The update determination table 241 shown in FIGS. 5 and 7 is in a state where the user of the conference terminal device 5 is not speaking.

  The update determination information is information on whether or not to update the material in the conference terminal device 5 at the local site in accordance with the update of the material provider. When the update is executed, “execution” information is stored, and when the update is interrupted, “interruption” information is stored. The update determination information may be transmitted from other sites as update information to be described later. In that case, the update determination information included in the update information transmitted from another base is stored in the corresponding part of the update determination table 241. When update information is not transmitted from another base, it is not stored in the update determination table 241.

  The update determination table 241 illustrated in FIG. 5 is an example in the case where no speech is detected in the conference terminal device 5 and update information is not received from another base. “Non-speech” information is stored as utterance information of the conference terminal device 5 at the local site, and “execution” information is stored as update determination information, not stored as attention material information. In this case, the CPU 21 updates the material in the conference terminal device 5 in accordance with the material update by the material provider.

  The update determination table 241 illustrated in FIG. 6 is an example in the case where update information is not received from another base, but an utterance is detected in the conference terminal device 5 and the material of interest is updated. Information of “utterance” is stored as utterance information of the conference terminal device 5, information of “update material” is stored as attention material information, and information of “interruption” is stored as update determination information. In this case, the CPU 21 does not update the material in the conference terminal device 5 regardless of the material update by the material provider.

  The update determination table 241 illustrated in FIG. 7 is an example when the conference terminal device 5 does not detect an utterance but receives update information from the conference terminal device 3 at another base. “Non-speech” information is stored as utterance information of the conference terminal device 5, and “execution” information is stored as update determination information. However, “interruption” information is stored as update determination information of the conference terminal device 3 which is another base. Also in this case, the CPU 21 does not update the material at its own base regardless of the material update by the material provider.

  Next, the material management table 242 will be described with reference to FIG. The material management table 242 manages the active state and inactive state of the window of each material displayed on the display unit 26 at the local site. The material management table 242 stores the window ID, terminal ID, active information, and the like of each material A and B displayed on the display unit 26.

  The window ID is an ID for individually identifying materials A and B displayed on the display unit 26. The window ID is a window handle assigned to each window by the OS when a window for displaying the materials A and B is created by the application. For example, window ID = “101” is assigned to material A. Window ID = “102” is assigned to document B. These “101” and “102” are stored in the material management table 242 as the window IDs of the materials A and B. Thereby, the materials A and B displayed as shared materials on the display unit 26 can be managed. Then, the material identified by the window ID in the active state is recognized as the material currently focused on by the user P at the local site.

  Next, the shared material update control process by the CPU 21 will be described with reference to the flowchart of FIG. In addition, a present Example demonstrates as a process which CPU21 of the conference terminal device 5 performs for convenience of explanation. This process is executed when the “shared material update control program” stored in the HDD 24 is called up when the connection with the conference terminals 3 and 4 is established.

  First, transmission / reception of video information and audio information is performed (S11). As video information, in addition to video captured by the camera 27, various materials (documents, diagrams, graphs, tables, etc.) stored in the HDD 24 are applicable. The sound collected by the microphone 28 corresponds to the sound information. Then, the video information received from another base is displayed on the display unit 26, and the audio information is output through the speaker 29. Thus, the video conference is executed in the conference terminal devices 3 to 5.

  Next, it is determined whether or not the image information of the shared material is received from another base (S12). When the image information of the shared material is not received (S12: NO), it is determined whether or not the conference is ended (S22). If the conference is finished (S22: YES), the process is finished as it is. If the conference has not ended yet (S22: NO), the process returns to S11 and the process is repeated.

  On the other hand, when the image information of the shared material is received (S12: YES), the received shared material is displayed on the display unit 26 (S13). Further, it is determined whether or not a material update command is received from the conference terminal device that is the provider of the shared material (S14). When the document update command is not received (S14: NO), it is not necessary to update the shared material, and it is determined whether or not the conference is ended (S22). When a document update command is received from the document provider (S14: YES), the conference terminal device 5 executes an update execution determination process to determine whether or not to update the document. S15).

  The update execution determination process will be described with reference to the flowchart of FIG. First, “execution” information is stored in the update determination information of the update determination table 241 stored in the HDD 24 (S31). Next, an utterance determination process is executed to determine whether or not the user P of the conference terminal apparatus 5 is speaking (S32).

  Here, the speech determination process will be described with reference to the flowchart of FIG. First, the volume level collected by the microphone 28 is acquired (S40). It is determined whether or not the acquired volume level is greater than or equal to a threshold value (S41). If the volume level is greater than or equal to the threshold (S42: YES), it is then determined whether or not the utterance information in the update determination table 241 is “non-utterance” (S42). If the utterance is started, the utterance information in the update determination table 241 is “non-utterance”. In this case (S42: YES), “utterance” is stored in the utterance information of the update determination table 241 (S43). Then, the speech determination process ends, and the process proceeds to S33 of the update execution determination process shown in FIG. If “utterance” is already stored as utterance information in the update determination table 241 in the previous determination (S42: NO), the utterance determination process is terminated as it is, and the update execution determination shown in FIG. The process proceeds to S33 of the process.

  On the other hand, when the acquired volume level is less than the threshold (S41: NO), it is determined whether or not the utterance information in the update determination table 241 is “utterance” (S44). If “speech” information is stored as utterance information (S44: YES), it is determined whether or not the timer counter for measuring the elapsed time during non-speech is in operation (S45). If the timer counter is not operating (S45: NO), the timer counter is started (S49). Then, the speech determination process ends, and the process proceeds to S33 of the update execution determination process shown in FIG.

  On the other hand, when the timer counter is operating (S45: YES), it is determined whether the elapsed time during non-speech is equal to or longer than a predetermined time (S46). If the elapsed time during non-speech is less than the predetermined time (S46: NO), the user's speech may not be finished yet. In this case, the speech determination process ends with the timer counter operated, and the process proceeds to S33 of the update execution determination process shown in FIG.

  If the elapsed time during non-speech is equal to or longer than the predetermined time (S46: YES), the user's non-speech state continues for a long time, and the user's speech is highly likely to end. In this case, it is determined that the previous utterance has been completed, and “non-utterance” information is stored as utterance information in the update determination table 241 (S47). Further, the timer counter is stopped and reset (S48). Thus, the speech determination process ends, and the process proceeds to S33 of the update execution determination process shown in FIG.

  If the acquired volume level is less than the threshold value (S41: NO) and “non-utterance” is stored as the utterance information in the update determination table 241 (S44: NO), the utterance determination process ends as it is. The process proceeds to S33 of the update execution determination process shown in FIG.

  Next, in S33 shown in FIG. 10, it is determined whether or not the utterance information stored in the update determination table 241 is “utterance”. When the information of “utterance” is stored (S33: YES), since the user P is speaking in the conference terminal device 5, subsequently, in order to detect the state of the material focused on by the user P The attention material detection process is executed (S34). On the other hand, when “non-utterance” information is stored as the utterance information in the update determination table 241 (S33: NO), the process proceeds to S16 shown in FIG. 9 without performing the attention material detection process (S34).

  Here, the attention material detection process will be described with reference to the flowchart of FIG. First, a material management table 242 is created in order to manage the materials A and B currently displayed on the display unit 26 (S50). For example, as shown in FIG. 4, materials A and B are displayed on the display unit 26 as shared materials. Since the document A is in an inactive state and the document B is in an active state, the document B is displayed on the forefront and the document A is displayed on the back thereof. The image information of these materials A and B is transmitted from the conference terminal devices 3 and 4, respectively. As shown in the material management table 242 of FIG. 8, the terminal ID of the provider is attached to these image information. Therefore, the conference terminal device 5 that has received the image information can recognize which conference terminal device the material is provided from.

  As described above, the window ID is assigned to each window of the materials A and B by the OS. Therefore, it is possible to detect which of the materials A and B is active. These pieces of information are registered in the material management table 242. For example, when the windows of the materials A and B are created on the display unit 26, as shown in FIG. 8, the materials A and B are in an active state together with their window ID and terminal ID, or The inactive state is registered in the material management table 242. Thereby, the attention material of the user P can be specified among each material currently displayed on the display part 26. FIG.

  Next, based on the created material management table 242, it is determined whether or not the material currently in the active state is the material that is the target of the update command (S51). As shown in FIG. 4, when the material B in the active state is the target of the update command (S51: YES), the information of “update material” is stored in the attention material information of the update determination table 241 (see FIG. 6). (S52). In this way, the target material detection process ends, and the process proceeds to S35 of the update execution determination process shown in FIG. On the other hand, when the material B in the active state is not the target of the update command (S51: NO), the information of “non-updated material” is stored as the attention material information of the update determination table 241 (S53). And it transfers to S35 of the update execution determination process shown in FIG.

  Subsequently, in S35 shown in FIG. 10, it is determined whether or not the attention material information is “updated material”. When the information of “update material” is stored (S35: YES), the information of “suspend” is stored as the update determination information of the update determination table 241 (S36). Then, the update execution determination process ends, and the process proceeds to S16 shown in FIG. On the other hand, when the information of “non-updated material” is stored (S35: YES), the process proceeds to S16 shown in FIG.

  Next, in S <b> 16 shown in FIG. 9, it is determined whether or not the utterance information in the update determination table 241 is “utterance”. If it is “utterance” (S16: YES), the update information about the material B is transmitted to the conference terminal device 3 which is a terminal other than the source of the material B that has transmitted the update command (S17). The update information is information including the update determination information of the conference terminal device 5 stored in the update determination table 241 and the terminal ID information of the conference terminal device 4 that provided the target material B of the update determination information. . The terminal ID information is acquired from the material management table 242 (FIG. 8). In the example illustrated in FIG. 4, since the conference terminal device 4 is the provider of the material B that has transmitted the update command, update information is not transmitted to the conference terminal device 4. If no utterance is detected by the conference terminal device 5, the update information is not transmitted to other bases. Therefore, the update determination information transmitted to other bases is only “interrupt” information.

  Further, it is determined whether update information has been received from another base (S18). When update information is received from another base (S18: YES), it is subsequently determined whether the target material of the update determination information included in the update information is in an active state in the conference terminal device 5 (S19). Here, the material management table 242 (see FIG. 8) is referred to. Then, for example, the terminal ID information of the material B in the active state is collated with the terminal ID information included in the received update information. If they match, the target material of the update determination information is in the active state (S19: YES), so the update determination information included in the received update information is stored in the update determination table 241 (S20). For example, when the update information about the material B is received from the conference terminal device 3, the information of “interruption” is stored in the update determination information of the conference terminal device 3 in the update determination table 241 (see FIG. 7).

  On the other hand, when the target material of the update determination information is not in the active state (S19: NO), in the conference terminal device 5, the update information is about a material different from the material B in the active state. In this case, the update determination table 241 does not store the update determination information, and proceeds to the next step S21. In addition, also when not receiving update information from another base (S18: NO), it does not do anything similarly and progresses to the next step of S21.

  Next, in the update determination table 241, it is determined whether or not there is a base whose information is “suspended” in the update determination information (S21). For example, in the example shown in FIG. 5, since there is no base for “suspend” information (S21: NO), the material update process is executed based on the update command received in S14 (S23). Then, it is determined whether or not the conference is finished (S22). If the conference is finished (S22: YES), the process is finished as it is. If the conference has not ended yet (S22: NO), the process returns to S11 and the process is repeated.

  In the example shown in FIG. 6, since the update determination information of the conference terminal device 5 is “suspend” information (S21: YES), the material update process is not executed regardless of the update command received in S14. As described above, when the update information is received from the conference terminal device at another base, the update determination information of the update information is displayed in the update determination table 241 only when the conference terminal device 5 targets the active material. Remembered. Thereby, for example, even when update information of a material different from the material B is received from the conference terminal device 3, the material B in the active state in the conference terminal device 5 is not updated by the update information. Therefore, the user P of the conference terminal device 5 can refer to the document B as it is, and can continue to say about the document B to the end.

  In the example shown in FIG. 7, the update determination information of the conference terminal device 3 that is another base is “interruption” information (S21: YES), so that the material update process is performed regardless of the update command received in S14. Is not executed. For example, when the user of the conference terminal device 3 speaks about the active material B, and the user P of the conference terminal device 5 activates the material B and listens to the speech, the user P Can hear the remarks of the user of the conference terminal device 3 while referring to the material B before the update.

  Further, as described above, in S17, the update information about the material B transmitted from the conference terminal device 5 at the local site to the other site is the conference terminal device 3 that is the source of the material B that is the target of the update information. Will not be sent. As a result, the conference terminal device 3 can freely update the material B owned by the own site to the material B ′ without being affected by the utterance at the other site.

  Then, it is determined whether or not the conference is finished (S22). If the conference is finished (S22: YES), the process is finished as it is. If the conference has not ended yet (S22: NO), the process returns to S11 and the process is repeated.

  By the way, when the user P speaks in the conference terminal device 5 and the updating of the material B is interrupted, the user B needs to update the material B to B ′. In this case, in the utterance determination process shown in FIG. 11, the elapsed time since the “utterance” information is stored in the utterance information of the update determination table 241 is acquired (S44). When the elapsed time is equal to or longer than the predetermined time (S45: YES), the previous utterance is finished. In such a case, “non-utterance” information is stored in the utterance information of the update determination table 241 (S46).

  Then, the process proceeds to S33 in FIG. 10, and since the utterance information stored in the update determination table 241 is not the “utterance” information (S33: NO), the process returns to S16 in FIG. Similarly, since the utterance information is not “utterance” (S16: NO), if no update information is received from another site in S18 (S18: NO), there is no information base whose update determination information is “suspended”. In this case (S21: NO), the update process is executed (S23), so the update process that has been interrupted is canceled. Therefore, when the user P finishes speaking, the material B can be updated to B ′.

  In the above description, the CPU 21 that executes S11 of FIG. 9 corresponds to the “reception unit” of the present invention, the CPU 21 that executes the process of S13 corresponds to the “display control unit” of the present invention, and performs the process of S14. The CPU 21 to execute corresponds to the “update determination unit” of the present invention, the CPU 21 to execute the process of S23 corresponds to the “update unit” of the present invention, and the CPU 21 to execute the process of S32 corresponds to the “utterance detection unit” of the present invention. The CPU 21 that executes the process of S34 in FIG. 10 corresponds to the “attention target information detection means” of the present invention, and the CPU 21 that executes the process of S36 in FIG. 10 and S21: YES in FIG. Corresponds to “update interruption means”.

  As described above, the conference terminal devices 3 to 5 of the present embodiment constitute the conference system 1. The conference terminal apparatuses 3 to 5 can share materials owned by the terminals with each other and display them on the display unit 26. When the user's utterance is detected at his / her base and the shared material provided from another base is in an active state, the user is likely to speak by referring to the shared material. Therefore, even when a shared material update command is received from another site, the shared material is not updated while the user is speaking. As a result, when the user makes a speech referring to the material, the user can refer to the material before the update at least until the speech is ended, regardless of the update instruction of the provider. In this case, update information including update determination information that does not update the shared material is transmitted to another site. Other sites also do not update shared materials based on the received update information update determination information. Therefore, the user at the other base who is the listener can also listen to the user's remark based on the material before update while referring to the material before update.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation is possible. For example, in the above-described embodiment, the material in the active state is regarded as the material that the user is paying attention to in the attention material detection in FIG. Alternatively, for example, the user's line-of-sight direction may be detected by the camera 27, and the material at the line-of-sight position on the display unit 26 may be regarded as the material that the user is paying attention to. Furthermore, the material at the pointer position on the display unit 26 by operating the mouse may be regarded as the material that the user is paying attention to. Therefore, the former will be described in turn as a first modification and the latter as a second modification.

  First, the first modification will be described. Various detection methods for the line-of-sight direction can be applied. For example, Takehiko Ohno, Gaze-based interface, information processing, Vol. 44, no. 7, pp. In non-patent documents of Nos. 726-732 and 2003, as a line-of-sight measurement method using near infrared rays, a true pupil is obtained from a three-dimensional model relating to the shape of a human eyeball, and observed pupil images and Purkinje images using an infrared camera. A method of estimating the position and the position of the Purkinje image and calculating the line-of-sight direction is shown.

  For example, Japanese Patent Laid-Open No. 2000-138872 detects the contours of both eyes from image data obtained by photographing a user, and determines the center position and radius of the eyeball from the geometrical positional relationship between the ends of the eyes and the nostrils. A method for estimating and detecting the gaze direction is shown.

  The attention material detection process in consideration of the line-of-sight direction will be described with reference to the flowchart of FIG. This process is a modification of the noted material detection process of the above embodiment, and is executed by the CPU 21. First, the user's line-of-sight direction is detected (S60). Here, the face direction centering on the user's eyes is captured by a camera, and the line-of-sight direction is detected by analyzing the obtained image. The line-of-sight position on the display unit 26 is detected from the detected line-of-sight direction. Next, it is determined whether or not there is a document displayed by the window on the detected line-of-sight position (S61). If there is no material on the line-of-sight position (S61: NO), the user is not paying attention to any material. In this case, information of “non-updated material” is stored as attention material information in the update determination table 241 (S65). And it transfers to S35 of the update execution determination process shown in FIG.

  On the other hand, if there is a document on the line-of-sight position (S61: YES), the currently viewed document is detected (S62). For example, when there is a material B on the line-of-sight position, information (identification information, position information, etc.) on the material B is detected. Further, it is determined whether or not the detected material is updated (S63). When the material on the line-of-sight position is the target of the update command and is updated (S63: YES), the information on “updated material” is stored as the material of interest in the update determination table 241 (S64). If it is not the target of the update command and is not updated (S63: NO), the information of “non-updated material” is stored as the material of interest in the update determination table 241 (S65). And it transfers to S35 of the update execution determination process shown in FIG. Even in such a first modification, the same function as the processing of the above embodiment can be exhibited.

  Next, a second modification will be described. The user can perform input operations such as selection and determination by moving the position of a pointer (for example, an arrow mark) displayed on the display unit 26 using a mouse (not shown) and clicking at a predetermined position.

  The attention material detection process in consideration of the operation of the mouse will be described with reference to the flowchart of FIG. This process is a modification of the noted material detection process of the above embodiment, and is executed by the CPU 21. First, the mouse pointer position is detected (S70). When there is a material that the user does not want to be updated by an operation at another site, the user moves the pointer over the material. Next, it is determined whether or not there is a material displayed by the window on the detected pointer position (S71). If there is no material on the pointer position (S71: NO), information of “non-updated material” is stored as attention material information in the update determination table 241 (S75). And it transfers to S35 of the update execution determination process shown in FIG.

  On the other hand, when there is a material on the pointer position (S71: YES), the material on the pointer position is detected (S72). For example, when there is a material B on the pointer position, information (identification information, position information, etc.) of the material B is detected. Further, it is determined whether or not the detected material is updated (S73). When the material on the pointer position is the target of the update command and is updated (S73: YES), the information of “updated material” is stored as attention material information of the update determination table 241 (S72). If it is not the target of the update command and is not updated (S73: NO), the information of “non-updated material” is stored as the material of interest in the update determination table 241 (S75). And it transfers to S35 of the update execution determination process shown in FIG. Even in such a second modification, the same function as the processing of the above embodiment can be exhibited.

  In addition to these first and second modifications, the present invention can be modified in various ways. For example, in the present embodiment, the conference system 1 including three conference terminal devices 3 to 5 has been described. However, the number of conference terminal devices may be larger or smaller.

1 Conference System 3-5 Conference Terminal Device 21 CPU
24 HDD

Claims (7)

A conference terminal device that transmits / receives material information to / from conference terminal devices at other bases via a network, displays the material information on a display unit, and performs a conference.
Receiving means for receiving other site material information which is the material information of the other site transmitted from the conference terminal device of the other site;
Display control means for displaying the other site material information received by the receiving means on the display means;
Update determination means for determining whether or not the other site material information has been updated in the conference terminal device of the other site;
When it is determined that the other site material information has been updated by the update determination unit, an update unit that updates the other site material information displayed on the display unit;
Utterance detection means for detecting the utterance of the user of the conference terminal device at the local site;
Attention object information detection means for detecting attention object information that is an attention object of the user for one or more of the material information displayed on the display means;
When the user's utterance is detected by the utterance detection unit and the other base material information is detected as the attention target information by the attention target information detection unit, the other base detected as the attention target information A conference terminal device comprising: an update interruption unit that interrupts the update of the document information by the update unit.   The attention object information detection means detects the material information that is the operation target of the user as the attention object information from among the plurality of material information displayed on the display means. The conference terminal device according to claim 1.   The attention object information detecting means detects the material information in the active state displayed in the forefront among the plurality of material information displayed on the display means as the attention object information. The conference terminal device according to claim 1 or 2. The attention object information detecting means includes a gaze direction detecting means for detecting a gaze direction of the user,
The conference terminal device according to claim 1, wherein the material information ahead of the line-of-sight direction detected by the line-of-sight direction detection unit is detected as the attention target information.   When the utterance of the user detected by the utterance detection unit is completed, or when the attention target information detected by the attention target information detection unit is no longer the other base material information, the update interruption unit The conference terminal apparatus according to claim 1, further comprising release means for canceling the update interruption. It is a communication method performed by a conference terminal device that performs transmission and reception of material information with a conference terminal device at another site via a network, displays the material information on a display unit, and performs a conference
Receiving step of receiving other site material information that is the material information of the other site transmitted from the conference terminal device of the other site;
A display control step of displaying the other site material information received in the receiving step on the display means;
An update determination step for determining whether or not the other site document information has been updated in the conference terminal device at the other site;
If it is determined in the update determination step that the other site material information has been updated, an update step of updating the other site material information displayed on the display means;
An utterance detection step of detecting an utterance of the user by an utterance detection means for detecting an utterance of the user of the conference terminal device at the local site;
Attention object information detection step for detecting attention object information that is an object of attention of the user for one or more of the material information displayed on the display means;
When the user's utterance is detected in the utterance detection step and the other base material information is detected as the attention target information in the attention target information detection step, the other base detected as the attention target information A communication method, comprising: an update interruption step for interrupting the update of the document information by the update step. A communication program for transmitting / receiving material information to / from a conference terminal device at another site via a network, controlling the operation of the conference terminal device for displaying the material information on a display unit and performing a conference,
On the computer,
Receiving step of receiving other site material information that is the material information of the other site transmitted from the conference terminal device of the other site;
A display control step of displaying the other site material information received in the receiving step on the display means;
An update determination step for determining whether or not the other site document information has been updated in the conference terminal device at the other site;
If it is determined in the update determination step that the other site material information has been updated, an update step of updating the other site material information displayed on the display means;
An utterance detection step of detecting an utterance of the user by an utterance detection means for detecting an utterance of the user of the conference terminal device at the local site;
Attention object information detection step for detecting attention object information that is an object of attention of the user for one or more of the material information displayed on the display means;
When the user's utterance is detected in the utterance detection step and the other base material information is detected as the attention target information in the attention target information detection step, the other base detected as the attention target information An update interruption step for interrupting the update in the update step for the document information is executed.

Images