JP2011228817A - Terminal device, monitoring device, terminal monitoring system, method for controlling terminal device, method for controlling monitoring device, control program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable confirming whether or not a user of a terminal device views an image on a display device of the terminal device, the image being displayed by a device for displaying an image to be synchronously displayed.SOLUTION: A participant terminal 1b having a synchronous mode for displaying a target image on a terminal display part 11 of the participant terminal, the target image being displayed by a presenter terminal 1a for displaying an target image to be synchronously displayed, comprises a line-of-sight detecting part 20 for determining whether or not a line of sight of a user of the participant terminal is directed to the terminal display part 11 of the participant terminal, a terminal conference control part 17 for determining whether or not the participant terminal is in the synchronous mode, and a participation state determination part 22 for transmitting information indicating that a synchronous seminar is being performed where the user views the target image in the synchronous mode to a conference server 2 for monitoring a participation state of the user of the participant terminal in a case where the line-of-sight detecting part 20 has determined that the line of sight of the user is directed to the terminal display part 11 of the participant terminal and the terminal conference control part 17 has determined that the participant terminal is in the synchronous mode.

Description

  The present invention relates to a terminal device or the like that displays an image displayed by a target image display device that displays a target image to be subjected to synchronous display on the display device of the device itself.

  In an electronic conference system in which a terminal device such as a personal computer is connected via a network to perform an electronic conference, the same image as the image displayed by the terminal device assigned to the conference presenter (hereinafter referred to as the presenter terminal) A device having a function of displaying on each terminal device (hereinafter referred to as a participant terminal) assigned to each participant is known.

  In such an electronic conference system, when the image is switched at the presenter terminal, the image is similarly switched at the participant terminal. Therefore, each participant confirms the image displayed on the presenter terminal at the participant terminal. Can participate in meetings.

  In addition, as a prior art regarding an electronic conference system, the following patent document 1 is mentioned, for example. Patent Document 1 describes an electronic conference system in which cameras are installed at a plurality of bases to photograph participants at the respective bases, and the captured images are transmitted to other bases. As described above, many electronic conference systems enable a conference between remote bases.

  Specifically, Patent Document 1 describes that a gaze position of a participant with respect to a display screen is detected, and a region of a video that the participant will gaze at is predicted from the detected gaze position. . Then, it is described that the video in the predicted area is compressed at a lower compression rate than other areas, and the compressed video is transmitted to the participant.

JP 2006-54830 A (published February 23, 2006)

  In an electronic conference system that synchronizes (links) the image displayed on the presenter terminal and the image displayed on the participant terminal, the presenter recognizes whether each participant is viewing the image displayed on the presenter terminal. There is a problem that it is difficult.

  For example, if the location where the presenter is located is different from the location where the participant is located, it is difficult for the presenter to determine whether the participant is viewing the image displayed on the participant terminal. It is.

  Here, for example, by using the technique for detecting the line-of-sight position on the display screen of the participant described in Patent Document 1, it is determined whether or not the participant is viewing the image displayed on the participant terminal. Is considered possible.

  However, in the technique of Patent Document 1, it is possible to determine which region of the displayed image the participant is viewing or not viewing the displayed image. Whether or not the image to be displayed is being viewed cannot be determined. This is because the participant may cause the participant terminal to display an image different from the image displayed on the presenter terminal.

  This is not a problem limited to the participant terminal of the electronic conference system, but is common to the terminal device that displays the image displayed on the display device of the own device on the display device of the own device. This is a problem that occurs.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to display an image displayed by a device that displays an image to be synchronized and displayed by the user of the terminal device on the terminal device. The object is to realize a terminal device or the like that makes it possible to specify whether or not a device is watching.

  In order to solve the above-described problem, the terminal device according to the present invention provides a synchronous display in which a target image displayed by a target image display device that displays a target image to be synchronized is displayed on the display device of the device itself. A terminal device having a mode, and a line-of-sight determining means for determining whether or not the user's line of sight of the device is directed to the display device of the device, and determining whether or not the device is in the synchronous display mode Information indicating the synchronization determination unit, the determination result of the line-of-sight determination unit, and the determination result of the synchronization determination unit is transmitted to the monitoring device that identifies the viewing state of the image displayed on the display device of the user of the device itself. It is characterized by comprising browsing state notification means.

  Moreover, in order to solve the said subject, the control method of the terminal device which concerns on this invention sets the target image which the target image display apparatus which displays the target image used as the object of synchronous display to the display apparatus of an own apparatus. A control method of a terminal device having a synchronous display mode for displaying, wherein a line-of-sight determination step for determining whether or not a user's line of sight of the own device is facing the display device of the own device, and the own device is in the synchronous display mode The state of viewing the image displayed on the display device of the user of the user's own device with the information indicating the determination result of the synchronization determination step for determining whether or not there exists, the determination result of the line-of-sight determination step, and the determination result of the synchronization determination step And a browsing state notifying step for transmitting to a monitoring device for identifying the device.

  According to said structure, the information which shows the determination result whether a user's eyes | visual_axis is suitable for the display apparatus, and the information which shows whether a terminal device is a synchronous display mode are transmitted to a monitoring apparatus.

  Therefore, it is possible to make the monitoring device specify whether or not the user of the terminal device is viewing the target image displayed by the target image display device on the display device of the terminal device. Note that the display device of the terminal device is only required to be connected to the terminal device and display an image in accordance with the control of the terminal device. A device independent of the device may be used.

  Further, the determination as to whether or not the user's line of sight faces the display device of the terminal device may be performed by the terminal device alone or jointly with other devices. For example, in the line-of-sight determination means or the line-of-sight determination step, the determination may be made by acquiring an image obtained by photographing the user's face from another device (such as a camera) and analyzing the image. Further, for example, information indicating the direction in which the user's line of sight is directed may be acquired from another apparatus (such as an apparatus that detects the user's line of sight), and the determination may be performed from the acquired information. Further, for example, it is determined whether or not the user's line of sight is directed to the display device by causing another device to determine whether or not the user's line of sight is directed to the display device and obtaining the determination result. May be.

  Similarly, the determination of whether or not the terminal device is in the synchronous display mode may be performed by the terminal device alone or jointly with other devices. For example, when the terminal device shifts to the synchronous display mode under the control of the own device, it may be determined whether or not it is the synchronous display mode based on whether or not the control to shift to the synchronous display mode is performed. Further, for example, when the terminal device shifts to the synchronous display mode under the control of another device, the synchronous display mode is checked by checking whether or not the other device has been controlled to shift to the synchronous display mode. It may be determined whether or not.

  Further, the browsing state notifying unit determines that the line-of-sight determining unit determines that the user's line of sight is facing the display device of the user apparatus, and the synchronization determining unit determines that the synchronous display mode is set. In this case, it is preferable that information indicating that the user is in the first browsing state browsing the target image in the synchronous display mode is transmitted to the monitoring device.

  According to said structure, since the information which shows that it is the 1st browsing state which the user is browsing the object image in synchronous display mode is transmitted to a monitoring apparatus, the monitoring apparatus is special, such as judgment of a browsing state. It is possible to grasp that the user of the terminal device is browsing the target image in the synchronous display mode without performing the process.

  In addition, the browsing state notifying unit is a non-browsing state in which the user does not look at the display device of the user apparatus when the line-of-sight determination unit determines that the user's line of sight is not directed to the display device of the user device. It is preferable to transmit information indicating that the information is to the monitoring device.

  According to the above configuration, since the information indicating the non-browsing state is transmitted to the monitoring device, the monitoring device allows the user of the terminal device to display the display device without performing special processing such as determination of the browsing state. You can grasp what you are not watching.

  Further, the synchronization determination means is an asynchronous display mode in which the own device is in the synchronous display mode or the own device can display an image different from the target image displayed on the target image display device on the display device of the own device. The viewing state notifying means determines that the line-of-sight determining means determines that the user's line of sight is facing the display device of the own apparatus, and the synchronization determining means is the asynchronous display mode. If it is determined that the information is in the second browsing state in which the user is browsing the image in the asynchronous display mode, it is preferable to transmit information to the monitoring device.

  According to the above configuration, when it is determined that the user's line of sight is directed to the display device of the own device and it is determined that the user is in the asynchronous display mode, the user views the image in the asynchronous display mode. Is transmitted to the monitoring device.

  Therefore, the user of the terminal device can cause the monitoring device to specify that the user does not see the image in the synchronous display mode while viewing the image on the display device of the terminal device.

  The target image display device displays any one of a plurality of predetermined target images, and when the device is in the asynchronous display mode, the target image display device displays any of the plurality of target images. Asynchronous state determination means for determining whether or not the asynchronous first state is displayed on the display state, the browsing state notification means is configured such that the line-of-sight determination means indicates that the user's line of sight is directed to the display device of the own device. Information indicating that the user is in the third browsing state in which the user is browsing the target image in the asynchronous display mode when the asynchronous state determination unit determines that the asynchronous first state is determined. It is preferable to transmit to the monitoring device.

  According to the above configuration, it is determined that the user's line of sight is directed to the display device of the own device, and in the asynchronous display mode, any of the plurality of target images displayed by the target image display device is determined. When it is determined that the image is displayed on the display device, information indicating that the user is in the third viewing state viewing the target image in the asynchronous display mode is transmitted to the monitoring device.

  Therefore, although the user of the terminal device is in the asynchronous display mode, the monitoring device specifies that one of the plurality of target images displayed by the target image display device is viewing the image on the display device of the terminal device. be able to.

  The target image display device displays any one of a plurality of predetermined target images, and when the device is in the asynchronous display mode, the target image display device displays any of the plurality of target images. An asynchronous state determination means for determining whether the first state is an asynchronous first state displayed on the display device or an asynchronous second state in which the target image is not displayed on the display device of the own device, and the browsing state notification means includes When the line-of-sight determination means determines that the user's line of sight is directed to the display device of the user apparatus, and the asynchronous state determination means determines the asynchronous second state, the user selects the target image. It is preferable to transmit information indicating the fourth browsing state that is not being browsed to the monitoring device.

  According to the above configuration, it is determined that the user's line of sight is directed to the display device of the own device, and it is determined that the target image is not displayed on the display device of the own device in the asynchronous display mode. In this case, information indicating that the user is in the fourth browsing state where the target image is not browsed is transmitted to the monitoring device.

  Therefore, the user of the terminal device can cause the monitoring device to specify that he / she is viewing the image on the display device of his / her terminal device but is not viewing the target image.

  In addition, the terminal device calculates a ratio of a period during which the user of the device is viewing the target image in the first browsing state in a period during which the target image display device displays one target image. It is preferable to provide a ratio calculating means.

  According to said structure, the ratio of the period when the user of an own apparatus is browsing the said target image in a 1st browsing state among the periods when the target image display apparatus is displaying one target image is calculated. In the first browsing state, since the user of the device does not look away from the display device or see other images, the above ratio can be used as an index for evaluating the target image. it can. For example, a target image with a high ratio can be evaluated as having a high degree of user attention, and a target image with a low ratio can be evaluated as having a low degree of user attention.

  In order to solve the above-described problem, the monitoring device according to the present invention displays the target image displayed by the target image display device that displays the target image to be synchronized and displayed on the display device of the device itself. It is a monitoring device for specifying a viewing state of an image displayed on the display device of a user who uses the terminal device having the synchronous display mode, and whether or not the user's line of sight faces the display device of the terminal device. Line-of-sight determination means for determining whether the terminal apparatus is in the synchronous display mode, and line-of-sight determination means for determining whether the user's line of sight faces the display device of the terminal apparatus. And when the synchronization determination means determines that the terminal device is in the synchronous display mode, the user browses the target image in the synchronous display mode. It is characterized by comprising a determining viewing state determining means that the state.

  And in order to solve the said subject, the control method of the monitoring apparatus which concerns on this invention displays the target image which the target image display apparatus which displays the target image used as the object of synchronous display displays the display apparatus of an own apparatus. A method for controlling a monitoring device for monitoring a viewing state of an image displayed on a display device of a user who uses a terminal device having a synchronous display mode to be displayed on the display device, wherein the user's line of sight is the display device of the terminal device A line-of-sight determination step for determining whether or not the terminal device is in the synchronous display mode, and a line-of-sight determination step for determining whether or not the terminal device is in the synchronous display mode. If the terminal is determined to be suitable for a device and the terminal device is determined to be in the synchronous display mode in the synchronization determination step, the user is in the synchronous display mode. Serial is characterized by comprising a viewing condition determining step determines that the first access state viewing a target image.

  According to said structure, when it determines with a user's eyes | visual_axis facing the display apparatus, and it determines with it being a synchronous display mode, the user is browsing the object image in synchronous display mode. It determines with it being a 1st browsing state.

  Therefore, it is possible to specify whether or not the user of the terminal device is viewing the target image displayed by the target image display device on the display device of his / her terminal device. Note that the display device of the terminal device is only required to be connected to the terminal device and display an image in accordance with the control of the terminal device. A device independent of the device may be used.

  Further, whether or not the user's line of sight is directed to the display device of the terminal device may be determined by the monitoring device alone or jointly with other devices. For example, in the line-of-sight determination means or the line-of-sight determination step, the determination may be made by acquiring an image obtained by photographing the user's face from another device (such as a camera) and analyzing the image. Further, for example, information indicating the direction in which the user's line of sight is directed may be acquired from another apparatus (such as an apparatus that detects the user's line of sight), and the determination may be performed from the acquired information. Further, for example, it is determined whether or not the user's line of sight is directed to the display device by causing another device to determine whether or not the user's line of sight is directed to the display device and obtaining the determination result. May be.

  Similarly, the determination as to whether or not the terminal device is in the synchronous display mode may be performed by the monitoring device alone or jointly with other devices. For example, when the terminal device shifts to the synchronous display mode under the control of the own device, it is determined whether or not the terminal device is in the synchronous display mode by checking whether the terminal device has been controlled to shift to the synchronous display mode. May be. Further, for example, when the terminal device shifts to the synchronous display mode under the control of another device, the synchronous display mode is checked by checking whether or not the other device has been controlled to shift to the synchronous display mode. It may be determined whether or not. Further, for example, in the case of shifting to the synchronous display mode by the control of the monitoring device, it is determined whether or not it is the synchronous display mode by checking whether or not the control for shifting to the synchronous display mode has been performed. Also good.

  Another monitoring device according to the present invention is a monitoring device that receives information indicating a browsing state of a user of the terminal device from the terminal device in order to solve the above-described problem, and receives the information from the terminal device. The display control means for displaying the browsing state indicated by the information is provided.

  According to said structure, since the information which shows the browsing state of the user of this terminal device is received from the said terminal device, and the browsing state which the received information shows is displayed, the user of the said terminal device is object image display apparatus. It is possible to make other users recognize whether or not the target image displayed by is viewed on the display device of the user's terminal device.

  Note that the display of the browsing state means that a display (for example, display of an image indicating the browsing state) that allows the other users to recognize the browsing state of the user of the terminal device is performed. In addition, when the monitoring device includes a display device, the display state of the browsing state may be the display device, or may be a display device that is directly or indirectly connected to the monitoring device. For example, the target image display device may be used.

  In addition, the monitoring device receives information indicating the viewing state of the user of the terminal device from the plurality of terminal devices that display the target image displayed by the target image display device on each display device, It is preferable that a browsing rate calculation unit that calculates a proportion of users browsing the target image in the synchronous display mode among users of the terminal device is provided.

  According to said structure, the information which shows the browsing state of the user of each terminal device is received, respectively, The user who is browsing the said target image in the said synchronous display mode among the users of each terminal device, ie, 1st browsing Calculate the percentage of users in the state. In the first browsing state, since the user of the terminal device does not look away from the display device or see other images, the above ratio is the target displayed by the target image display device. It can be used as an index for evaluating the current degree of attention to an image. For example, when the above ratio is high, many users are paying attention to the target image displayed on the target image display device, and when the above ratio is low, attention is paid to the target image displayed on the target image display device. It can be evaluated that there are few users.

  Further, in order to solve the above-described problem, the terminal monitoring system according to the present invention displays the target image displayed by the target image display device that displays the target image that is the target of synchronous display on the display device of the device itself. A terminal monitoring system comprising: a terminal device having a synchronous display mode; and a monitoring device for identifying a viewing state of an image displayed on the display device of a user of the terminal device, wherein the line of sight of the user of the terminal device , Line-of-sight determination means for determining whether or not the terminal device is suitable for the display device of the terminal device is provided in the terminal device or the monitoring device, and determines whether or not the terminal device is in the synchronous display mode. Synchronization determination means is provided in the terminal device or the monitoring device, acquires information indicating the determination result of the line-of-sight determination means and the determination result of the synchronization determination means, and determines the line-of-sight determination When the stage determines that the user's line of sight is facing the display device of the terminal device, and the synchronization determination means determines that the terminal device is in the synchronous display mode, A browsing state determination unit that determines that the user is in the first browsing state browsing the target image in the synchronous display mode is provided in the terminal device or the monitoring device, and the determination result of the browsing state determination unit The monitoring apparatus acquires information indicating the above.

  According to the above configuration, in either the terminal device or the monitoring device, it is determined whether or not the user's line of sight is facing the display device of the terminal device, and whether or not the terminal device is in the synchronous display mode. judge. Then, in either the terminal device or the monitoring device, it is determined whether or not the user is in the first browsing state from these determination results, and the monitoring device acquires the determination result.

  Therefore, the monitoring device can specify whether or not the user of the terminal device is viewing the target image displayed by the target image display device on the display device of his / her terminal device. When the terminal device includes the browsing state determination unit, the monitoring device may receive and acquire information indicating the determination result of the browsing state determination unit from the terminal device.

  The terminal device and the monitoring device may be realized by a computer. In this case, the terminal device or the monitoring device is operated by the computer by operating the computer as each unit of the terminal device or the monitoring device. A control program to be realized and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  As described above, the terminal device according to the present invention includes a line-of-sight determination unit that determines whether or not the user's line of sight of the own device is facing the display device of the own device, and whether or not the own device is in a synchronous display mode. And a synchronization determination unit that determines the viewing state of an image displayed on the display device of the user of the user's device, with the information indicating the determination result of the gaze determination unit and the determination result of the synchronization determination unit It is the structure provided with the browsing state notification means to transmit to.

  In addition, as described above, the terminal device control method according to the present invention includes a line-of-sight determination step for determining whether or not the line of sight of the user of the own device faces the display device of the own device, and the own device displays the synchronous display. Viewing the image displayed on the display device of the user of the user's own device with information indicating the synchronization determination step for determining whether the mode is selected, the determination result of the line-of-sight determination step, and the determination result of the synchronization determination step A browsing state notification step that is transmitted to a monitoring device that identifies the state.

  The monitoring device according to the present invention, as described above, determines whether or not the user's line of sight is directed to the display device of the terminal device, and whether or not the terminal device is in the synchronous display mode. And the line-of-sight determination unit determines that the user's line of sight is facing the display device of the terminal device, and the synchronization determination unit determines that the terminal device is in the synchronous display mode. And a browsing state determination unit that determines that the user is in the first browsing state browsing the target image in the synchronous display mode.

  In addition, as described above, the control method of the monitoring device according to the present invention includes the line-of-sight determination step for determining whether or not the user's line of sight faces the display device of the terminal device, and the terminal device is in the synchronous display mode. It is determined that the user's line of sight is facing the display device of the terminal device in the synchronization determination step and the line-of-sight determination step, and the terminal device displays the synchronous display in the synchronization determination step. A browsing state determination step for determining that the user is in a first browsing state in which the user is browsing the target image in the synchronous display mode when the mode is determined.

  In the terminal monitoring system according to the present invention, as described above, the line-of-sight determination means for determining whether the line of sight of the user of the terminal device is facing the display device of the terminal device is provided in the terminal device or the monitoring device. Synchronization determination means for determining whether or not the terminal device is in a synchronous display mode is provided in the terminal device or the monitoring device, and the determination result of the line-of-sight determination means and the synchronization determination means Information indicating the determination result is obtained, and the line-of-sight determination unit determines that the user's line of sight is directed to the display device of the terminal device, and the synchronization determination unit includes the terminal device described above. When it is determined that the user is in the synchronous display mode, the browsing state determination unit that determines that the user is in the first browsing state browsing the target image in the synchronous display mode is the terminal device or Is provided in the monitoring device, the information indicating the judgment result of the viewing state determination unit is configured to the monitoring apparatus acquires.

  According to the above configuration, it is possible to cause the monitoring device to specify whether or not the user of the terminal device is viewing the target image displayed by the target image display device on the display device of his / her terminal device. Play.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an embodiment of the present invention and is a block diagram illustrating a main configuration of a participant terminal. It is a figure which shows the outline | summary of the electronic conference system concerning this embodiment. It is a block diagram which shows the principal part structure of the conference server contained in the said electronic conference system. It is a figure which shows an example of the meeting data which the said meeting server stores. It is a figure which shows an example of the agenda data which the said conference server stores. It is a figure which shows an example of the participating user data which the said conference server stores. It is a figure which shows an example of the connection terminal discrimination | determination data which the said conference server stores. It is a figure which shows an example of the participation status data list which the said conference server stores. It is a figure which shows an example of the participation state counter data list which the said conference server stores. It is a flowchart which shows an example of the conference setting process which the said conference server performs. It is a figure which shows an example of the participation status data which the said participant terminal produces | generates. It is a figure which shows an example of the participation state counter data which the said participant terminal produces | generates. It is a block diagram which shows the principal part structure of the presenter terminal contained in the said electronic conference system. It is a block diagram which shows the principal part structure of the moderator terminal contained in the said electronic conference system. It is a flowchart which shows an example of the process in the said electronic conference system. It is a flowchart which shows an example of the presentation process which a participant terminal performs in the said electronic conference system. It is a flowchart which shows an example of the presentation process which a conference server performs in the said electronic conference system. It is a flowchart which shows an example of the presentation process which a presenter terminal performs in the said electronic conference system. It is a figure which shows an example of the display screen of the said presenter terminal. It is a flowchart which shows an example of the presentation process which a participant terminal performs in the said electronic conference system. It is a flowchart which shows an example of the presentation process which a conference server performs in the said electronic conference system. It is a flowchart which shows an example of the presentation process which a presenter terminal performs in the said electronic conference system. It is a figure which shows an example of the display screen of the said presenter terminal. It is a flowchart which shows an example of the presentation process which a moderator terminal performs in the said electronic conference system. It is a figure which shows an example of the display screen of the said moderator terminal.

  Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.

[System Overview]
The outline | summary of the electronic conference system of this embodiment is demonstrated based on FIG. FIG. 2 is a diagram showing an outline of the electronic conference system (terminal monitoring system) 3 according to the present embodiment. As illustrated, the electronic conference system 3 is a system in which a conference server (monitoring device) 2 and a plurality of communication terminal devices (terminal device, target image display device, monitoring device) 1 are communicatively connected.

  In the electronic conference system 3, one communication terminal device (terminal device) 1 is allocated to each participant and presenter of the conference. Then, the same material is displayed on each communication terminal device 1 to hold a conference. The electronic conference system 3 is connected to a PC 4 for setting a conference.

  When a conference is held by the electronic conference system 3, the communication terminal device 1 of each participant in the conference is assigned a role determined in advance according to the user. That is, as shown in the figure, a communication terminal device 1 of a user (hereinafter referred to as a presenter) who makes a presentation at a conference is assigned a role as a presenter terminal (target image display device, monitoring device) 1a. The user (participant) who participates in the conference without performing the conference is assigned a role as the participant terminal (terminal unit) 1b to the communication terminal device 1 (hereinafter referred to as a participant), and the user (hereinafter, moderator) who proceeds with the conference The communication terminal device 1 is assigned a role as a moderator terminal (monitoring device) 1c.

  Then, the conference server 2 also displays images such as materials displayed on the presenter terminal 1a on the participant terminal 1b and the moderator terminal 1c. Thereby, in the electronic conference system 3, the image displayed on the presenter terminal 1a and the image displayed on the participant terminal 1b and the moderator terminal 1c are linked.

  That is, as shown in the figure, when “Document 2” is displayed on the presenter terminal 1a, “Document 2” is also displayed on the participant terminal 1b and the moderator terminal 1c. When the material to be displayed is switched in the presenter terminal 1a, the material is similarly switched in the participant terminal 1b and the moderator terminal 1c.

  The participant terminal 1b is connected to a camera 5 that captures a participant who is using the participant terminal 1b.

  Although details will be described later, the moderator terminal 1c displays not only the image displayed on the presenter terminal 1a but also information necessary for proceeding with the conference.

  Further, in the present embodiment, it is assumed that the communication terminal device 1 at a remote place is connected via a network to perform a conference. That is, it is assumed that the communication terminal device 1 shown in FIG. 2 is installed in a plurality of conference rooms. Of course, all the communication terminal devices 1 may be installed in one conference room.

  Furthermore, in the illustrated example, a plurality of participant terminals 1b are shown, but one participant terminal 1b may be provided. Further, the moderator terminal 1c may not be provided, and a plurality of moderator terminals 1c may be provided. Further, the conference server 2 may not be included in the electronic conference system 3. In this case, any communication terminal device 1 (for example, the presenter terminal 1a) may have the function of the conference server 2.

  That is, the electronic conference system 3 can be constructed by connecting at least two communication terminal devices 1 for communication. The communication connection between the communication terminal devices 1 may be an indirect connection via another device such as the conference server 2 or a direct connection.

  Furthermore, the electronic conference system 3 can be applied to uses other than the conference. For example, it is possible to conduct a lecture by connecting the communication terminal device 1 of the instructor and the communication terminal device 1 of the student.

[Configuration of main parts of communication terminal device 1 and conference server 2]
Next, the details of the communication terminal device 1 (presenter terminal 1a, participant terminal 1b, moderator terminal 1c) and conference server 2 included in the electronic conference system 3 will be described with reference to FIGS. To do.

<Configuration of conference server 2>
First, details of the conference server 2 will be described with reference to FIGS. FIG. 3 is a block diagram showing a main configuration of the conference server 2 included in the electronic conference system 3.

  As illustrated, the conference server 2 includes a server communication unit 42, a server control unit 43, and a server storage unit 44.

  The server operation unit 40 receives user input operations on the conference server 2. The server operation unit 40 only needs to have a function of receiving an input operation of the user and transmitting an operation signal indicating the received operation content to the server control unit 43. For example, the server operation unit 40 may be configured with a keyboard, a mouse, or the like. . Of course, an input operation may be accepted by a touch panel method.

  The server display unit 41 displays an image according to an instruction from the server control unit 43. The server display unit 41 only needs to display an image in accordance with an instruction from the server control unit 43, and may be configured by, for example, a liquid crystal display device or an organic EL display device.

  The server communication unit 42 is for the conference server 2 to communicate with other devices. In the electronic conference system 3, the conference server 2 communicates with the communication terminal device 1 and the PC 4 via the server communication unit 42.

  The server control unit 43 controls the operation of the conference server 2 in an integrated manner, and includes a conference setting unit 50, a server conference control unit 51, and a participation state data management unit (viewing rate calculation means) 52.

  The conference setting unit 50 performs conference setting processing and generates various data necessary for conducting a conference by the electronic conference system 3. Details of the conference setting process will be described later.

  The server conference control unit 51 performs control related to the conference. Specifically, the server conference control unit 51 communicates with the terminal conference control unit 17 described later, and controls the start and end of the conference and the start and end of the presentation. In addition, the server conference control unit 51 determines whether the communication terminal device 1 is operated in the presenter mode, the participant mode, or the moderator mode. To instruct.

  And the server meeting control part 51 communicates with the presenter terminal 1a and the participant terminal 1b, and controls so that the image which the presenter terminal 1a displays is also displayed on the participant terminal 1b and the moderator terminal 1c. To do.

  The participation state data management unit 52 receives data indicating the participation state of the participant from the participant terminal 1b, and among the participants participating in the conference from the received data, The ratio (participation rate) is calculated.

  Here, the participation state (browsing state) is information for determining whether or not the participant is paying attention to the presentation, and what screen the participant is displaying on the participant terminal 1b. Whether you are watching or not looking at the screen. In the present embodiment, 4 during synchronous listening (first browsing state), asynchronous listening (second browsing state), independent listening (third browsing state), and other work (fourth browsing state, non-browsing state). Two states are used to distinguish participants' participation states. Details of each state will be described later.

  In the present embodiment, the participation state data management unit 52 calculates, as the participation rate, the proportion of participants who are participating in the conference and whose participation state is in the synchronized listening mode. However, the present invention is not limited to this, and the participation state data management unit 52 may calculate, as the participation rate, the proportion of the participants who are participating in the conference whose participation state is in the synchronous listening or independent listening. Good.

  Specifically, the participation state data management unit 52 receives the participation state data from each participant terminal 1b, and calculates the participation rate of the participant from the received participation state data of each participant terminal 1b. . Then, the participation state data management unit 52 stores the calculated participation rate in the server storage unit 44 as a participation state data list.

  Also, the participation state data management unit 52 receives participation state counter data from each participant terminal 1b, and during the unit counter period during the unit counter period for each unit counter from the received participation state counter data. The average value of the period is calculated, and the participation rate is calculated from the calculated average value. The participation state data management unit 52 stores the calculated participation rate in the server storage unit 44 as a participation state counter data list.

  Here, the participation state counter data is data indicating the participation state counter of each participant, and the participation state counter is the participation state for how long the participant is in the unit counter period. It is shown. Furthermore, the period of the unit counter is a period counted by the unit counter, and indicates a period of one division obtained by dividing a period in which a meeting or presentation is performed by a predetermined division. In the present embodiment, the unit counter period is a period in which one page of the material displayed on the presenter terminal 1a is displayed.

  The server storage unit 44 stores various data used by the conference server 2. The server storage unit 44 only needs to be able to store and read data, and may be, for example, a hard disk. In the illustrated server storage unit 44, conference data 60, agenda data 61, participation user data 62, presentation material data 63, connection terminal determination data 66, participation state data list 67, and participation state counter data list 68 are stored. Yes.

  The conference data 60 is data defining a conference, and includes at least a conference ID and the date and time of the conference. The meeting data 60 may be data as shown in FIG. FIG. 4 is a diagram illustrating an example of the conference data 60.

  The conference data 60 in FIG. 4 is data in which a conference ID, a conference name, a date and time, and a conference room name to be used are associated with each other. Thereby, it is possible to specify when, where, and what kind of meeting is held. For example, it is possible to specify that the conference 1 is held in the first conference room at 10:00 to 11:00 on March 5, 2010.

  The agenda data 61 is data indicating the proceeding of each meeting specified by the meeting data 60. The agenda data 61 is data indicating who is presenting in what order in the conference, and may be data as shown in FIG. 5, for example. FIG. 5 is a diagram illustrating an example of the agenda data 61.

  The agenda data 61 in FIG. 5 is data in which a conference ID, an agenda name, a presenter name, a presenter name, a presentation time, a presentation material, and a presentation order are associated with each other. Thereby, it is possible to specify when and who will make an announcement in what order using what materials for each meeting specified by the meeting ID of the meeting data 60. Further, by including the name of the moderator in the agenda data 61, it is possible to specify who will host the conference for each conference specified by the conference ID of the conference data 60.

  In the example of FIG. 5, the name of the presenter is used to identify the presenter, but information other than the name of the presenter may be used as long as the information can identify the presenter. For example, the presenter may be specified using the presenter's name, or the presenter may be specified by an ID or the like.

  Participating user data 62 is data for specifying participants (including presenters and presenters) of each meeting specified by the meeting data 60. The participating user data 62 may be data as shown in FIG. 6, for example. FIG. 6 is a diagram illustrating an example of the participating user data 62.

  6 is data in which a conference ID, a user ID, a user name, and a role (role) in the conference are associated with each other. Thereby, it is possible to specify who participates in what role for each conference specified by the conference ID of the conference data 60. In the illustrated example, the role of the user who makes the presentation is “presenter / participant”, the role of the user who does not make the presentation is “participant”, and the role of the user who makes the moderation is “moderator”. . Note that “presenter / participant” means making a presentation and participating in the conference as a participant except at the time of presentation.

  The presentation material data 63 is data of materials used for the presentation. The data of the material registered as “presentation material” in the agenda data 61 is stored in the server storage unit 44 as the presentation material data 63.

  The connection terminal determination data 66 is data generated by the server conference control unit 51 when login is performed by the communication terminal device 1 in the electronic conference system 3, and information specifying the communication terminal device 1 and its communication This data is associated with information that identifies a user who has logged in to the terminal device 1. The connection terminal determination data 66 may be data as shown in FIG. 7, for example. FIG. 7 is a diagram illustrating an example of the connection terminal determination data 66.

  7 is data in which the user ID, the IP address of the communication terminal device 1, and the conference ID are associated with each other. Thereby, it is possible to identify which user is using which communication terminal device 1 in a certain conference.

  The participation state data list 67 indicates the participation rate calculated by the participation state data management unit 52 based on the participation state data, and is used when displaying the participation rate on the presenter terminal 1a. The participation state data list 67 may be data as shown in FIG. 8, for example. FIG. 8 is a diagram illustrating an example of the participation state data list 67.

  The participation state data list 67 shown in FIG. 8 includes a participation rate. As described above, the participation rate indicates the proportion of the participants who are participating in the conference and whose participation state is in the synchronized listening course. Further, the participation state data list 67 shown in FIG. 8 includes the participation state. The participation state included in the participation state data list 67 is a total of the participation states of the participants used by the participation state data management unit 52 to calculate the participation rate. In the participation state data list 67 shown in FIG. 8, the result of totaling the number of participants during the synchronous listening (◯), independent listening (△), and other work (×) is shown as the value of the participation state. Has been.

  The participation state data list 67 shown in FIG. 8 includes a conference ID, an agenda name, presentation materials, the number of pages, and the name of the presenter. These pieces of information are information for specifying the conference, agenda, presentation material, the number of pages, and the presenter at that time when the associated participation state is determined.

  The participation state counter data list 68 shows the participation rate calculated by the participation state data management unit 52 for each period of the unit counter based on the participation state counter data, and the presenter terminal 1a receives the average of the participation rate. It is used when displaying values. The participation state counter data list 68 may be data as shown in FIG. 9, for example. FIG. 9 is a diagram illustrating an example of the participation state counter data list 68.

  The participation state counter data list 68 shown in FIG. 9 includes a participation rate. The participation rate of the participation state counter data list 68 indicates the ratio of the average value of the period during the synchronous listening of the participant to the unit counter in the corresponding unit counter period.

  The participation state counter data list 68 shown in FIG. 9 includes an average synchronous attendance counter and a unit counter. The average synchronous attendance counter is the average value of the participant's synchronous attendance counter during the corresponding unit counter period. As for the average synchronous attendance counter, the participation state data management unit 52 receives the participation state counter data from each participant terminal 1b, and calculates the average value of the synchronous attendance counter of each participant for each period of the unit counter. Is generated by The unit counter is a unit counter included in the participation state counter data received from each participant terminal 1b by the participation state data management unit 52. In addition, according to the participation state counter data received by the participation state data management unit 52 from each participant terminal 1b, the participation state counter data list 68 includes an average independent listening counter and an average-specific work counter in addition to the average synchronous listening counter. You may go out.

  Here, the synchronous listening counter is a counter that is incremented when it is determined that the synchronous listening is being performed in the process of confirming the participation state performed at regular intervals within the period of the corresponding unit counter. Further, the independent audit counter and the separate work counter are incremented when it is determined that the independent audit is being performed and the separate work is being performed in the process of confirming the participation state performed every predetermined time within the period of the corresponding unit counter. It is a counter. These synchronous audit counter, independent audit counter, and separate work counter are collectively referred to as a participation state counter.

  The participation state counter data list 68 shown in FIG. 9 includes a conference ID, an agenda name, presentation materials, the number of pages, and the name of the presenter. These pieces of information are information for specifying the conference, agenda, presentation material, the number of pages, and the presenter at that time in the corresponding unit counter period.

<Conference setting process>
Next, the conference setting process for generating each of the data stored in the server storage unit 44 will be described with reference to FIG. FIG. 10 is a flowchart illustrating an example of the conference setting process. The conference setting process is a process performed by the conference setting unit 50 of the conference server 2 according to control from a terminal device connected to the conference server 2 such as the PC 4 in FIG. The conference setting process can also be performed by an input operation from the server operation unit 40 of the conference server 2.

  First, the conference setting unit 50 creates conference data 60 (S1). Specifically, the conference setting unit 50 allows the user to input the conference name, the date and time of the conference, and the conference room from, for example, the PC 4 in FIG. 2, and attaches the conference ID to these input data. The conference data 60 as shown in FIG.

  Next, the conference setting unit 50 creates the agenda data 61 (S2). Specifically, the conference setting unit 50 allows the user to input the agenda name, the presenter name, the presenter name, the presentation time, the name of the material used for the presentation, and the order of presentation. The conference ID is attached to the data, and agenda data 61 as shown in FIG. 5 is generated.

  Subsequently, the conference setting unit 50 registers participants (S3). Specifically, the conference setting unit 50 causes the user to input a user ID and a user name. Then, the input ID and name are associated with a role (role in the conference, that is, a participant or a presenter) and a conference ID, and the participating user data 62 as shown in FIG. 6 is generated.

  In addition, since the user registered as the presenter in the agenda data 61 participates as a participant when not making a presentation, the role is “presenter / participant” and the agenda data 61 serves as a moderator. The registered user has the role “moderator”. For a user who is not registered in the agenda data 61 by either the presenter or the presenter, the role is “participant”. However, the present invention is not limited to this example. For example, in the case where only the presentation is performed and the user leaves the office without listening to the other presenters, the presenter may be registered as “presenter”. Further, the presenter may not only conduct the presenting, but may also be a presenter or a participant in another presentation. In this case, the role is set to “moderator / presenter”, “moderator / participant”, “moderator / presenter / participant”, or the like. Furthermore, you may make it make a user select a role.

  Next, the conference setting unit 50 sets various conditions (S4). Specifically, the conference setting unit 50 sets the execution timing of the participation state determination process and the participation state count process.

  Here, the participation state determination processing means that the line-of-sight detection unit 20, the separate operation detection unit 21, and the terminal conference control unit 17 perform detection / determination processing, respectively, and based on these detection / determination results, the participation state determination unit Reference numeral 22 denotes a process for determining the participation state of the participant. In addition, with the participation state counting process, the line-of-sight detection unit 20, the separate motion detection unit 21, and the terminal conference control unit 17 perform detection / determination processing, respectively, and based on these detection / determination results, the participation state determination unit 22 Shows a process of determining the participation state of the participant and counting the participation state counter and the unit counter based on the determination result.

  Note that these setting contents are stored in the server storage unit 44 by the conference setting unit 50. Then, the server conference control unit 51 and the participation state data management unit 52 appropriately read out these setting contents and execute processing. In addition, the conference setting unit 50 may perform setting processing based on a user instruction input via the PC 4 or the communication terminal device 1, or set the setting value determined as default in the conference server 2 as it is. May be.

  As described above, all the data necessary for the conference setting is created, and the conference setting unit 50 stores the created data in the server storage unit 44, and the conference creation ends (S5).

<Configuration of participant terminal 1b>
Next, the configuration of the participant terminal 1b will be described based on FIG. As shown in FIG. 1, the participant terminal 1 b includes a camera 5, a terminal operation unit 10, a terminal display unit (display device) 11, a terminal communication unit 12, a terminal control unit 13, and a terminal storage unit 14.

  The camera 5 photographs a user (participant) who is using the participant terminal 1b. The camera 5 outputs the captured image to the terminal control unit 13. In addition, although mentioned later for details, in this invention, the terminal control part 13 determines whether the participant is seeing the image currently displayed on the terminal display part 11 of the participant terminal 1b. Therefore, the camera 5 only needs to generate information such as an image for the terminal control unit 13 to make a determination. The camera 5 may be included in the participant terminal 1b or may be a separate device from the participant terminal 1b.

  The terminal operation unit 10 receives a user input operation on the participant terminal 1b. The terminal operation unit 10 only needs to have a function of receiving an input operation of the user and transmitting an operation signal indicating the received operation content to the terminal control unit 13, and may be configured by a keyboard, a mouse, or the like, for example. . Of course, an input operation may be accepted by a touch panel method.

  The terminal display unit 11 is a member commonly provided in the communication terminal device 1 and displays an image in accordance with an instruction from the terminal control unit 13. The terminal display unit 11 only needs to display an image in accordance with an instruction from the terminal control unit 13, and can be configured by, for example, a liquid crystal display device or an organic EL display device. Here, an example in which the communication terminal device 1 includes the terminal display unit 11 is shown, but the terminal display unit 11 is connected to the communication terminal device 1 so as to function as a display device of the communication terminal device 1. It only has to be done. That is, the terminal display unit 11 and the communication terminal device 1 may be configured separately.

  The terminal communication unit 12 is for the participant terminal 1b to communicate with other devices. In the electronic conference system 3, the participant terminal 1 b communicates with the conference server 2 and the presenter terminal 1 a via the terminal communication unit 12.

  The terminal control unit 13 controls the overall operation of the participant terminal 1b. The terminal control unit 13 includes a terminal conference control unit (synchronization determination unit) 17 and a display control unit (display control unit) 18, a line-of-sight detection unit (line-of-sight determination unit) 20, another operation detection unit (asynchronous state determination unit) 21, And a participation state determination unit (browsing state notification means, browsing state determination means) 22.

  The terminal conference control unit 17 performs control for executing a conference by the electronic conference system 3. Specifically, the terminal conference control unit 17 communicates with the server conference control unit 51 to control the start and end of the conference and the start and end of the presentation. Further, the terminal conference control unit 17 determines whether to operate the communication terminal device 1 in the presenter mode, the participant mode, or the moderator mode during the conference according to the instruction of the server conference control unit 51. decide.

  Then, the terminal conference control unit 17 communicates with the server conference control unit 51 so that the image displayed on the communication terminal device 1 operating in the presenter mode, that is, the presenter terminal 1a operates in the participant mode. Control is performed so as to be displayed on the communication terminal device 1 that is being operated, that is, the terminal display unit 11 of the participant terminal 1b, and the communication terminal device 1 that is operating in the moderator mode, that is, the moderator terminal 1c.

  This is because the presenter terminal 1a transmits the image displayed by the own device to the server conference control unit 51, and the server conference control unit 51 transmits the received image to the participant terminal 1b and the moderator terminal 1c. This is realized by displaying. Note that the content of the input operation accepted by the presenter terminal 1a is notified to the server conference control unit 51, and the server conference control unit 51 that has received this notification generates an image according to the notification content, and announces the generated image. It can also be realized by transmitting to the participant terminal 1a, the participant terminal 1b, and the moderator terminal 1c.

  The terminal conference control unit 17 controls switching between a synchronous mode (synchronous display mode) and an asynchronous mode (asynchronous display mode).

  The synchronization mode is a mode in which the image displayed on the presenter terminal 1a, that is, the same image as the image of the material used for the presentation is displayed on the terminal display unit 11 of the participant terminal 1b. In other words, the synchronous mode is a mode in which the target image displayed by the presenter terminal 1a that displays the target image to be synchronized is displayed on the terminal display unit 11 of the participant terminal 1b. During the conference, the terminal conference control unit 17 basically sets the synchronization mode. In the synchronous mode, the terminal conference control unit 17 of the participant terminal 1b receives the image displayed by the presenter terminal 1a from the server conference control unit 51 and causes the terminal display unit 11 of the participant terminal 1b to display the image.

  For example, when the presenter terminal 1a displays the first page of the material 001-1, the first page of the material 001-1 is also displayed on the terminal display unit 11 of the participant terminal 1b. When the presenter operates the presenter terminal 1a to switch the page of the document 001-1 to the next page, the image displayed on the terminal display unit 11 of the participant terminal 1b is the document 001-1. Switch to the second page.

  The asynchronous mode is a mode in which the participant terminal 1b can display an image different from the image displayed on the presenter terminal 1a on the terminal display unit 11 of the participant terminal 1b independently of the presenter terminal 1a. The asynchronous mode includes an independent mode (asynchronous first state) and another work mode (asynchronous second state).

  The independent mode is a mode in which presentation materials stored in the conference server 2 are freely viewed and edited on the participant terminal 1b independently of the presenter terminal 1a. In other words, the independent mode is a mode in which any one of the images (presentation materials stored in the conference server 2) displayed on the electronic conference system 3 by the presenter terminal 1a is displayed on the terminal display unit 11 of the participant terminal 1b. It is.

  Each participant confirms a page or material other than the material page displayed on the terminal display unit 11 of the presenter terminal 1a by moving the participant terminal 1b to the independent mode, When writing or when it is difficult to see the image displayed on the presenter terminal 1 a, the image can be enlarged and displayed on the terminal display unit 11.

  That is, in the independent mode, a page different from the page of the material displayed on the presenter terminal 1a can be displayed on the terminal display unit 11 of the participant terminal 1b. Further, materials different from the materials displayed on the terminal display unit 11 of the presenter terminal 1a (materials 001-2, 001-3, etc. shown in FIG. 5) are displayed on the terminal display unit 11 of the participant terminal 1b. be able to.

  In the independent mode, the terminal conference control unit 17 receives an image corresponding to the input operation received by the terminal operation unit 10 from the server conference control unit 51 and causes the terminal display unit 11 to display the image. For example, when the terminal operation unit 10 receives a page feed input operation, the terminal conference control unit 17 requests the server conference control unit 51 for an image of the material on the next page and receives it, and the display control unit 18 is displayed on the terminal display unit 11. When presentation material data 63 is stored in the participant terminal 1b, it is not necessary to acquire an image from the conference server 2.

  The separate work mode is a mode in which none of the images (presentation materials stored in the conference server 2) displayed on the electronic conference system 3 by the presenter terminal 1a is displayed on the terminal display unit 11. Specifically, the separate work mode is a mode in which an image other than the image based on the presentation material data 63 is displayed on the terminal display unit 11. For example, a state in which an image generated by an application such as document editing software, spreadsheet software, or an internet browser is displayed on the terminal display unit 11 is another work mode.

  The trigger for shifting to another work mode is not particularly limited, but here, the trigger is shifted when application software other than the electronic conference system 3 is started or the conference by the electronic conference system 3 is ended. And

  In addition, the trigger for shifting to the independent mode is not particularly limited, but here, in the synchronous mode, the terminal conference control unit 17 shifts to the independent mode with the user operation detected by the terminal operation unit 10 as a trigger. It is assumed that Since the user operates the terminal operation unit 10 when performing an input operation, according to the above-described configuration, the user can be shifted to the independent mode without being aware of the mode switching operation between the synchronous mode and the independent mode. Can do.

  The trigger for shifting from the asynchronous mode (independent mode or separate work mode) to the synchronous mode is not particularly limited. For example, an icon or the like for shifting to the synchronous mode may be displayed, and the operation of the icon may be used as a trigger, or when the period during which no user operation is detected continues for a certain period after the shift to the asynchronous mode. The mode may be shifted to the synchronous mode.

  The display control unit 18 causes the terminal display unit 11 to display an image in accordance with an instruction from the terminal control unit 13.

  The line-of-sight detection unit 20 acquires an image of a participant who uses the device from the camera 5 and analyzes the acquired image to detect whether the participant is looking at the terminal display unit 11. is there. The line-of-sight detection unit 20 outputs the detected result to the participation state determination unit 22.

  In addition, when the participant can see the presenter, for example, when the participant and the presenter are in the same room, it is considered that the participant is looking at the image displayed on the presenter or the presenter terminal 1a. It is done. In such a case, since the participant is not looking away or performing a work different from the conference, the participation status in the conference is displayed on the terminal display unit 11 in the synchronous mode. It may be determined that the image is equivalent to the state of viewing the image.

  For this reason, when the gaze detection unit 20 detects that the participant is looking at the image displayed by the presenter or the presenter terminal 1a, the gaze detection unit 20 displays the terminal display of the own device in the participation state determination unit 22. The same information as that indicating that the screen of the unit 11 is viewed may be output to the participation state determination unit 22.

  The separate operation detection unit 21 determines whether or not the own apparatus is in another work mode. In the present embodiment, the separate operation detection unit 21 determines that the operation mode is the separate operation mode when detecting that the application software is activated or the conference by the electronic conference system 3 is ended. However, the condition for determining the separate work mode is not limited to this. For example, when the image (meeting material or the like) received from the conference server 2 is displayed on the terminal display unit 11 on the full screen, the separate work mode is not set. If the image generated by the other application software is displayed on at least a part of the terminal display unit 11, it may be determined that the operation mode is the separate operation mode.

  The participation state determination unit 22 receives the detection result and the determination result from the line-of-sight detection unit 20, the terminal conference control unit 17, and the separate operation detection unit 21, and determines the participation state of the participant who uses the own device. Is.

  Specifically, when the participation state determination unit 22 acquires information indicating that the line of sight is detected from the line of sight detection unit 20 and acquires information indicating that it is in the synchronous mode from the terminal conference control unit 17, the participation state is determined. “Synchronous listening (○)”. In addition, the participation state determination unit 22 acquires information indicating that the line of sight is detected from the line of sight detection unit 20, acquires information indicating the independent mode from the terminal conference control unit 17, and receives another operation mode from the separate operation detection unit 21. If the information indicating that it is not is acquired, the participation state is set to “independent listening (Δ)”. In addition, when the participation state determination unit 22 acquires information indicating that there is no line-of-sight detection from the line-of-sight detection unit 20 and when acquiring information indicating that the operation mode is different from the separate operation detection unit 21, the participation state is determined. “Separate work (×)”.

  Here, during synchronous listening (first viewing state) indicates a state in which the participant is watching the screen displayed on the participant terminal 1b and the participant terminal 1b is operating in the synchronous mode. In addition, during independent listening (third viewing state) indicates a state in which the participant is watching the screen displayed on the participant terminal 1b and the participant terminal 1b is operating in the independent mode. In another work (fourth viewing state), the participant is watching the screen displayed on the participant terminal 1b and the participant terminal 1b is operating in the separate work mode, or the participant is The state which does not look at the screen of participant terminal 1b is shown.

  In addition, when the specification is such that other application software cannot be started in the participant terminal 1b, the participation state determination unit 22 participates without using the detection result from the separate operation detection unit 21. The state may be determined.

  In this case, when the participation state determination unit 22 acquires information indicating that the line of sight is detected from the line of sight detection unit 20 and acquires information indicating that the terminal conference control unit 17 is not in the synchronous mode, that is, the asynchronous mode, The participation status is determined as “asynchronous listening”. Here, “asynchronous listening” (second browsing state) indicates a state in which the participant is watching the screen of the participant terminal 1b and the participant terminal 1b is operating in the asynchronous mode.

  Even if the participant terminal 1b is in the independent mode, the same image as that displayed on the presenter terminal 1a may be displayed on the participant terminal 1b. Therefore, even if the participation state determination unit 22 acquires information indicating that the line of sight is detected from the line of sight detection unit 20 and acquires information indicating that the terminal conference control unit 17 indicates the independent mode, the terminal conference When the control unit 17 detects that the image displayed on the presenter terminal 1a is the same as the image displayed on the terminal display unit 11 of the own device, the participation state determination unit 22 sets the participation state to “ It may be determined that “synchronous listening (◯)” is performed.

  When instructed by the terminal conference control unit 17 to execute the participation state determination process, the participation state determination unit 22 generates participation state data as a determination result as shown in FIG. It transmits to the participation state data management part 52. FIG. 11 is a diagram illustrating an example of participation state data generated by each participant terminal 1b. In the example illustrated in FIG. 11, the participation state data in FIG. 11A is data indicating the participation state of the participant “Mr. B”, and the participation state data in FIG. 11B is the participant “Mr. C”. The participation state data in FIG. 11C is data indicating the participation state of the participant “Mr. D”, and the participation state data in FIG. 11D is the participant “X”. The participation state data of FIG. 11E is data indicating the participation state of the participant “Mr. Y”.

  In addition, the participation state determination unit 22 counts the participation state from the server conference control unit 51 via the terminal conference control unit 17 when instructed by the terminal conference control unit 17 to execute the participation state counting process. Get the information you need to Here, since it is assumed that the participation state is counted for each page of the presentation material, information indicating the presentation material and the number of pages is acquired. When counting by conference unit, information for identifying the conference (conference ID etc.) is obtained. When counting by presenter unit, information for identifying the presenter (presenter name, user ID, etc.) is acquired. .

  Then, the participation state determination unit 22 includes information necessary for counting the unit counter, such as information (participant name, user ID, etc.) identifying a user who currently uses the device, a synchronous attendance counter, and a unit counter. Are associated and stored in the terminal storage unit 14 as a participation state counter. The participation status counter may include an independent attendance counter and another work counter.

  The participation state determination unit 22 measures a unit counter, a synchronous audit counter, an independent audit counter, and another work counter. When the participation state determination unit 22 determines that the participant's participation state is “synchronous listening (◯)”, the participation state determination unit 22 increments the synchronous listening counter, and when it determines that “independent listening (△)”, When it is determined that “different work (×)” is determined, another work counter is incremented. In addition, the participation state determination unit 22 increments the unit counter when a predetermined time has elapsed from the start of measurement of the unit counter.

  Whether the participation state determination unit 22 has received a notification from the terminal conference control unit 17 indicating that the presentation material has been switched or the presentation material page has been switched in the presenter terminal 1a after incrementing the unit counter. Confirm. When the participation state determination unit 22 receives notification of switching of presentation materials or switching of pages of presentation materials, the participation state determination unit 22 ends the measurement of the unit counter and the synchronous audit counter currently being measured. Next, the participation state determination unit 22 acquires information regarding the switched material (the material currently displayed on the terminal display unit 11 of the presenter terminal 1a) from the server conference control unit 51. The participation state determination unit 22 associates information (for example, a participant name, a user ID, etc.) identifying a user who currently uses the device, a new synchronous attendance counter, and a unit counter with the information related to the acquired material. Then, the participation state counter is generated, and the generated participation state counter is stored in the terminal storage unit 14. Then, the participation state determination unit 22 starts measuring the unit counter and the synchronous listening counter of the generated participation state counter, and performs the participation state counter measurement process and the unit counter measurement process again.

  In addition, when the terminal conference control unit 17 receives a notification of the end of presentation from the server conference control unit 51, the participation state determination unit 22 ends the measurement of the counter and the participation state counter stored in the terminal storage unit 14. Is transmitted to the participation state data management unit 52 of the conference server 2.

  The timing at which the participation state determination unit 22 transmits the participation state counter data stored in the terminal storage unit 14 to the participation state data management unit 52 is not limited to this. For example, the participation state determination unit 22 may transmit the generated participation state counter data each time the unit counter is counted. Moreover, the participation state determination part 22 may transmit the data of the participation state counter stored in the terminal memory | storage part 14, after a meeting is complete | finished.

  The terminal storage unit 14 stores various data used by the participant terminal 1b. The terminal storage unit 14 may be anything that can store and read data, and may be a hard disk, for example. In the illustrated example, a participation state counter 30 is stored in the terminal storage unit 14.

  In the present embodiment, the participant terminal 1b transmits the participation state data or the participation state counter data, and the participation rate is calculated by the conference server 2 or the like. However, the present invention is not limited to this. For example, the participant terminal 1b may include a participation rate calculation unit (ratio calculation means) that calculates a participation rate, and calculate each participation rate from the generated participation state data or participation state counter data. In this case, the participant terminal 1b may transmit only the participation rate to the conference server 2 or the like, or may transmit the participation state data or the participation state counter data and the participation rate to the conference server 2 or the like. .

  The participation state counter 30 counts the participation state of the participant who uses his / her device in the participant terminal 1b. The participation state counter 30 may be data as shown in FIG. 12, for example. FIG. 12 is a diagram illustrating an example of the participation state counter 30.

  The participation state counter 30 shown in FIG. 12 includes a synchronous attendance counter and a unit counter. As described above, the unit counter is a counter that is incremented at regular intervals during a predetermined period (for example, the presenter terminal 1a displays a certain conference material). The synchronous attendance counter is a counter that is incremented when it is determined that the synchronous attendance is being performed in the process of confirming the participation state performed at regular intervals within the predetermined period. In addition to the synchronous attendance counter, an independent attendance counter and another work counter may be included.

  12 includes a participant name, a conference ID, an agenda name, presentation materials, the number of pages, and a presenter name. These pieces of information are information for specifying the participants, the conference, the agenda, the presentation material, the number of pages, and the presenter at that time, which are counted by the synchronous attendance counter.

  The participation state counter 30 in FIG. 12A is stored in the participant terminal 1b used by the participant “Mr. A”. Similarly, the participation state counter 30 in FIGS. 5B to 5F is used by the participants “Mr. B”, “Mr. C”, “D”, “X”, and “Y”, respectively. It is stored in the participant terminal 1b.

  In addition, in the participation state counter 30 in FIG. 5A, the synchronized attendance counter and the unit counter are “−” when the agenda is 001-1. This is because “A” is the presenter at Agenda 001-1. The same applies to the agenda 001-2 in FIG.

<Configuration of presenter terminal 1a>
Next, the structure of the presenter terminal 1a is demonstrated based on FIG. As shown in FIG. 13, the presenter terminal 1 a includes a terminal operation unit 10, a terminal display unit 11, a terminal communication unit 12, a terminal control unit 13, and a terminal storage unit 14. In addition, the same reference number is attached | subjected about the block which has the same function as the block of the participant terminal 1b demonstrated in FIG. 1, and the description is abbreviate | omitted.

  As illustrated, the terminal control unit 13 of the presenter terminal 1 a includes a terminal conference control unit 17, a display control unit (display control means) 18, and a participation state data list acquisition unit 23.

  The participation state data list acquisition unit 23 receives the participation state data list or the participation state counter data list from the server conference control unit 51. In addition, the participation state data list acquisition unit 23 instructs the display control unit 18 to indicate information indicating the participation state of each participant identified from the received participation state data list or the participation state counter data list. It is displayed on the display unit 11.

  Thereby, the presenter can confirm the participation state of each participant on the terminal display unit 11, and can proceed with the presentation while considering the participation state.

<Configuration of moderator terminal 1c>
Next, the configuration of the moderator terminal 1c will be described with reference to FIG. As shown in FIG. 14, the moderator terminal 1 c includes a terminal operation unit 10, a terminal display unit 11, a terminal communication unit 12, a terminal control unit 13, and a terminal storage unit 14. In addition, the same reference number is attached | subjected about the block which has the same function as the block of the participant terminal 1b demonstrated in FIG. 1, and the description is abbreviate | omitted.

  As illustrated, the terminal control unit 13 includes a terminal conference control unit 17, a display control unit 18, and a participation rate calculation unit (viewing rate calculation means) 24.

  The participation rate calculation unit 24 receives the participation state data from the participation state determination unit 22 of each participant terminal 1b, and calculates the participation rate of the participant from the received participation state data. Then, the participation rate calculation unit 24 instructs the display control unit 18 to display the received participation state data and the calculated participation rate on the terminal display unit 11 of its own device.

  In addition, the participation rate calculation part 24 may also acquire the information which identifies the participant who uses this participant terminal 1b with the participation state data from the participation state determination part 22 of each participant terminal 1b. Thereby, the participation rate calculation unit 24 can specify which participant's participation state the received participation state data indicates.

  Further, as shown in the figure, the terminal storage unit 14 of the moderator terminal 1c stores conference data 31, agenda data 32, participating user data 33, presentation material data 34, and connection terminal determination data 35. Based on these data, the moderator terminal 1c displays information necessary for the moderator to proceed with the meeting or presentation. For example, as shown in FIG. 25, the remaining scheduled time of the presentation and the like may be displayed on the terminal display unit 11 of the moderator terminal 1c.

  Here, it is assumed that the moderator terminal 1 c acquires these data from the conference server 2 and stores them in the terminal storage unit 14. That is, the conference data 31, the agenda data 32, the participating user data 33, the presentation material data 34, and the connected terminal determination data 35 stored in the terminal storage unit 14 are stored in the server storage unit 44 of the conference server 2. It is assumed that the conference data 60, agenda data 61, participating user data 62, presentation material data 63, and connection terminal determination data 66 are received and stored.

[Conference flow]
Next, a conference flow in the electronic conference system 3 will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of processing in the electronic conference system 3. Here, it is assumed that the presenter, the participant, and the presenter of the conference have entered the user ID into the communication terminal device 1 assigned to each person and have logged in by the time of the conference. Yes.

  Then, the server conference control unit 51 that has received this user ID associates the user ID, the IP address of the communication terminal device 1, and the ID of the conference with connection terminal determination data 66 as shown in FIG. It is assumed that it is generated and stored in the server storage unit 44.

  When the date and time included in the conference data 60 match the current date and time, the server conference control unit 51 of the conference server 2 starts a conference with the conference ID associated with the date and time in the conference data (S11). ). When the conference is started, the server conference control unit 51 transmits a notification indicating that the conference is started to each IP address included in the connection terminal determination data 66. In the communication terminal device 1 that has received this notification, the terminal conference control unit 17 starts to operate.

  The server conference control unit 51 refers to the connection terminal determination data 66, the agenda data 61, and the participating user data 62, and the communication terminal device 1 used by the presenter, the communication terminal device 1 used by the participant, and Each communication terminal device 1 used by the presenter is specified. The communication terminal device 1 used by the presenter is instructed to function as the presenter terminal 1a, the communication terminal device 1 used by the participant is instructed to function as the participant terminal 1b, and the presenter Is instructed to function as the moderator terminal 1c. That is, the function of the communication terminal device 1 is not determined before the start of the conference or presentation, and the presenter terminal 1a, the participant terminal 1b, and the presenter are instructed by the role at the start of the conference or presentation. It will function in any of the terminals 1c.

  Subsequently, the server conference control unit 51 of the conference server 2 starts the presentation process (S12). In addition, the terminal conference control unit 17 of the communication terminal device 1 (presenter terminal 1a, participant terminal 1b, and moderator terminal 1b) that has received an instruction from the server conference control unit 51 also starts the presentation process. The presentation process performed by the conference server 2 and the communication terminal device 1 will be described later.

  When the presentation is finished, the server conference control unit 51 of the conference server 2 confirms whether the conference is finished (S13). The end of the conference may be determined by referring to the agenda data 61 and confirming whether or not the presentation by the last presenter has ended, or whether or not the date and time of the conference data 60 have passed. You may judge by confirming.

  Then, when it is confirmed that the conference has ended (YES in S13), the server conference control unit 51 ends the conference. On the other hand, when it is confirmed that the conference has not ended (NO in S13), the server conference control unit 51 returns to the process of S12 and starts the announcement process of the next announcement.

  In the above description, an example is shown in which the conference or presentation is started and the conference is ended by the determination of the server conference control unit 51 of the conference server 2, but these determinations may be performed by the communication terminal device 1. In this case, when it is determined that the communication terminal device 1 starts the conference or the announcement and when it is determined that the conference is to be ended, the conference server 2 is instructed to that effect and the conference or the announcement is started, and the conference is ended. Let

  For example, in the presenter terminal 1a, when the start icon 80 in FIG. 19 (the contents will be described later) is operated, the terminal conference control unit 17 of the communication terminal device 1 starts the conference or the presentation. The server conference control unit 51 that gives an instruction to the server 2 and confirms this instruction may start a meeting or presentation. Similarly, when the end icon 81 in FIG. 19 is operated, the terminal conference control unit 17 instructs the conference server 2 to end the conference or presentation, and the server conference control unit 51 that confirms this instruction causes the conference Or you may end announcement.

[First embodiment]
Next, a first example of the presentation process performed by the conference server 2 and the communication terminal device 1 in S12 of FIG. 15 will be described. Here, the case where the present participation rate of each participant is displayed on the presenter terminal 1a is demonstrated based on FIGS. 16-19 as a 1st Example of the presentation process which the conference server 2 and the communication terminal device 1 perform. To do.

<Presentation process of participant terminal 1b>
First, based on FIG. 16, the presentation process of the participant terminal 1b is demonstrated. FIG. 16 is a flowchart illustrating an example of the presentation process performed by the participant terminal 1b.

  When the conference is started, the participant terminal 1b receives a notification to start the conference from the server conference control unit 51 of the conference server 2. When the notification is received, in the participant terminal 1b, the terminal conference control unit 17 starts operation.

  The terminal conference control unit 17 that has started the operation first performs a presentation setting process (S21). Specifically, since the server conference control unit 51 of the conference server 2 transmits an image of the first page of the presentation material used in the presentation to the participant terminal 1b, the terminal conference control unit 17 receives this and displays it. The control unit 18 is instructed to display on the terminal display unit 11. As a result, the first page of the presentation material is displayed on the terminal display unit 11 of the participant terminal 1b.

  Further, the terminal conference control unit 17 notifies the server conference control unit 51 of the completion of the setting process. When the server conference control unit 51 confirms that the notification of the setting process has been received from all the communication terminal devices 1 (presenter terminal 1a, participant terminal 1b, and moderator terminal 1c), the server conference control unit 51 It is notified that preparation for the presentation is complete, and the presenter terminal 1a is ready to start the presentation.

  After the presentation is started by the presenter terminal 1a, the participation state determination unit 22 of the participant terminal 1b waits for an instruction to execute the participation state determination process from the terminal conference control unit 17. When the reception of this execution instruction is confirmed, the participation state determination process is started.

  In the participation state determination process, first, the participation state determination unit 22 instructs the line-of-sight detection unit 20 to perform line-of-sight detection. Upon receiving this instruction, the line-of-sight detection unit 20 acquires an image from the camera 5, analyzes the acquired image, and a participant using the own device displays the terminal display unit 11 (screen thereof) of the own device. It is detected whether the user is watching (S22). Then, the line-of-sight detection unit 20 outputs the detected result to the participation state determination unit 22.

  In addition, the participation state determination unit 22 instructs the terminal conference control unit 17 to notify the operation mode of the own device. Upon receiving this instruction, the terminal conference control unit 17 determines whether the own device is operating in the synchronous mode or the independent mode (S23). Then, the terminal conference control unit 17 outputs the determined result to the participation state determination unit 22.

  Here, when the information indicating that the terminal conference control unit 17 operates in the asynchronous mode is output (NO in S23), the participation state determination unit 22 determines whether or not the presentation material is displayed, That is, it is confirmed whether or not it is an independent mode. Specifically, the participation state determination unit 22 instructs the separate operation detection unit 21 to notify whether the own apparatus is in the independent mode or the separate work mode. Upon receiving this instruction, the separate operation detection unit 21 determines whether the own apparatus is in the independent mode or the separate work mode (S25), and outputs the determination result to the participation state determination unit 22.

  Next, when the participation state determination unit 22 receives the detection result and the determination result from the line-of-sight detection unit 20, the terminal conference control unit 17, and the separate operation detection unit 21, the participation state determination unit 22 determines the participation state of the participant who uses the device. Determine (S24, S26, S27).

  Specifically, the participation state determination unit 22 acquires information indicating that there is a line-of-sight detection from the line-of-sight detection unit 20 (YES in S22), and acquires information indicating that it is in the synchronous mode from the terminal conference control unit 17 ( In S23, the participation state is set to “synchronous listening (◯)” (S24).

  On the other hand, the participation state determination unit 22 obtains information with gaze detection from the gaze detection unit 20 (YES in S22), and obtains information indicating the independent mode from the terminal conference control unit 17 (NO in S23). When the information indicating that it is not the separate work mode is acquired from the separate operation detection unit 21 (YES in S25), the participation state is set to “independent listening (Δ)” (S26).

  In addition, the participation state determination unit 22 acquires information indicating that there is no line-of-sight detection from the line-of-sight detection unit 20 (NO in S22), and acquires information indicating that the operation mode is different from the separate operation detection unit 21 ( In NO at S25, the participation state is set to “separate work (×)” (S27).

  Then, the participation state determination unit 22 transmits participation state data indicating the result determined in S24, S26, or S27 to the participation state data management unit 52 of the conference server 2. The participation state determination unit 22 repeats the processing of S22 to S27 (participation state determination processing) at a constant cycle until an instruction is received from the terminal conference control unit 17.

  The terminal conference controller 17 monitors the end of the presentation (S28), and when it is determined that the presentation has ended (YES in S28), it instructs the participation state determination unit 22 to perform the participation state determination process. Terminate.

<Presentation processing of the conference server 2>
Next, the presentation process of the conference server 2 will be described based on FIG. FIG. 17 is a flowchart illustrating an example of the presentation process performed by the conference server 2.

  The server conference control unit 51 executes a setting process (S31). Specifically, first, the server conference control unit 51 refers to the agenda data 61 and identifies the next announcement and the announcement material data 63 used in the announcement. Then, the image of the first page in the specified presentation material data 63 is transmitted to each IP address included in the connection terminal determination data 66.

  When the server conference control unit 51 receives the notification of the completion of the setting process from all the communication terminal devices 1, the server conference control unit 51 notifies the presenter terminal 1a that the preparation for the presentation is completed, and starts the presenting from the presenter terminal 1a. Each communication terminal device 1 (presenter terminal 1a, participant terminal 1b, and moderator) that the presentation starts when the announcement is made or when the presentation start time specified in the conference data 60 is reached The terminal 1c) is notified (S32).

  Next, the server conference control unit 51 transmits an instruction to execute the participation state determination process to each participant terminal 1b and waits for reception of participation state data from the participant terminal 1b (S33). Then, when the participation state data management unit 52 confirms that the participation state data has been received from the participation state determination unit 22 of each participant terminal 1b (YES in S33), the current participation rate is determined from the received participation state data. Calculate (S34).

  Subsequently, the participation state data management unit 52 generates a participation state data list 67 indicating the calculated participation rate, and transmits it to the presenter terminal 1a (S35). Further, the participation state data management unit 52 stores the generated participation state data list 67 in the server storage unit 44.

  Here, the server conference control unit 51 confirms whether or not to end the presentation (S36). If the presentation is not ended (NO in S33), the process returns to the process of S33 and waits for reception of participation state data. . On the other hand, if the presentation is to be ended (YES in S33), each participant terminal 1b is instructed to end the participation state determination process and the presentation is ended (S37).

<Presentation process of the presenter terminal 1a>
Finally, the presentation process of the presenter terminal 1a will be described with reference to FIG. FIG. 18 is a flowchart illustrating an example of the presentation process performed by the presenter terminal 1a.

  In the presenter terminal 1a that has received the notification to start the conference from the conference server 2, the terminal conference control unit 17 starts its operation. And the terminal conference control part 17 performs the setting process of presentation (S41). As a result, the first page of the presentation material is displayed on the terminal display unit 11 of the presenter terminal 1a.

  Next, the terminal conference control unit 17 notifies the server conference control unit 51 of the completion of the setting process. When the setting process of each communication terminal device 1 is completed, the server conference control unit 51 notifies the presenter terminal 1a that preparation for the presentation is complete, and starts the presentation. The terminal conference control unit 17 of the presenter terminal 1a displays that the preparation for the presentation is completed on the terminal display unit 11 of the own apparatus, and prompts the presenter to start the presentation.

  After the presentation is started, the participation state data list acquisition unit 23 of the presenter terminal 1a waits for reception of information indicating the participation rate, that is, the participation state data list (S42). Then, when the reception of the participation state data list transmitted by the server conference control unit 51 is confirmed (YES in S42), the participation rate indicated in the received participation state data list is instructed to the display control unit 18 to the terminal display unit 11. It is displayed (S43).

  FIG. 19 shows an example of the screen displayed on the terminal display unit 11 at this time. FIG. 19 is a diagram illustrating an example of a display screen displayed by the presenter terminal 1a. In the illustrated example, the screen displayed on the terminal display unit 11 of the presenter terminal 1a includes a material display area 70 for displaying materials transmitted from the conference server 2, and icons used for operations of the presenter terminal 1a. An icon display area 71 to be displayed and a participation rate display area 72 to display the participation rate indicated by the participation state data list transmitted from the conference server 2 are included.

  In the material display area 70, “Meeting material” is displayed. In addition, in the icon display area 71, a start icon 80 for starting a presentation or a meeting, an end icon 81 for ending a presentation or a meeting, and each participant terminal 1b perform a participation state determination process. A participation rate determination icon 82 for causing the conference server 2 to calculate the participation rate is displayed. The icons displayed in the icon display area 71 are not limited to the illustrated example, and other icons for executing the functions of the presenter terminal 1a may be displayed.

  The participation rate display area 72 displays the participation rate and the number of participants according to the participation state based on the participation state data list received from the conference server 2 by the participation state data list acquisition unit 23. Specifically, “conference participation rate: 60%” is displayed as information indicating the participation rate, and “synchronous listening: 3 people, independent listening: “One person, working separately: One person” is displayed. Note that the display screen is not limited to the example of FIG. 19 as long as information that allows the presenter to recognize the participation state of each participant is displayed.

  The terminal conference control unit 17 monitors whether or not to end the presentation (S44). When the presentation is not ended (NO in S44), the process returns to S42. That is, the participation state data list acquisition unit 23 performs reception processing of the participation state data list during the presentation. On the other hand, when ending the presentation (YES in S44), the terminal conference control unit 17 causes the participation state data list acquisition unit 23 to end the reception process of the participation state data list and ends the presentation process.

  As in the first embodiment, each communication terminal device 1 and the conference server 2 execute the presentation process, thereby displaying the participation rate of the current participant on the terminal display unit 11 of the presenter terminal 1a. be able to.

  The conference server is configured so that the presenter terminal 1a executes the conference start / presentation start / end process and the participation state determination process when the presenter operates the icons 80, 81, and 82 shown in FIG. 2 may be instructed.

  Moreover, the display screen of the presenter terminal 1a shown in FIG. 19 is an example. For example, the display control unit 18 may cause the terminal display unit 11 to display only the participation rate. Moreover, the display control part 18 may identify each participant and may display each participation state like the example of a display screen shown in below-mentioned FIG. In this case, the presenter terminal 1a acquires the participation state data list from the conference server 2, and also acquires the participation state data associated with the information for identifying each participant.

  Further, the moderator terminal 1c may receive the participation state data list from the conference server 2 and display a display screen as shown in FIG. 19 on the terminal display unit 11 of the moderator terminal 1c. In this case, the moderator terminal 1c only needs to have the same function as the participation state data list acquisition unit 23 of the presenter terminal 1a.

[Second Embodiment]
Next, a second embodiment of the presentation process performed by the conference server 2 and the communication terminal device 1 in S12 of FIG. 15 will be described. Here, the case where the participation rate for each page of the presentation material is displayed on the presenter terminal 1a as a second embodiment of the presentation process performed by the conference server 2 and the communication terminal device 1 will be described with reference to FIGS. To do.

<Presentation process of participant terminal 1b>
First, based on FIG. 20, the presentation process of the participant terminal 1b is demonstrated. FIG. 20 is a flowchart illustrating an example of the presentation process performed by the participant terminal 1b.

  When the terminal conference control unit 17 receives a notification to start the conference from the server conference control unit 51, the terminal conference control unit 17 starts the operation and executes the setting process for the presentation (S51). The setting process in S51 is the same as the setting process in S21 of FIG.

  When the setting process ends, the terminal conference control unit 17 notifies the server conference control unit 51 of the completion of the setting process. When the setting process of each communication terminal device 1 is completed, the server conference control unit 51 notifies the presenter terminal 1a that preparation for the presentation is complete, and starts the presentation.

  After the presentation is started by the presenter terminal 1a, the participation state determination unit 22 of the participant terminal 1b waits for an execution instruction of the participation state count process from the terminal conference control unit 17. When the reception of the execution instruction is confirmed, the participation state counting process is started.

  The terminal conference control unit 17 also instructs the execution timing of the participation state count process. Here, the terminal conference control unit 17 instructs the participation state counting process to be executed at predetermined time intervals. The instruction from the terminal conference control unit 17 is based on the instruction received from the server conference control unit 51.

  In the participation state counting process, first, the participation state determination unit 22 specifies the material currently displayed on the terminal display unit 11 of the participant terminal 1b from the server conference control unit 51 via the terminal conference control unit 17. Get information to do. In the second embodiment, as shown in FIG. 12, at least the presentation material and its page number are acquired in order to count the participation state for each page of the material.

  The participation state determination unit 22 associates the information acquired from the server conference control unit 51 with information (here, the name of the participant) that identifies a user who uses the device, and a synchronous attendance counter and a unit counter. A participation state counter is generated and stored in the terminal storage unit 14. Then, the participation state determination unit 22 starts counting by the unit counter and the synchronous attendance counter (S52, S53).

  Subsequently, the participation state determination unit 22 instructs the line-of-sight detection unit 20 to perform line-of-sight detection. Upon receiving this instruction, the line-of-sight detection unit 20 acquires an image from the camera 5 at a predetermined time interval, analyzes the acquired image, and a participant using the own device recognizes the terminal display unit 11 of the own device. It is detected whether or not the screen is viewed (S54). Then, the line-of-sight detection unit 20 outputs the detected result to the participation state determination unit 22 each time.

  In addition, the participation state determination unit 22 instructs the terminal conference control unit 17 to notify the operation mode of the own device. Upon receiving this instruction, the terminal conference control unit 17 determines whether the own apparatus is operating in the synchronous mode or the independent mode at predetermined time intervals (S55). And the terminal conference control part 17 outputs the determined result to the participating state determination part 22 each time.

  Next, the participation state determination part 22 will determine the participation state of the participant who is using the own apparatus, if a detection result and a determination result are received from the gaze detection part 20 and the terminal conference control part 17. Here, when the participation state determination unit 22 determines that the participation state is “synchronous listening (O)” (YES in S54 and YES in S55), the participation state determination unit 22 increments the synchronous listening counter (S56). Further, the participation state determination unit 22 does not increment when the participation state is not “synchronous listening (◯)” (NO in S54, YES in S54, and NO in S55).

  In addition, the participation state determination unit 22 increments the unit counter when a predetermined time has elapsed from the start of measurement of the unit counter (S57).

  After incrementing the unit counter, the participation state determination unit 22 checks whether or not the terminal conference control unit 17 has been notified of the presentation material switching or the presentation material page switching in the presenter terminal 1a (S58). . If there is no notification of switching the presentation material or the page of the presentation material (NO in S58), the participation state determination unit 22 subsequently checks whether the terminal conference control unit 17 has been notified of the end of the presentation. (S59). If there is no notification of the end of the presentation (NO in S59), the series of processing from S54 to S57 is repeated.

  In S58, when the notification of the presentation material switching or the page switching of the presentation material is received (YES in S58), the participation state determination unit 22 ends the counting by the currently measured unit counter and the synchronous audit counter. . Thus, the participation state counting process for one page of the presentation material is completed.

  Next, the participation state determination unit 22 acquires information regarding the switched material (the material currently displayed on the terminal display unit 11 of the presenter terminal 1a) from the server conference control unit 51. The participation state determination unit 22 associates information (in this case, the name of the participant) identifying the user who uses the device, the new synchronous attendance counter, and the unit counter with the information on the acquired material in association with the participation state counter. And the generated participation state counter is stored in the terminal storage unit 14. Then, the participation state determination unit 22 starts counting with the unit counter and the synchronous attendance counter of the generated participation state counter (S52, S53), and executes a series of processes from S54 to S57.

  In S59, when the participation state determination unit 22 receives a notification of the end of the presentation (YES in S59), the participation state determination unit 22 ends the counting and the participation state counter stored in the terminal storage unit 14 Is transmitted to the participation state data management unit 52 of the conference server 2 (S60), and the presentation process is terminated.

  In the second embodiment, similarly to the first embodiment, it is determined whether the separate operation detection unit 21 is in the separate work mode, and the participation state determination unit 22 determines that “synchronous listening (◯)”. It may be determined whether the state is “independent listening (Δ)” or “different work (×)”. In this case, the participation state determination unit 22 generates a participation state counter including a synchronous audit counter, an independent audit counter, and another work counter. If the participation state is determined to be “independent listening (Δ)”, the independent listening counter is incremented. If the participation state is determined to be “different work (×)”, the separate work counter is incremented.

  In this embodiment, the unit counter is counted for each page of the material. However, the present invention is not limited to this. For example, it may be every document, every agenda, every meeting, every presenter.

  In this embodiment, the unit counter is measured at the participant terminal 1b, but the present invention is not limited to this. For example, the unit counter may be measured by the conference server 2.

<Presentation processing of the conference server 2>
Next, the presentation process of the conference server 2 will be described based on FIG. FIG. 21 is a flowchart illustrating an example of the presentation process performed by the conference server 2.

  The server conference control unit 51 executes terminal setting processing as in the example of FIG. 17 (S71). When the server conference control unit 51 receives the notification of the completion of the setting process from all the communication terminal devices 1, the server conference control unit 51 notifies the terminal conference control unit 17 of the presenter terminal 1a that the preparation for the presentation has been completed. The apparatus 1 is notified of the start of presentation (S72).

  Next, the server conference control unit 51 instructs the participant terminal 1b to execute the participation state counting process. The server conference control unit 51 also instructs the execution timing of the participation state count process.

  Here, when it is confirmed that the presentation is to be ended, the server conference control unit 51 transmits a notification of the end of the presentation to each communication terminal device 1 (S73). Then, the participation state data management unit 52 waits for participation state counter data to be transmitted from each participant terminal 1b (S74).

  When the participation state counter data is received from the participation state determination unit 22 of each participant terminal 1b (YES in S74), the participation state data management unit 52 uses the received participation state counter data for each page of the presentation material. An average value of the participation rates of the participants is calculated (S75). The participation state data management unit 52 stores the calculated average value of the participation rates in the server storage unit 44 as a participation state counter data list.

  Then, the server conference control unit 51 transmits the participation state counter data list stored in the server storage unit 44 to the terminal conference control unit 17 of the presenter terminal 1a (S76), and ends the presentation process.

  In S74, if the participation state data management unit 52 cannot receive the participation state counter data within a certain period (NO in S74), an error process or the like is performed and the presentation process is terminated.

<Presentation process of the presenter terminal 1a>
Finally, the presentation process of the presenter terminal 1a will be described with reference to FIG. FIG. 22 is a flowchart illustrating an example of the presentation process performed by the presenter terminal 1a.

  When the terminal conference control unit 17 receives a notification to start the conference from the server conference control unit 51, the terminal conference control unit 17 starts the operation and executes the setting process for the presentation (S81). The setting process in S81 is the same as the setting process in S41 shown in FIG.

  Next, the terminal conference control unit 17 notifies the server conference control unit 51 of the completion of the setting process. When the setting process of each communication terminal device 1 is completed, the server conference control unit 51 notifies the presenter terminal 1a that preparation for the presentation is complete, and starts the presentation. The presenter terminal 1a displays on the terminal display unit 11 of its own device that the preparation for the presentation is complete, and prompts the presenter to start the presentation. Thereafter, an announcement is made, and when the terminal conference control unit 17 receives a notification of the end of the announcement from the server conference control unit 51, the announcement ends.

  Here, the participation state data list acquisition unit 23 waits for the information indicating the participation rate, that is, the participation state counter data list to be transmitted from the server conference control unit 51 (S82). When the participation state data list acquisition unit 23 receives the participation state counter data list (YES in S82), the participation state data list acquisition unit 23 calculates the average participation rate of each participant for each page of the presentation material from the received participation state data list. read out. And the read average value is instruct | indicated to the display control part 18, and it is made to display on the terminal display part 11 (S83), and a presentation process is complete | finished. FIG. 23 shows a screen example displayed on the terminal display unit 11 at this time.

  As shown in FIG. 23, the participation rate for each presentation material specified from the participation state counter data list transmitted from the conference server 2 is displayed on the terminal display unit 11 of the presenter terminal 1a.

  Specifically, the presenter name, the presentation material name, and the participation rate for each presentation material included in the participation state counter data list are displayed. More specifically, the participation rate for each presentation material when the presenter “Mr. A” made a presentation using the presentation material “Document 001-1” and the presenter using the presentation material “Document 001-2” The participation rate for each presentation material when “Mr. B” made a presentation is displayed. Furthermore, as the participation rate for each presentation material, the overall participation rate (participation rate during the period when a certain presentation material was displayed) of the presentation material ("Document 001-1" or "Document 001-2") and The participation rate of each page of the presentation material (participation rate during the period when each page of the presentation material was displayed) is displayed.

  The display screen displayed by the presenter terminal 1a is not limited to the example of FIG. 23 as long as the presenter can recognize the participation state for each presentation material. In addition, here, an example is shown in which the participation state for each presentation material is displayed. However, for example, the participation state totaled in other aggregation units, such as the participation state for each agenda, may be displayed.

  On the other hand, in S82, if the participation state data list acquisition unit 23 cannot receive the participation state counter data list within a certain period (NO in S82), an error process or the like is performed, and the presentation process is terminated.

  As in the second embodiment, each communication terminal device 1 and the conference server 2 perform processing to display the participation rate of each page of the material used for the presentation on the terminal display unit 11 of the presenter terminal 1a. Can be made.

  As a result, the presenter can grasp the participation rate of the total participants during his presentation, as well as which page of the material used for the presentation the participant was paying attention to, etc. I can grasp it. Then, the presenter can improve the presentation method and the method of creating the presentation material based on the participation rate data for each page after the presentation is completed.

  In this embodiment, the participant counter data list is transmitted from the participant terminal 1b to the presenter terminal 1a via the conference server 2 after the end of the presentation, but the present invention is not limited to this. For example, the presenter terminal 1a may receive it after the conference ends. Each time the participant terminal 1b counts the unit counter, the participant terminal 1b transmits participation state counter data to the conference server 2, and the presenter terminal 1a displays the current participation rate and the participation rate up to that point. The cumulative result may be displayed.

  In addition, the conference server 2 may transmit the participant counter data list to the presenter terminal 1 c, the PC 4 connected to the conference server 2, and the like instead of the presenter terminal 1 a.

[Third embodiment]
Next, a third embodiment of the presentation process performed by the conference server 2 and the communication terminal device 1 in S12 of FIG. 15 will be described. Here, as a third embodiment of the presentation process performed by the conference server 2 and the communication terminal device 1, the participation state data is directly transmitted from each participant terminal 1b to the presenter terminal 1c, and the present participation is given to the presenter terminal 1c. A case where the participation rate of the person is displayed will be described with reference to FIGS.

  In the third embodiment, the participant terminal 1b determines the current participation state and generates participation state data as in the first embodiment. The third embodiment is different from the first embodiment only in that the destination to which the participant terminal 1b transmits the participation state data is not the conference server 2 but the moderator terminal 1c. Therefore, description of the presentation process of the participant terminal 1b and the conference server 2 is omitted here.

<Presentation process of the moderator terminal 1c>
Based on FIG. 24, the presentation process of the moderator terminal 1c in a 3rd Example is demonstrated. FIG. 24 is a flowchart illustrating an example of the presentation process performed by the moderator terminal 1c.

  When the conference is started, the moderator terminal 1c receives a notification that the conference is started from the server conference control unit 51 of the conference server 2, and the terminal conference control unit 17 of the moderator terminal 1c starts operating accordingly. To do.

  First, the terminal conference control unit 17 performs a presentation setting process in the same manner as the participant terminal 1b (S91). Specifically, since the server conference control unit 51 of the conference server 2 transmits an image of the first page of the presentation material used in the presentation to the participant terminal 1b, the terminal conference control unit 17 receives this and displays it. The control unit 18 is instructed to display on the terminal display unit 11. As a result, the first page of the presentation material is displayed on the terminal display unit 11 of the moderator terminal 1c.

  Further, the terminal conference control unit 17 notifies the server conference control unit 51 of the completion of the setting process. When the setting process of each communication terminal device 1 is completed, the server conference control unit 51 notifies the presenter terminal 1a that preparation for the presentation is complete, and starts the presentation. At this time, the server conference control unit 51 may notify the moderator terminal 1c that preparations for presentation are complete. When the moderator terminal 1c receives the notification, the moderator terminal 1c displays on the terminal display unit 11 of its own device that the preparation for the presentation is complete, and prompts the moderator to proceed with the conference.

  During the presentation, when the participation rate calculation unit 24 receives the participation state data from each participant terminal 1b (YES in S92), the participation rate calculation unit 24 calculates the participation rate from the received participation state data of each participant terminal 1b (S93).

  Subsequently, the participation rate calculation unit 24 instructs the display control unit 18 to display the calculated participation rate on the terminal display unit 11 (S94). FIG. 25 shows an example of a screen displayed on the terminal display unit 11 at this time.

  The display screen displayed on the terminal display unit 11 of the moderator terminal 1c shown in FIG. 25 includes a material display area 70, an icon display area 71, and a participation rate display area 72, as in the example of FIG. In addition, a moderator progress information display area 73 for displaying information for proceeding with the conference is included.

  In the material display area 70, “Meeting material” is displayed. In addition, the icon display area 71 includes a start icon 80 for starting a presentation or a conference, an end icon 81 for ending a presentation or a conference, and causing each participant terminal 1b to perform a participation state determination process. An participation rate determination icon 82 for causing 2 to calculate the participation rate is displayed. The icons displayed in the icon display area 71 are not limited to the illustrated example, and other icons for executing the functions of the moderator terminal 1c may be displayed.

  In the illustrated example, the participation rate display area 72 includes the participation rate calculated by the participation rate calculation unit 24 from the participation state data list received from the conference server 2 and each participation specified from the received participation state data list. The participant's participation status is displayed. Specifically, “Meeting participation rate: 60%” is displayed as information indicating the participation rate, and “Mr. A: presenter, Mr. B: ○, Mr. C: as information indicating the participation status of each participant”. ○, Mr. D: ○, Mr. X: Δ, Mr. Y: × ”are displayed.

  The moderator progress information display area 73 is identified from the conference data 31, agenda data 32, participating user data 33, presentation material data 34, and connected terminal determination data 35 stored in the terminal storage unit 14. Information necessary for progress is displayed. In the illustrated example, referring to the “announcement time” included in the agenda data 32 in the moderator progress information display area 73, the remaining time of the presenter who is presently presenting is “remaining announcement time 9:34”. It is displayed as. Of course, the information displayed in the moderator progress information display area 73 is not limited to the “remaining presentation time”. For example, information indicating the next presentation content (presenter, presentation material, presentation time, etc.) is displayed. Also good.

  The participation rate calculation unit 24 performs the reception process of the participation state data until the terminal conference control unit 17 receives the announcement completion notification from the server conference control unit 51, and receives the announcement completion notification (YES in S95). End the presentation process.

  As in the above-described third embodiment, each communication terminal device 1 and the conference server 2 process, so that the participation rate of the current participant can be displayed on the terminal display unit 11 of the moderator terminal 1c. .

  In addition, as shown in FIG. 25, in order to identify a participant and to display the participation state of each participant, in S92, the participation rate calculation unit 24, the participation state determination unit 22 of each participant terminal 1b. In addition to receiving the participation state data, it is also necessary to acquire information for identifying the participant who uses each participant terminal 1b. That is, in this embodiment, the participation state determination unit 22 of each participant terminal 1b associates the participation state data with information for identifying the participant who uses the device, and the participation rate calculation unit of the moderator terminal 1c. 24.

  Also in the first and second embodiments, as in the third embodiment, the presenter terminal 1a receives the participation state data or the participation state counter data directly from the participant terminal 1b. May be. In this case, the presenter terminal 1a only needs to have the same function as the participation rate calculation unit 24 of the moderator terminal 1c.

  In addition, when the moderator operates the icons 80, 81, and 82 shown in FIG. 25, the moderator terminal 1c executes the conference and presentation start / end processing and the participation state determination processing. 2 may be instructed.

[Modification]
In the above embodiment, the configuration in which the participant terminal 1b determines whether the line of sight of the participant who uses the device is facing the terminal display unit 11 of the device (the line of sight detection unit 20 and the participation state determination unit 22). And a configuration for determining the operation mode of the own device (terminal conference control unit 17, separate operation detection unit 21 and participation state determination unit 22), determination result of participant's line of sight, and determination result of operation mode of participant terminal 1b Although the example provided with the structure (participation state determination part 22) which determines a participant's participation state based on this was demonstrated, these structures may be provided in apparatuses other than the participant terminal 1b.

  That is, a configuration for determining whether the line of sight of the participant who uses each participant terminal 1b is facing the terminal display unit 11 of the participant terminal 1b, a configuration for determining the operation mode of each participant terminal 1b, The configuration for determining the participation state of each participant based on the determination result of each participant's line of sight and the determination result of the operation mode of each participant terminal 1b is a communication terminal device 1 (announcement) included in the electronic conference system 3 The participant terminal 1a, the participant terminal 1b, the moderator terminal 1c) or the conference server 2 may be provided.

  In the above embodiment, the conference server 2 or the moderator terminal 1c has been described as having an arrangement for calculating the participation rate (participation state data management unit 52 or participation rate calculation unit 24). The apparatus may be provided in a device other than the conference server 2 or the moderator terminal 1c.

  That is, the configuration for calculating the participation rate is provided in at least one of the communication terminal device 1 (presenter terminal 1a, participant terminal 1b, moderator terminal 1c) or the conference server 2 included in the electronic conference system 3. Just do it.

  That is, any device included in the electronic conference system 3 includes a configuration for determining whether or not the line of sight of the participant who uses each participant terminal 1b is facing the terminal display unit 11 of the participant terminal 1b. In addition, any device included in the electronic conference system 3 includes a configuration for determining the operation mode of each participant terminal 1b, and further, any device included in the electronic conference system 3 includes a line of sight of each participant. The determination result and the determination result of the operation mode of each participant terminal 1b may be received, and the participation state of each participant may be determined based on the received determination results. Any device included in the electronic conference system 3 only has to have a configuration for receiving the determination result of each participant's participation state and calculating the participation rate from the received determination result.

  For example, in the first embodiment described above, the conference server 2 determines whether or not the line of sight of the participant who uses each participant terminal 1b is facing the terminal display unit 11 of the participant terminal 1b. A configuration for determining the operation mode of each participant terminal 1b, and a configuration for determining the participation state of each participant based on the determination result of each participant's line of sight and the determination result of the operation mode of each participant terminal 1b. You may have.

  In this case, when the conference server 2 starts the participation state determination process, the conference server 2 requests each participant terminal 1b to transmit an image captured by the camera 5. Then, the conference server 2 analyzes the received image taken by the camera 5 of each participant terminal 1b, and the participant using each participant terminal 1b receives the terminal display unit 11 of the participant terminal 1b. Determine whether you are watching. The process of analyzing the image and detecting the direction of the user's line of sight may be performed by another device having a function of detecting the direction of the line of sight. It is determined whether the terminal display unit 11 is being viewed.

  In addition, when the conference server 2 starts the participation state determination process, the conference server 2 instructs each participant terminal 1b to notify the operation mode of the participant terminal 1b. The conference server 2 receives information indicating the operation mode from each participant terminal 1b, that is, information indicating whether the mode is the synchronous mode or the asynchronous mode. Since the conference server 2 has a function of controlling the operation mode of each participant terminal 1b, the operation mode of the participant terminal 1b is communicated with the participant terminal 1b. It can also be specified without.

  Here, when the conference server 2 receives information indicating the asynchronous mode from any of the participant terminals 1b, the conference server 2 further determines whether the participant terminal 1b is in the independent mode or in another work mode. Instruct to notify if there is. And the conference server 2 receives the information which shows whether it is an independent mode or another work mode from each participant terminal 1b.

  Next, the conference server 2 determines the participation state of each participant terminal 1b based on the result of the gaze detection of the participant who uses each participant terminal 1b and the operation mode of each participant terminal 1b. That is, the conference server 2 generates participation state data for each participant terminal. Then, the conference server 2 calculates the current participation rate from the generated participation state data, and generates a participation state data list including the calculated participation rate.

  Moreover, although the example which connected the camera 5 which image | photographs a participant to each participant terminal 1b was demonstrated in the said embodiment, the camera 5 should just be what can image | photograph each participant, and eyes | visual_axis What is necessary is just to be connected to the apparatus which performs determination. For example, in a meeting room where a plurality of participants are present, at least one camera 5 capable of photographing the plurality of participants is installed, and the camera 5 and a device for determining the line of sight of the participants are connected by communication. It only has to be.

[Supplement]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the communication terminal device 1 and the conference server 2, particularly the terminal control unit 13 and the server control unit 43, may be configured by hardware logic, or realized by software using a CPU as follows. May be.

  That is, the communication terminal device 1 and the conference server 2 are configured such that a CPU (central processing unit) that executes instructions of a control program for realizing each function, a ROM (read only memory) that stores the program, and a RAM ( random access memory), a storage device (recording medium) such as a memory for storing the program and various data. The object of the present invention is to record the program code (execution format program, intermediate code program, source program) of the control program for the communication terminal device 1 and the conference server 2 which are software for realizing the functions described above so as to be read by a computer. This can also be achieved by supplying the recording medium to the communication terminal device 1 and the conference server 2 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Moreover, the communication terminal device 1 and the conference server 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention makes it possible to determine whether or not a conference participant is participating in a conference, and is particularly applicable to a terminal device that constitutes an electronic conference system.

1 Communication terminal device (terminal device, target image display device, monitoring device)
1a Presenter terminal (target image display device, monitoring device)
1b Participant terminal (terminal equipment)
1c Moderator terminal (monitoring device)
2 Conference server (monitoring device)
3 Electronic conference system (terminal monitoring system)
11 Terminal display unit (display device)
17 Terminal conference controller (synchronization determination means)
18 Display control unit (display control means)
20 Gaze detection unit (gaze determination means)
21 Separate motion detection unit (asynchronous state determination means)
22 Participation state determination unit (browsing state notification means, browsing state determination means)
24 Participation rate calculation unit (ratio calculation means, browsing rate calculation means)
52 Participation state data management unit (ratio calculation means, browsing rate calculation means)

Claims (16)

A terminal device having a synchronous display mode for displaying a target image displayed by a target image display device that displays a target image as a target of synchronous display on the display device of the device itself,
Line-of-sight determining means for determining whether the line of sight of the user of the own apparatus is facing the display apparatus of the own apparatus;
Synchronization determination means for determining whether or not the device is in the synchronous display mode;
Browsing state notification means for transmitting information indicating the determination result of the line-of-sight determination means and the determination result of the synchronization determination means to a monitoring device that identifies the browsing state of an image displayed on the display device of the user of the device; A terminal device comprising:   The browsing state notifying unit determines that the line-of-sight determining unit determines that the user's line of sight is facing the display device of the user apparatus, and the synchronization determining unit determines that the synchronous display mode is set. The terminal device according to claim 1, wherein information indicating that the user is in a first browsing state in which the user is browsing the target image in a synchronous display mode is transmitted to the monitoring device.   The browsing state notifying unit is a non-browsing state in which the user does not look at the display device of the user apparatus when the line-of-sight determination unit determines that the user's line of sight is not directed to the display device of the user apparatus. The terminal device according to claim 1, wherein information indicating this is transmitted to the monitoring device. The synchronization determination unit is an asynchronous display mode in which the own device is in the synchronous display mode or the own device can display an image different from the target image displayed on the target image display device on the display device of the own device. Determine whether
The browsing state notifying unit determines that the line-of-sight determination unit determines that the user's line of sight is facing the display device of the user apparatus, and the synchronization determination unit determines that the asynchronous display mode is set. The terminal according to any one of claims 1 to 3, wherein information indicating that the user is in a second browsing state in which an image is browsed in an asynchronous display mode is transmitted to the monitoring device. apparatus. The target image display device displays any of a plurality of predetermined target images,
When the device is in the asynchronous display mode, the device includes an asynchronous state determination unit that determines whether or not the device is in the first asynchronous state in which any of the plurality of target images is displayed on the display device of the device,
The browsing state notifying unit determines that the line-of-sight determining unit determines that the user's line of sight is facing the display device of the user apparatus, and the asynchronous state determining unit determines that the asynchronous first state is present. The terminal device according to claim 4, wherein the terminal device transmits information indicating that the user is in a third browsing state browsing the target image in the asynchronous display mode to the monitoring device. The target image display device displays any of a plurality of predetermined target images,
When the own device is in the asynchronous display mode, it is in an asynchronous first state in which any of the plurality of target images is displayed on the display device of the own device, or the target image is displayed on the display device of the own device. Asynchronous state determination means for determining whether or not the asynchronous second state is not performed,
The browsing state notifying unit determines that the line-of-sight determining unit determines that the user's line of sight is facing the display device of the user apparatus, and the asynchronous state determining unit determines that the asynchronous second state is present. The terminal device according to claim 4, wherein information indicating that the user is in a fourth browsing state not browsing the target image is transmitted to the monitoring device.   A ratio calculating means for calculating a ratio of a period during which the user of the apparatus is viewing the target image in the first browsing state in a period during which the target image display apparatus displays one target image; The terminal device according to claim 2, wherein: The target image displayed by the target image display device that displays the target image that is the target of the synchronous display is displayed on the display device of the user who uses the terminal device having the synchronous display mode for displaying the target image on the display device of the own device. A monitoring device for identifying a browsing state of an image to be displayed,
Line-of-sight determining means for determining whether the line of sight of the user is facing the display device of the terminal device;
Synchronization determination means for determining whether or not the terminal device is in the synchronous display mode;
When the line-of-sight determination means determines that the user's line of sight is facing the display device of the terminal device, and the synchronization determination means determines that the terminal device is in the synchronous display mode And a browsing state determination unit that determines that the user is in a first browsing state browsing the target image in a synchronous display mode.   A monitoring device that receives information indicating a browsing status of a user of the terminal device from the terminal device according to any one of claims 1 to 7, wherein the browsing status indicated by the information received from the terminal device is displayed. A monitoring device comprising display control means. From each of the plurality of terminal devices that display the target image displayed by the target image display device on each display device, information indicating the viewing status of the user of the terminal device is received,
The monitoring apparatus according to claim 9, further comprising: a browsing rate calculating unit that calculates a ratio of users browsing the target image in the synchronous display mode among users of the terminal device. A terminal device having a synchronous display mode for displaying a target image displayed by a target image display device that displays a target image to be a target of synchronous display on the display device of the device itself, and the display of the user of the terminal device A terminal monitoring system including a monitoring device for specifying a browsing state of an image displayed on the device,
Line-of-sight determining means for determining whether the line of sight of the user of the terminal device is facing the display device of the terminal device is provided in the terminal device or the monitoring device,
Synchronization determination means for determining whether or not the terminal device is in the synchronous display mode is provided in the terminal device or the monitoring device,
The information indicating the determination result of the line-of-sight determination unit and the determination result of the synchronization determination unit is acquired, and the line-of-sight determination unit determines that the user's line of sight faces the display device of the terminal device, When the synchronization determination unit determines that the terminal device is in the synchronous display mode, the browsing is determined to be in the first browsing state in which the user is browsing the target image in the synchronous display mode. State determination means is provided in the terminal device or the monitoring device,
The terminal monitoring system, wherein the monitoring device acquires information indicating a determination result of the browsing state determination unit. A control method for a terminal device having a synchronous display mode for displaying a target image displayed by a target image display device that displays a target image to be a target of synchronous display on the display device of the device,
A line-of-sight determination step for determining whether or not the line of sight of the user of the own apparatus is facing the display apparatus of the own apparatus
A synchronization determination step for determining whether or not the device is in the synchronous display mode;
Viewing state notification step for transmitting information indicating the determination result of the line-of-sight determination step and the determination result of the synchronization determination step to a monitoring device that identifies the viewing state of the image displayed on the display device of the user of the device itself A control method for a terminal device. The target image displayed by the target image display device that displays the target image that is the target of the synchronous display is displayed on the display device of the user who uses the terminal device having the synchronous display mode for displaying the target image on the display device of the own device. A monitoring device control method for monitoring a viewing state of an image to be displayed,
A line-of-sight determination step of determining whether or not the user's line of sight is facing the display device of the terminal device;
A synchronization determination step of determining whether or not the terminal device is in the synchronous display mode;
When the line of sight determination step determines that the user's line of sight is facing the display device of the terminal device, and the synchronization determination step determines that the terminal device is in the synchronous display mode, And a browsing state determination step for determining that the user is in a first browsing state in which the user is browsing the target image in a synchronous display mode.   A control program for operating the terminal device according to any one of claims 1 to 7, wherein the control program causes a computer to function as each of the means.   A control program for operating the monitoring device according to any one of claims 8 to 10, wherein the control program causes a computer to function as each of the means.   The computer-readable recording medium which recorded the control program of Claim 14 or 15.

Images