JP2011030063A - Video conference system, server apparatus, and video conference program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video conference system, server apparatus and video conference program by which a video conference can be carried out inexpensively. <P>SOLUTION: A main speaker determining section 21 of a server apparatus 3 determines a main speaker of a video conference and, based on a determination result of the main speaker determining section 21, a video display control section 22 of the server apparatus 3 specifies a first main speaker and a second main speaker. The video display control section 22 then controls a display device 11 of any other point in such a way that video images of the first main speaker and the second main speaker are displayed at different positions on left and right sides, controls a display device 11 at a point, where the first main speaker is located, in such a way that the video image of the second main speaker is displayed at a left/right position different from a video display position of the second main speaker on the display device 11 of the other point, and controls a display device 11 of a point, where the second main speaker is located, in such a way that the video image of the first main speaker is displayed at a left/right position different from the video display position of the first main speaker on the display device 11 of the other point. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video conference system, a server device, and a video conference program for conducting a video conference between attendees at a plurality of points.

  2. Description of the Related Art Conventionally, a video conference system is known in which a video conference is performed between attendees at a plurality of locations by capturing and transmitting images of attendees at the plurality of locations using an imaging device and a display device. In such a video conference system, in order to give attendees a sense of realism that the same conference is being held, it is an important issue to match attendees' line of sight with each other.

  From such a background, a display device that synthesizes and displays a plurality of communication destination images on a single screen, and a plurality of images that capture a front image of the communication source from an angle at which the line of sight coincides with the image of the communication destination in conversation. A video conferencing system has been proposed that includes a front-side imaging device and a plurality of side-view imaging devices that capture a video of a communication source's profile from an angle at which the lines of sight of attendees in conversation match. ).

  This video conferencing system detects which attendee (conversation source) is talking to which attendee. The video conferencing system then transmits a corresponding video out of the video captured by the plurality of front imaging devices to the communication destination during the conversation according to the detection result, and the video captured by the plurality of profile imaging devices. Is transmitted to a communication destination other than the communication destination in conversation.

  According to the conventional video conferencing system, at least {(number of attendees of video conference −1) × 2} must be installed at each point where attendees are present. Along with this, the required number of installed image pickup devices increases, making it difficult to conduct video conferencing at low cost.

  The present invention has been made in view of the above, and an object of the present invention is to provide a video conference system, a server device, and a video conference program capable of performing a video conference at low cost.

  In order to solve the above-described problems and achieve the object, the present invention provides a plurality of video conference terminal devices arranged at points where attendees of a video conference are present, and a plurality of video conference terminals via telecommunication lines. Each of the video conference terminal devices includes a display device that displays images of other attendees at least at two different positions on the left and right sides of the attendant facing the display screen of the display device. An image pickup device that captures video, and the server device includes a main speaker determination unit that determines a main speaker of a video conference, and a video display control unit that controls a video of an attendee displayed by the display device, The video display control unit identifies the first and second main speakers based on the determination result of the main speaker determination unit, and the first main speaker and the second main speaker captured by the imaging device. The position of the video is different The display device of the point where the attendees other than the first and second main speakers are present is controlled so that the video of the second main speaker captured by the imaging device is displayed. The display of the point where the first main speaker is located so that it is displayed in the left and right positions different from the video display position of the second main speaker in the display device of the point where the attendees other than the second main speaker are present An image of the first main speaker in a display device at a point where an attendee other than the first and second main speakers is present, with the video of the first main speaker captured by the imaging device controlled by the apparatus The display device at the point where the second main speaker is located is controlled so as to be displayed at the left and right positions different from the display position.

  In order to solve the above-described problems and achieve the object, the present invention provides a main speaker determination unit that determines a main speaker of a video conference from among video conference attendees at a plurality of points, A video display control unit for controlling the video of the attendee to be displayed on a display device arranged at each point where the attendee is present, the video display control unit based on the determination result of the main speaker determination unit Attendees other than the first and second main speakers are identified so that the second main speaker is identified and the images of the first main speaker and the second main speaker are displayed at different positions. The video display position of the second main speaker in the display device of the point where the attendees other than the first and second main speakers are present is controlled. The first main speaker is displayed so that it is displayed at a different left and right position. The display device of the first main speaker is controlled, and the video display position of the first main speaker in the display device of the point where attendees other than the first and second main speakers are present is controlled. Controls the display device at the point where the second main speaker is located so that they are displayed at different left and right positions.

  In order to solve the above-described problems and achieve the object, the present invention provides a main speaker determination step for determining a main speaker of a video conference from among video conference attendees at a plurality of points, And a video display control step for controlling the video of the attendee to be displayed on the display device arranged at each point where the attendee is present. The video display control step is performed based on the result of the main speaker determination step. Steps for identifying the first and second main speakers, and other than the first and second main speakers so that the images of the first main speaker and the second main speaker are displayed at different positions on the left and right Controlling the display device at the point where the attendee is present and the video of the second main speaker is the second main in the display device at the point where attendees other than the first and second main speakers are present Video display of the speaker A step of controlling the display device at the point where the first main speaker is located so that the left and right positions are different from the position of the device, and images of the first main speaker are the first and second main messages. The display device at the point where the second main speaker is located is controlled such that the display device is displayed at a left and right position different from the video display position of the first main speaker in the display device where the attendee other than the present person is present. Steps.

  According to the present invention, since the required number of installed image pickup devices does not depend on the number of attendees of the video conference, the video conference can be performed at a low cost.

FIG. 1 is a block diagram showing a configuration of a video conference system according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing a screen configuration of the display device shown in FIG. FIG. 3 is a schematic diagram showing a layout of the left camera and the right camera shown in FIG. FIG. 4 is a block diagram showing an internal configuration of the main speaker determination unit shown in FIG. FIG. 5 is a schematic diagram illustrating a display output example of the display device illustrated in FIG. 1. FIG. 6A is a diagram for explaining an image captured by the right camera when the attendee is looking at the right screen area, and FIG. 6B is a view of the attendee looking at the right screen area. FIG. 6C is a diagram for explaining an image photographed by the right camera when the attendee is looking at the left screen area. FIG. 6D is a diagram for explaining an image captured by the left camera when the attendee is looking at the left screen area. FIG. 7 is a schematic diagram illustrating an example of attendee images displayed on the display devices at the points P1 to P4. FIG. 8 is a flowchart showing the flow of the video display control process. FIG. 9 is a flowchart showing a subroutine of the process in step S4 shown in FIG. FIG. 10 is a flowchart showing a subroutine of the process in step S5 shown in FIG. FIG. 11 is a flowchart showing a subroutine of the process in step S23 shown in FIG. FIG. 12 is a flowchart showing a subroutine of the process in step S24 shown in FIG. FIG. 13 is a flowchart showing a subroutine of the process of step S25 shown in FIG. FIG. 14 is a flowchart showing a subroutine of the process in step S6 shown in FIG. FIG. 15 is a flowchart showing a subroutine of the process in step S8 shown in FIG. FIG. 16 is a schematic diagram illustrating a display screen example of the display device of the attendees at the other point n, the first main speaker, and the second main speaker. FIG. 17 is a diagram illustrating an example of a change in the display screen accompanying a change in the main speaker.

  Hereinafter, the configuration and operation of a video conference system according to an embodiment of the present invention will be described with reference to the drawings.

[Configuration of video conferencing system]
First, the configuration of the video conference system according to the embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 1, the video conference system 1 according to the embodiment of the present invention controls the video conference terminal device 2 provided at each point where the attendee of the video conference is present and the operation of the video conference terminal device 2. The video conference terminal device 2 and the server device 3 are configured to be capable of communicating information with each other via the telecommunication line 4.

[Configuration of video conference terminal]
The video conference terminal device 2 includes a display device 11 and an imaging device 12. The display device 11 is configured by a known display device such as a liquid crystal display device or a CRT (Cathode Ray Tube) device, and as shown in FIG. 2, a left screen region 31, a right screen region 32, a lower screen region 33, and a statement Rate display areas 34 and 35 are provided. The left screen area 31, the right screen area 32, and the lower screen area 33 display images of attendees of the video conference transmitted from the server device 3. In the present embodiment, as shown in FIG. 3, the left screen area 32 and the right screen area 33 are located at the same distance from the center position to the center position of the display screen (distance a shown in FIG. 3). Has been placed.

  The speech rate display area 34 and the speech rate display area 35 display the ratio (speech rate) of the attendee's speech time displayed in the left screen region 31 and the right screen region 32 during the video conference time, respectively. In this embodiment, the length of the black bar displayed in the speech rate display areas 34 and 35 indicates the speech rate of the attendee. That is, in this embodiment, when the black bar is not displayed in the speech rate display areas 34 and 35, the attendance rate of the attendee is 0%, and the black bar extends from the left end to the right end of the speech rate display areas 34 and 35. If displayed, the attendance rate for attendees is 100%.

  The imaging device 12 includes a left camera 12a and a right camera 12b that capture images of attendees (operators) facing the display device 11. As shown in FIG. 3, the left camera 12 a is arranged at a position where the angle formed by the line-of-sight direction and the imaging direction when the operator is viewing the attendee's image displayed in the left screen area 31 is θ. Then, an image near the front of the operator is taken at an angle θ. On the other hand, the right camera 12b is arranged at a position where the angle formed by the line-of-sight direction and the imaging direction when the operator is viewing the attendee's video displayed in the left screen area 31 is β. An image of the profile of the operator who is watching is taken at an angle β.

  In order to match the line of sight of the attendee displayed in the left screen area 31 or the right screen area 32 with the line of sight of the operator, the left camera 12a and the right gamer 12b are arranged so that the angle θ is as small as possible. It is desirable. In order to correctly photograph the operator's profile, it is desirable to arrange the left camera 12a and the right camera 12b so that the angle β falls within the range of 60 to 90 °. However, in order to make it easy to distinguish the front image and the profile image of the operator, it is desirable that the angle β is at least twice the angle θ.

[Configuration of server device]
The server device 3 is configured by a known information processing device such as a workstation, and includes a main speaker determination unit 21 and a video display control unit 22. The functions of the main speaker determination unit 21 and the video display control unit 22 are realized on the server device 3 by a CPU (Central Processing Unit) inside the information processing device reading out and executing a video conference program from a storage medium. It is like that.

  The video conferencing program is a file in an installable or executable format and is recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). Provided. Further, the video conference program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the video conference program may be provided or distributed via a network such as the Internet. Further, the video conference program may be provided by being incorporated in advance in a ROM or the like.

  As shown in FIG. 4, the main speaker determination unit 21 includes a volume detection unit 41, a volume history storage unit 42, a main message book specification unit 43, and a speech time calculation unit 44. The volume detection unit 41 receives the attendee's video from the video conference terminal device 2 at each point where the video conference attendee is present, and detects the speech volume of each attendee based on the received video information. The volume history storage unit 42 stores the speech volume of each attendee detected by the volume detection unit 41 for each unit time. The main speaker specifying unit 43 determines the average utterance volume within a predetermined time based on the utterance volume per unit time of each attendee stored in the volume history storage unit 42 (for example, the average utterance volume for the last 5 to 10 seconds). And the attendee with the highest calculated average utterance volume is identified as the main speaker. The speech time calculation unit 44 calculates the speech rate of each attendee based on the speech volume per unit time of each attendee stored in the volume history storage unit 42.

  The video display control unit 22 controls the video and information displayed on the display device 11 at each point based on the processing result of the main speaker determination unit 21 (the main speaker specifying unit 43 and the speech time calculation unit 44). As will be described in detail later, when a video conference is being held between attendees at the points P1 to P4, when the attendant at the point P1 has a conversation with the attendant at the point P2, the video display control unit 22 For example, as shown in FIG. 5, the images of the attendees at the points P2 and P3 are displayed on the left screen region 31 and the right screen region 32 of the display device 11 at the point P1, respectively.

  When the attendee is looking at the right screen area 32, the right camera 12b captures a video in which the attendee is small and facing the left as shown in FIG. 6A, and the left camera 12a is shown in FIG. 6B. Take a picture of the attendee as shown, pointing to the left. On the other hand, when the attendee is looking at the left screen area 31, the right camera 12b captures an image in which the attendee is greatly facing right as shown in FIG. 6C, and the left camera 12a is shown in FIG. Shoot an image in which the attendee is small and facing right as shown in (d).

  Therefore, when the attendee at the point P1 is talking to the attendee at the point P2, the video display control unit 22 performs the left screen region 31 and the right screen region of the display device 11 at the point P1, as shown in FIG. In 32, an image in which the attendee at the point P2 is small and facing right and an image in which the attendee at the point P3 is facing large and left are displayed. In addition, the video display control unit 22 displays the video in which the attendee at the point P4 is greatly facing the right and the attendant at the point P1 is small in the left screen region 31 and the right screen region 32 of the display device 11 at the point P2. Display the video you are facing. In addition, the video display control unit 22 includes a video in which the attendee at the point P1 is greatly facing the right and the attendant at the point P2 in the left screen region 31 and the right screen region 32 of the display device 11 at the point P3 and the point P4, respectively. Display a video that is facing large left.

[Video display control processing]
In such a video conference system 1, the server device 3 can perform a video conference at a low cost by executing the following video display control process. The operation of the server device 3 when executing this video display control process will be described below with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 8 starts when the server apparatus 3 is instructed to start a video conference, and the video display control process proceeds to step S1.

  In the process of step S1, the video display control unit 22 executes initial setting of the video display control process. Specifically, the video display control unit 22 designates the attendee whose primary order is the first as the “first primary speaker” based on the primary order of the attendees of the video conference input by the video conference organizer. , The second participant in the main order is set as “second main speaker”, and the point where the third participant in the main order is set as “default point”.

  The video display control unit 22 displays the video of the “second main speaker” and the left captured by the right camera 12b in the left screen region 31 and the right screen region 32 of the display device 11 of “first main speaker”, respectively. The display screen of the display device 11 of the “first main speaker” is set so as to display the video of the attendee at the “default location” photographed by the camera 12a. The video display control unit 22 displays the video of the attendee at the “default location” taken by the right camera 12b in the left screen area 31 and the right screen area 32 of the display device 11 of “second main speaker” and the left. The display screen of the display device 11 of the “second main speaker” is set so as to display the video of the “first main speaker” taken by the camera 12a.

  The video display control unit 22 displays the left side of the display device 11 of attendees other than “first main speaker” and “second main speaker” (participants in “other locations” including “default location”). An image of “first main speaker” captured by the right camera 12b and an image of “second main speaker” captured by the left camera 12a are displayed in the screen area 31 and the right screen area 32, respectively. The display screen of the “other point” display device 11 is set. The video display control unit 22 then classifies the information set for each attendee, that is, the types of “first main speaker”, “second main speaker”, “default location”, and “other locations”. Information and display information indicating video to be displayed on the left screen area 31 and the right screen area 32 of the display device 11 are stored as attribute information of each attendee. Thereby, the process of step S1 is completed and the video display control process proceeds to the process of step S2.

  In the process of step S <b> 2, the video display control unit 22 acquires information on the main speaker from the main speaker determination unit 21. Thereby, the process of step S2 is completed and the video display control process proceeds to the process of step S3.

  In the process of step S3, the video display control unit 22 determines whether or not the main speaker has changed by comparing the attendee's attribute information with the information of the main speaker acquired by the process of step S2. . If the main speaker is not changed as a result of the determination, the video display control unit 22 advances the video display control process to the process of step S7. On the other hand, when the main speaker changes, the video display control unit 22 advances the video display control process to the process of step S4.

  In the process of step S4, the video display control unit 22 specifies new main speakers (“first main speaker” and “second main speaker”), and attributes of each attendee based on the specified result. Update information (main speaker identification process). Details of the main speaker specifying process will be described later with reference to a flowchart shown in FIG. Thereby, the process of step S4 is completed, and the video display control process proceeds to the process of step S5.

  In the process of step S5, the video display control unit 22 displays the video in the left screen area 31 and the right screen area 32 of each attendee's display device 11 in accordance with the attribute information of each attendee updated in the process of step S4. Attendees to be determined (video display point determination processing). Details of this video display point determination processing will be described later with reference to the flowcharts shown in FIGS. Thereby, the process of step S5 is completed, and the video display control process proceeds to the processes of step S6 and step S7.

  In the process of step S6, the video display control unit 22 displays the attendee video (left camera 12a) displayed on the left screen area 31 and the right screen area 32 of the display device 11 of each attendee based on the processing result of step S5. Alternatively, the image captured by the right camera 12b) is identified (image identification process). Details of this video specifying process will be described later with reference to a flowchart shown in FIG. Thereby, the process of step S6 is completed, and the video display control process proceeds to the process of step S8.

  In the process of step S7, the video display control unit 22 sends a speech rate of attendees at the “first main speaker”, “second main speaker”, and “default location” from the main speaker determination unit 21. Get information about. Thereby, the process of step S7 is completed, and the video display control process proceeds to the process of step S8.

  In the process of step S8, the video display control unit 22 selects the video to be transmitted to the display device 11 at each point based on the process result of step S6, and is selected together with the information regarding the speech rate acquired by the process of step S7. The video is transmitted to the display device 11 at each point (information transmission process). Details of this information transmission processing will be described later with reference to the flowchart shown in FIG. Thereby, the process of step S8 is completed, and the video display control process proceeds to the process of step S9.

  In the process of step S9, the video display control unit 22 determines whether or not a video conference end instruction has been input. As a result of the determination, when the video conference end instruction is not input, the video display control unit 22 returns the video display control process to the process of step S2. On the other hand, when a video conference end instruction is input, the video display control unit 22 ends the series of video display control processing.

[Key speaker identification processing]
Next, the main speaker specifying process in step S4 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 9 starts at the timing when it is determined that the main speaker has changed in the process of step S3, and the main speaker specifying process proceeds to the process of step S11.

  In the process of step S11, the video display control unit 22 reads the attribute information of each attendee. Thereby, the process of step S11 is completed, and the main speaker specifying process proceeds to the process of step S12.

  In the process of step S12, the video display control unit 22 replaces the classification information in the attribute information with the original classification information (attribute information of each attendee before the main speaker changes). Thereby, the process of step S12 is completed, and the main speaker specifying process proceeds to the process of step S13.

  In the processing of step S13, the video display control unit 22 sets the classification information of attendees classified as “second main speaker” in the original classification information as “first main speaker”. Thereby, the process of step S13 is completed, and the main speaker specifying process proceeds to the process of step S14.

  In the processing of step S14, the video display control unit 22 sets the classification information of the new main speaker as “second main speaker”. Thereby, the process in step S14 is completed, the main speaker specifying process is completed in step S14, and the main speaker specifying process proceeds to the process in step S15.

  In the process of step S15, the video display control unit 22 sets “first main speaker” and “second main speaker” set in “first main speaker” and the original classification information by the processes of steps S13 and S14. Among the attendees excluding the attendees set as attendees at the “first primary speaker” and “default location”, the classification information of the attendee with the highest priority set in the initial setting process is “ Set to “Default point”. Thereby, the process of step S15 is completed, and the main speaker specifying process proceeds to the process of step S16.

  In the process of step S16, the video display control unit 22 includes the “first main speaker” and the “second main speaker” including the attendees classified as attendees at the “default location” by the process of step S15. Set the classification information of attendees other than "Other points". Thereby, the process of step S16 is completed, and the main speaker specifying process proceeds to the process of step S17.

  In the process of step S17, the video display control unit 22 updates the attribute information of each attendee based on the process results of steps S12 to S16. Thereby, the process of step S17 is completed and a series of main speaker specific processes are completed.

[Video display point determination processing]
Next, the video display point determination process in step S5 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 10 starts at the timing when the process of step S4 is completed, and the video display point determination process proceeds to the process of step S21.

  In step S21, the video display control unit 22 reads the attribute information of each attendee. Thereby, the process of step S21 is completed, and the video display point determination process proceeds to the process of step S22.

  In the process of step S22, the video display control unit 22 extracts the classification information, the original classification information, and the display information of each attendee from the attribute information read out by the process of step S21. Thereby, the process of step S22 is completed, and the video display point determination process proceeds to the process of step S23.

  In the process of step S23, the video display control unit 22 determines the location of the attendee who displays the video on the display device 11 of “other location” based on the classification information and the display information extracted by the process of step S22. (First display point determination process). The details of the first display point determination process will be described later with reference to the flowchart shown in FIG. Thereby, the process of step S23 is completed, and the video display point determination process proceeds to the process of step S24.

  In the process of step S24, the video display control unit 22 displays the video on the display device 11 of the “first main speaker” based on the classification information and the display information extracted by the process of step S22. A point is determined (second display point determination process). Details of the second display point determination process will be described later with reference to the flowchart shown in FIG. Thereby, the process of step S24 is completed, and the video display point determination process proceeds to the process of step S25.

In the process of step S25, the video display control unit 22 displays the video on the display device 11 of the “second main speaker” based on the classification information and the display information extracted by the process of step S22. A point is determined (third display point determination process). The details of the third display point determination process will be described later with reference to the flowchart shown in FIG. Thereby, the process of step S25 is completed and a series of video display point determination processes are completed.
[First display point determination process]
Next, the first display spot determination process in step S23 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 11 starts at the timing when the process of step S22 is completed, and the first display point determination process proceeds to the process of step S31.

  In the process of step S31, the video display control unit 22 displays the screen area (the left screen area 31 or the video image of the “first main speaker”) based on the display information extracted by the process of step S22. The right screen area 32) is detected. Thereby, the process of step S31 is completed and a 1st display point determination process progresses to the process of step S32.

  In the process of step S32, the video display control unit 22 sets the display screen so that the video of “second main speaker” is displayed in the screen area opposite to the screen area detected by the process of step S31. . Thereby, the process of step S32 is completed and a 1st display point determination process progresses to the process of step S33.

  In the process of step S33, the video display control unit 22 displays “first main speaker” in a screen area opposite to the screen area that is set to the screen that displays the video of “second main speaker” in the process of step S32. Set the display screen to display the "Speaker" video. Thereby, the process of step S33 is completed, and the first display point determination process proceeds to the process of step S34.

In the process of step S34, the video display control unit 22 updates the display information (attribute information) of attendees classified as attendees at “other locations” based on the processing results of steps S32 and S33. Thereby, the process of step S34 is completed and a series of 1st display point determination processes are complete | finished.
[Second display point determination process]
Next, the second display spot determination process in step S24 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 12 starts at the timing when the process of step S23 is completed, and the second display point determination process proceeds to the process of step S41.

  In the process of step S41, the video display control unit 22 displays the screen area (the left screen area 31 or the video image of the “second main speaker”) based on the display information extracted by the process of step S22. The right screen area 32) is detected. Thereby, the process of step S41 is completed and the second display point determination process proceeds to the process of step S42.

  In the process of step S42, the video display control unit 22 sets the display screen to display the video of “second main speaker” in the screen area opposite to the screen area detected by the process of step S41. . Thereby, the process of step S42 is completed and a 2nd display point determination process progresses to the process of step S43.

  In the process of step S43, the video display control unit 22 sets the screen that displays the video of “second main speaker” in the process of step S42 to the “default location” in the screen area opposite to the set screen area. Set the display screen to display the attendee's video. Thereby, the process of step S43 is completed, and the second display point determination process proceeds to the process of step S44.

In the process of step S44, the video display control unit 22 updates the display information (attribute information) of attendees classified as “first main speaker” based on the processing results of steps S42 and S43. Thereby, the process of step S44 is completed and a series of 2nd display point determination processes are complete | finished.
[Third display point determination process]
Next, the third display point determination process in step S25 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 13 starts at the timing when the process of step S24 is completed, and the third display point determination process proceeds to the process of step S51.

  In the process of step S51, the video display control unit 22 displays the screen area (the left screen area 31 or the video image of the “first main speaker”) based on the display information extracted by the process of step S22. The right screen area 32) is detected. Thereby, the process of step S51 is completed, and the third display point determination process proceeds to the process of step S52.

  In the process of step S52, the video display control unit 22 sets the display screen so that the video of “first main speaker” is displayed in the screen area opposite to the screen area detected by the process of step S51. . Thereby, the process of step S52 is completed, and the third display point determination process proceeds to the process of step S53.

  In the process of step S53, the video display control unit 22 sets “default point” in the screen area opposite to the screen area set as the screen displaying the video of “first main speaker” by the process of step S52. Set the display screen to display the video of attendees in Thereby, the process of step S53 is completed, and the third display point determination process proceeds to the process of step S54.

In the process of step S54, the video display control unit 22 updates the display information (attribute information) of attendees classified as “second main speaker” based on the processing results of steps S52 and S53. Thereby, the process of step S54 is completed and a series of 3rd display point determination processes are complete | finished.
[Video specific processing]
Next, the video specifying process in step S6 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 14 starts at the timing when the process of step S5 is completed, and the video display point determination process proceeds to the process of step S61.

  In the process of step S61, the video display control unit 22 reads the attribute information of each attendee. Thereby, the process of step S61 is completed, and the video specifying process proceeds to the process of step S61.

  In the process of step S62, the video display control unit 22 extracts the classification information, the original classification information, and the display information of each attendee from the attribute information read out in the process of step S61. Thereby, the process of step S62 is completed, and the video specifying process proceeds to the process of step S63.

  In the process of step S <b> 63, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” on which the video of “second main speaker” is displayed. The video of “second main speaker” is set to the video of “second main speaker” displayed on display device 11 of “first main speaker”. Thereby, the process of step S63 is completed, and the video specifying process proceeds to the process of step S64.

  In the process of step S <b> 64, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” on which the video of “first main speaker” is displayed. The video of “first main speaker” is set to the video of “first main speaker” displayed on display device 11 of “second main speaker”. Thereby, the process of step S64 is completed and the video specifying process proceeds to the process of step S65.

  In the process of step S65, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” on which the video of “second main speaker” is displayed. The video of “first main speaker” is set to the video of “first main speaker” displayed on display device 11 of “other location”. Thereby, the process of step S65 is completed, and the video specifying process proceeds to the process of step S66.

  In the process of step S <b> 66, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” on which the video of “first main speaker” is displayed. The video of “second main speaker” is set to the video of “second main speaker” displayed on display device 11 of “other location”. Thereby, the process of step S66 is completed, and the video specifying process proceeds to the process of step S67.

  In the process of step S67, the video display control unit 22 detects the attendees classified as attendees at the “default location” based on the classification information extracted by the process of step S62. Thereby, the process of step S67 is completed, and the video specifying process proceeds to the process of step S68.

  In the process of step S <b> 68, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” on which the video of “first main speaker” is displayed. The video of the attendee at the “default location” is set to the video of the attendee at the “default location” displayed on the display device 11 of the “first main speaker”. Thereby, the process of step S68 is completed and the video specifying process proceeds to the process of step S69.

  In the process of step S <b> 69, the video display control unit 22 is photographed by the imaging device on the same side as the screen area of the display device 11 of “other point” where the video of “second main speaker” is displayed. The video of the attendee at the “default location” is set as the video of the attendee at the “default location” displayed on the display device 11 of the “second main speaker”. Thereby, the process of step S68 is completed and the video specifying process proceeds to the process of step S69.

  In the process of step S70, the image display control unit 22 has the imaging devices on both sides of the attendees other than the attendees at the “first main speaker”, the “second main speaker”, and the “default location”. Is set as the attendee's video to be displayed in the lower screen area 33 of the display device 11 of “first main speaker” and “second main speaker”. Thereby, the process of step S70 is completed, and the video specifying process proceeds to the process of step S71.

  In the process of step S71, the video display control unit 22 captures images on both sides of attendees other than the attendees at the “first main speaker”, the “second main speaker”, and the “other point n”. The video shot by the device is set as the video of the attendee to be displayed in the lower screen area 33 of the display device 11 of “other point n”. Thereby, the process of step S71 is completed, and a series of video specifying processes ends.

[Information transmission processing]
Finally, the information transmission process in step S8 will be described in detail with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 15 starts at the timing when the processes in steps S6 and S7 are completed, and the information transmission process proceeds to the process in step S81.

  In the process of step S81, the video display control unit 22 receives videos taken by the left camera 12a and the right camera 12b of the imaging device 12 of each attendee. Thereby, the process of step S81 is completed, and the information transmission process proceeds to the process of step S82.

  In the process of step S82, the video display control unit 22 acquires the information set by the process of step S6 and the information of the speech rate acquired by the process of step S7. Thereby, the process of step S82 is completed and the information transmission process proceeds to the process of step S83.

  In the process of step S83, the video display control unit 22 selects a video to be transmitted to each attendee's display device 11 from the video received by the process of step S81, based on the information acquired by the process of step S82. . Thereby, the process of step S83 is completed, and the information transmission process proceeds to the process of step S84.

  In the process of step S84, the video display control unit 22 displays the video to be displayed in the left video area 31, the right video area 32, and the lower video area 33 together with information on the speech rate based on the processing result of step S83. To the display device 11. Specifically, as shown in FIG. 16, videos of “first main speaker” and “second main speaker” in the left screen region 31 and the right screen region 32 of the display device 11 at another point n, respectively. Is displayed on the same side as the right screen area 32 (left screen area 31) on which the video of “second main speaker” (“first main speaker”) is displayed. The video of the “first main speaker” (“second main speaker”) taken by the right camera 12b (left camera 12a) set to “second main speaker” (first It is transmitted as an image of “first main speaker” (“second main speaker”) displayed on the display device 11 of the main speaker. When the other point n is not the “default point”, the video display control unit 22 transmits the video shot by the left camera 12a and the right camera 12b at the other point n to each point, and the other point n is the “default point”. Is the video of the attendee at another point n taken by the left camera 12a set on the same side as the left screen area 31 on which the video of “first main speaker” is displayed. It is transmitted as an image of the attendee at the “default location” displayed on the display device 11 of “first main speaker” and “second main speaker”. Thereby, the process of step S84 is completed and a series of information transmission processes are completed.

  The above video display control process will be described more specifically as follows. Consider a video conference between attendees at points A through F. If the organizer of the video conference sets the main order of attendees in the order of point F, point C, point A, point D, point B, and point E in the initialization process, “first main speaker”, “ The attendees at “second main speaker” and “default location” are the attendees at location F, location C, and location A in that order. Accordingly, at this stage, as shown in FIG. 17, the images of the attendees at the points C and F are displayed on the left screen region 31 and the right screen region 32 of the display device 11 at another point, respectively. In the left screen area 31 and the right screen area 32 of the “speaker” display device 11, the images of the attendees at the points A and C are displayed, respectively, and the left screen region of the “second main speaker” display device 11. In 31 and the right screen area 32, images of attendees at the points F and A are displayed, respectively.

  Next, assuming that the main speaker changes in the order of point B → point C → point B → point D → point F → point A as shown in FIG. 17, the attendee described above at the stage where the main speaker is specified. After replacing the category of (the first primary speaker, the second primary speaker, and the attendees at the default location) with the original classification, the attendees at location C and location B were each It is classified into “first main speaker” and “second main speaker”. The video images of the attendees at the points C and B are displayed on the left screen region 31 and the right screen region 32 of the display device 11 at other points, respectively, so that the display position of the “first main speaker” is not changed. To do. In addition, in the left screen area 31 and the right screen area 32 of the display device 11 of the “first main speaker” (the attendee at the point C), a “second main speaker” (the attendee at the point B), respectively. And the video of the attendee at the “default location” (the attendee at the location D) is displayed, and the left screen area 31 and the right of the display device 11 of the “second main speaker” (the attendee at the location B) In the screen area 32, images of “attendees at the default location” and “first main speaker” are displayed. In this case, the “default point” excludes “first main speaker” and “second main speaker” (preferably “first main speaker” and “default point” in the original classification information). (Excluding attendees categorized as attendees in) (showing the location of the attendee with the highest primary order among attendees).

  Next, when the main speaker changes from an attendee at point B to an attendee at point C, the classification information when the main speaker is an attendee at point B at the timing when the main speaker changes Is replaced with the original classification, and the attendees at point B and point C are classified as "first main speaker" and "second main speaker", respectively. In the left screen area 31 and the right screen area 32 of the display device 11 at another point, the images of the attendees at the point C and the point B are displayed, respectively. The left screen region 31 and the right screen region 32 of the display device 11 of the “first main speaker” (the attendee at the point B) each have an attendee at the “default point” (the attendee at the point F). And the video of the “second main speaker” (the attendee at the point C) is displayed, and the left screen region 31 and the right of the display device 11 of the “second main speaker” (the attendee at the point C) In the screen area 32, images of “first main speaker” and “attendees at the default location” are displayed. Thereafter, the same processing is repeated.

  As is clear from the above description, in the video conference system 1 according to the embodiment of the present invention, the main speaker determination unit 21 of the server device 3 determines the main speaker of the video conference and displays the video on the server device 3. The control unit 22 identifies the “first main speaker” and the “second main speaker” based on the determination result of the main speaker determination unit 21. Then, the video display control unit 22 controls the display device 11 for “other points” so that the videos of the “first main speaker” and the “second main speaker” are displayed at different positions on the left and right. The “first main speaker” is displayed so that the video of the “second main speaker” is displayed at the left and right positions different from the video display position of the “second main speaker” on the display device 11 of “other points”. The display device 11 at the point where the sound is present is controlled, and the video of the “first main speaker” is displayed at the left and right positions different from the video display position of the “first main speaker” on the display device 11 of “other points”. In this manner, the display device 11 at the point where the “second main speaker” is present is controlled.

  That is, in the video conference system 1 according to the embodiment of the present invention, the video display control unit 22 of the server device 3 controls the display device 11 at each point based on the main speaker determined by the main speaker determination unit 21. Decide who attends to view the video. According to such a configuration, since the necessary number of installed image pickup devices does not depend on the number of attendees of the video conference, the video conference can be performed at a low cost. Even if the number of attendees increases during the conference, it is not always necessary to add imaging devices for the increased attendees, so that the video conference can proceed smoothly. Further, it is possible to display the line-of-sight matching video in which the main speaker faces the other attendee's display device 11 without using a special device such as a curved screen with a special wide viewing angle.

  In the video conference system 1 according to the embodiment of the present invention, the video display control unit 22 replaces the video of the “second main speaker” with the video of the new main speaker when the main speaker changes. The “other point” display device 11 is controlled. According to such a configuration, even if the main speaker changes, the video of the “first main speaker” before the main speaker changes is displayed at the same position. The change is reduced and the video conference can proceed smoothly.

  In the video conference system 1 according to the embodiment of the present invention, the main speaker determination unit 21 determines the main speaker of the video conference based on the utterance volume of the attendee attending the video conference. The speaker can be accurately determined. Further, in the video conference system 1 according to the embodiment of the present invention, the main speaker determination unit 21 calculates the ratio of the speaking time of each attendee during the conference time of the video conference as the speaking rate of each attendee, and the video The display control unit 22 acquires information on the speech rate of the attendee who displays the video on the display device 11 from the main speaker specifying unit 21, and displays the acquired information on the speech rate together with the video of the attendee. 11, it is possible to predict and compare the attendees' statements, and the operator can take appropriate actions.

  In the video conference system 1 according to the embodiment of the present invention, the imaging device is such that the left camera 12a and the right camera 12b view one of the images of the two attendees displayed on the display device 11. The angle β formed by the operator's line-of-sight direction and the imaging direction of the right camera 12b is at least twice the angle θ formed by the line-of-sight direction and the imaging direction of the left camera 12a, and 60 degrees or more and 90 degrees or less. Therefore, it is possible to accurately shoot a line-of-sight image in which the main speaker is facing.

  As mentioned above, although embodiment which applied the invention made | formed by this inventor was demonstrated, this invention is not limited with the description and drawing which make a part of indication of this invention by this embodiment. That is, other embodiments, examples, operational techniques, and the like made by those skilled in the art based on the above-described embodiments are all included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Video conference system 2 Video conference terminal device 3 Server apparatus 4 Electric communication line 11 Display apparatus 12 Imaging device 12a Left camera 12b Right camera 21 Main speaker determination part 22 Video display control part 31 Left screen area 32 Right screen area 33 Lower screen Areas 34 and 35 Speech rate display area 41 Volume detection unit 42 Volume history storage unit 43 Main speaker identification unit 44 Speech time calculation unit

Japanese Patent No. 3587106

Claims (7)

A plurality of video conference terminal devices arranged at each point where the attendees of the video conference are present;
A server device connected to the plurality of video conference terminal devices via a telecommunication line,
Each video conference terminal device
A display device for displaying images of other attendees at least at two different positions on the left and right;
An imaging device for capturing images of attendees facing the display screen of the display device,
The server device
A main speaker determination unit for determining a main speaker of the video conference;
A video display control unit for controlling the video of attendees displayed by the display device,
The video display control unit
Identifying the first and second main speakers based on the determination result of the main speaker determination unit,
Attendees other than the first and second main speakers such that the images of the first main speaker and the second main speaker captured by the imaging device are displayed at different positions on the left and right. Control the display device at the point where
The video of the second main speaker taken at the display device at the point where the attendees other than the first and second main speakers are present is the video of the second main speaker captured by the imaging device. Controlling the display device at the point where the first main speaker is located so that the left and right positions are different from the display position;
The video of the first main speaker taken at the display device at the point where the attendees other than the first and second main speakers are present is the video of the first main speaker captured by the imaging device. A video conference system, wherein a display device at a point where the second main speaker is located is controlled so as to be displayed at a left and right position different from a display position.   The video display control unit determines whether or not the main speaker has changed based on the determination result of the main speaker determination unit. If the main speaker has changed, the video display control unit displays the video of the second main speaker. 2. The video conference system according to claim 1, wherein a display device at a point where attendees other than the first and second main speakers are present is controlled so as to be replaced with a video of a new main speaker.   The said main speaker determination part determines the main speaker of the said video conference based on the utterance volume of the participant who attends the said video conference, The Claim 1 or Claim 2 characterized by the above-mentioned. Video conferencing system.   The main speaker determination unit calculates a speech rate of each attendee during the meeting time of the video conference as a speech rate of each attendee, and the video display control unit displays a video on the display device. The information about the speaking rate of the attending attendee is acquired from the main speaker determining unit, and the display device is controlled to display the acquired information regarding the speaking rate together with the video of the attendee. The video conference system according to claim 3.   The imaging device includes a first imaging device that captures an attendee's video from a first imaging direction and a second imaging device that captures an attendee's video from a second imaging direction. The second imaging device has an angle formed by the operator's line-of-sight direction and the first imaging direction when viewing one of the two attendee images displayed on the display device. 5. The apparatus according to any one of claims 1 to 4, wherein the lens is disposed at a position that is at least twice as large as an angle formed by the line-of-sight direction and the second imaging direction and within a range of 60 degrees to 90 degrees. The video conference system according to any one of the above. A main speaker determination unit that determines a main speaker of a video conference from among video conference attendees at a plurality of points;
A video display control unit that controls the video of attendees to be displayed on a display device arranged at each point where the attendees of the video conference are present,
The video display control unit
Identifying the first and second main speakers based on the determination result of the main speaker determination unit,
A display device for a point where attendees other than the first and second main speakers are present so that the images of the first main speaker and the second main speaker are displayed at different positions on the left and right. Control
The left and right positions where the video of the second main speaker is different from the video display position of the second main speaker in a display device at a point where attendees other than the first and second main speakers are present The display device of the point where the first main speaker is located,
Left and right positions where the video of the first main speaker is different from the video display position of the first main speaker in a display device at a point where attendees other than the first and second main speakers are present The server device is characterized in that the display device at the point where the second main speaker is located is controlled so as to be displayed on the screen. A main speaker determination step for determining a video conference main speaker from video conference attendees at a plurality of points;
Causing the computer to execute a video display control step of controlling the video of the attendee to be displayed on a display device arranged at each point where the attendee of the video conference is present;
The video display control step includes:
Identifying first and second primary speakers based on the result of the primary speaker determination step;
A display device for a point where attendees other than the first and second main speakers are present so that the images of the first main speaker and the second main speaker are displayed at different positions on the left and right. Controlling step;
The left and right positions where the video of the second main speaker is different from the video display position of the second main speaker in a display device at a point where attendees other than the first and second main speakers are present Controlling a display device at the point where the first primary speaker is present, as shown in FIG.
Left and right positions where the video of the first main speaker is different from the video display position of the first main speaker in a display device at a point where attendees other than the first and second main speakers are present A video conferencing program comprising: controlling a display device at a point where the second main speaker is located so as to be displayed on the screen.

Images