JP2010074668A - Conference terminal device and server apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a conference terminal device capable of holding a natural conference by displaying a video image from a transmission source device from a direction intuitively imaged in a view from a user who operates the conference terminal device in a multi-point television conference system. <P>SOLUTION: A conference terminal device includes: a receiving unit for receiving video data from first and second terminal devices; a video output unit for outputting the received video data onto two display devices disposed at right and left sides; an acquisition unit for acquiring geographical positional information of the conference terminal device and of the first and second terminal devices; a determination unit for determining, by using the positional information, whether the first and second terminal devices are geographically located at the left or right sides with the conference terminal device as a criterion; and an assignment unit for determining assignment so as to output the video data from the first or second terminal device determined as the right side or the left side by the determination unit onto the right-side or left-side display device. In accordance with the assignment, the video output unit outputs the received video data onto the corresponding display device. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a conference terminal device and a server device. In particular, the present invention relates to a conference terminal device and a server device for participating in a conference between at least three points through at least three conference terminal devices via a network.

  In recent years, broadband networks have become widespread, and television (video) telephones and television (video) conferences on broadband networks are widely used.

  Conventionally, a conference terminal device used for a video phone or a video conference can only transmit and receive a low resolution video at a low bit rate. In recent years, due to improvements in network performance and the like, these conference terminal apparatuses can use high-definition video (HD (High Definition) video). Moreover, such a conference terminal device is spreading.

  In addition, a plurality of HD videos captured by a plurality of cameras are transmitted to the counterpart conference terminal device, and a plurality of HD videos are received and output to a plurality of display devices (displays) of the conference terminal device. A conference terminal device of a video conference system called a telepresence system, which has improved the performance, is also beginning to be used.

  For example, a conference terminal device of a video conference system that receives a plurality of HD videos and outputs them to a plurality of regions divided on one display device (display) of the self-conference terminal device has been proposed (Patent Document 1 below). ).

  Hereinafter, a method for outputting a plurality of received video data and audio data in a conventional video conference system will be described with reference to the drawings.

  FIG. 13 is a diagram for explaining screen display when one display is connected to the conference terminal device in the conventional video conference system.

  In the video conference system shown in FIG. 13, a conference terminal device 801, a conference terminal device 802, and a conference terminal device 803 are connected to each other via a network. In addition, one display device (display) is connected to each of the conference terminal device 801, the conference terminal device 802, and the conference terminal device 803, and further, an audio input device, an audio reproduction device, a video input device, and the like are connected. .

  The conference terminal device 801 receives video data and audio data from the conference terminal device 802 and the conference terminal device 803, and displays the video received from each of them in one of the areas divided on one display device 8010. In addition, the video being transmitted by itself is also displayed in one of the areas as the own video.

  At this time, as a method of deciding which video is to be displayed at which position of the divided area, a method of allocating from the upper left position of the divided area in the order of participation in the conference, the own video is divided into smaller areas or divided There is a method of assigning to a fixed position such as an upper right position in an area. Or, it is a method of linking with voice recognition or the like, and assigning video data from the conference terminal device that is speaking at that time to a predetermined position such as a large divided area or a left upper position in the divided area.

  Here, the video data of the conference terminal device 801, the conference terminal device 802, and the conference terminal device 803 displayed on the display device 8010 uses the conference terminal device 801, the conference terminal device 802, and the conference terminal device 803, respectively. User A, user B, and user C participating in the conference. In the example of FIG. 13, the display device 8010 is divided into four areas 8012, 8013, 8014, and 8015.

  As for audio, there is a method of synthesizing and reproducing all audio without distinguishing the position of the area where the video data is displayed, and a method of reproducing only the audio of the conference terminal device of the speaking participant. used.

  The same applies to a case where a plurality of display devices and a plurality of sound reproduction devices (speakers) are connected to the conference terminal device 801, the conference terminal device 802, and the conference terminal device 803, respectively. In other words, the conference terminal devices 801 to 803 determine which display device from which the other conference terminal device displays the video data according to the order of participation in the conference. FIG. 14 shows an example in which three display devices are connected to the conference terminal device.

  FIG. 14 is a diagram for explaining a screen display when three display devices (displays) are connected to a conference terminal device in a conventional video conference system.

  In FIG. 14, as in FIG. 13, the conference terminal device 901, the conference terminal device 902, and the conference terminal device 903 are connected to each other via a network.

The conference terminal device 901 is connected to three display devices 9010, 9011, and 9012. In the example of FIG. 14, the conference terminal device 901 displays its own video (user A) on the left display device 9010, displays the video from the conference terminal device 902 (user B) on the right display device 9012, and A video (user C) from the terminal device 903 is displayed on the central display device 9011.
Japanese Patent No. 3604734

  However, in the conventional conference terminal device, at the time of the conference, the display position of the video from the partner conference terminal device is only determined by the connection order to the video conference system or the speaker in the conference participant. In other words, the display position of the video from the partner conference terminal device is determined without considering the geographical location information of the partner conference terminal device (particularly, the relative positional relationship between the host conference terminal device and the partner conference terminal device). There is a problem that will be done.

  The problem will be described below.

  FIG. 15 is a diagram for explaining a problem in the conventional video conference system. Components similar to those in FIG. 14 are denoted by the same reference numerals, and detailed description thereof is omitted.

  In FIG. 15, it is assumed that the conference terminal device 902 is in the north and the conference terminal device 903 is in the east, based on the location of the conference terminal device 901. In this case, in the conference terminal device 901, the video (user B) is displayed on the left display device 9010 from the conference terminal device 902, and the audio is output from the vicinity of the left display device 9010. It is natural to match the intuitive image (impression) of user A to be used. The video of the conference terminal device 903 is displayed on the right display device 9012, and the sound is output from the vicinity of the right display device 9012, which matches the intuitive image of the user A who uses the conference terminal device 901. And natural.

  However, such a point is not considered in the conventional conference terminal device. In the above-described conventional conference terminal device, as shown in FIG. 13, the display position of the video from the partner conference terminal device is determined by the order of connection to the video conference system or the speaker in the conference participant during the conference. .

  Therefore, the present invention solves the above-mentioned problem, and the video from the conference terminal device as the transmission source from the direction that is intuitively viewed from the user operating the conference terminal device in the multi-point video conference system. Is displayed, and it is an object to provide a conference terminal device and a server device capable of performing a natural conference.

  In order to solve the above problems, a conference terminal device according to the present invention is a conference terminal device for performing a video conference between at least three points via a network with the first and second terminal devices. The video data received by the data receiver is output to a data receiver that receives video data from the first and second terminal devices and at least two display devices arranged side by side in the left-right direction. Using the position information, the position information acquisition section for acquiring geographical position information of the conference terminal device, the first terminal device, and the second terminal device, and the first information using the position information. And each of the second terminal devices is determined to be in a right position by a determination unit that determines whether the second terminal device is geographically left or right with respect to the conference terminal device, and the determination unit. Video data from the first terminal device or the second terminal device is output to a display device arranged in the right direction among at least two display devices arranged in the left-right direction, and the position on the left side The video data from the first terminal device or the second terminal device determined to be in the display device is displayed on the display device arranged in the left direction among the at least two display devices arranged in the left-right direction. The video output unit outputs the video data received by the data receiving unit to the corresponding display device according to the allocation determined by the allocation unit. It is characterized by that.

  According to this configuration, the geographical location information of the other first and second conference terminal devices participating in the video conference and the conference terminal device that is the own terminal is obtained, and the obtained location information is used to obtain the first location information. The left and right positions when viewed from the own terminal in the first or second conference terminal device are determined. Based on the determination, the video from the first or second conference terminal device at the right position (the former video) is transferred to the right display device (the display device that is not the leftmost) and the other first or second video. The assignment of the video from the conference terminal device (the latter video) is determined so as to be output to the display device on the left side of the right-side display device that displays the former video. As a result, the video from the first or second conference terminal device that is geographically on the right side when viewed from the conference terminal device that is the terminal is displayed from the display device on the right side when viewed from the user, and geographically The video from the first or second conference terminal device on the left side is displayed from the display device in the left side direction. Therefore, it is possible to hold a natural meeting that matches the intuitive image of the user participating in the meeting.

  Here, the assignment unit determines the left-right direction as viewed from the user in the direction of viewing the at least two display devices from the user of the conference terminal device as the left-right direction of the at least two display devices. May be determined.

  With this configuration, the positions of the two display devices can be determined in the left-right direction as viewed from the user. In other words, regardless of the installation direction (east, west, north, and south) of the conference terminal device at the site where the conference terminal device is located, the determination unit determines that the left display device is at the left position as viewed from the user. The video data from the left terminal device is assigned. Also, video data from the right terminal device determined to be in the right position by the determination unit is assigned to the right display device as viewed from the user. Thereby, a natural meeting can be performed by displaying a video from the display device in a direction that matches the intuitive image of the user participating in the meeting.

  Of the corners formed by a line connecting the first terminal device, the conference terminal device, and the second terminal device in order using the position information, the first information is centered on the conference terminal device. When an angle formed counterclockwise from one terminal device to the second terminal device is smaller than 180 degrees, it is determined that the first terminal device is geographically on the right side with respect to the conference terminal device. When the angle formed around the left is greater than 180 degrees, it may be determined that the second terminal device is geographically on the right side with respect to the conference terminal device.

  In order to solve the above problems, a server device according to the present invention outputs received video data to at least two display devices arranged side by side in the left-right direction, and is a video conference between at least three points. Is a server device that controls the first, second, and third conference terminal devices for performing communication via the network, and is a geographical location of the first, second, and third conference terminal devices. Each of the second conference terminal device and the third conference terminal device using the location information acquisition unit for acquiring correct location information and the first conference terminal device as a reference. The video data from the second conference terminal device or the third conference terminal device determined to be in the right position by the determination unit that determines whether the position is left or right Said The second conference terminal device or the third conference terminal output to the display device arranged in the right direction out of at least two display devices arranged side by side in the right direction and determined to be in the left position An allocation unit that determines allocation so as to output the video data from the device to the display device arranged in the left direction among at least two display devices arranged side by side in the left-right direction; and the allocation unit A notification unit that notifies the determined assignment to the first conference terminal device.

  Here, the assignment unit determines the left-right direction as viewed from the user in the direction of viewing the at least two display devices from the user of the conference terminal device as the left-right direction of the at least two display devices. May be determined.

  In addition, the determination unit uses the position information, and among the corners formed by a line that sequentially connects the second conference terminal device, the first conference terminal device, and the third conference terminal device, When an angle made counterclockwise from the second conference terminal device to the first conference terminal device with respect to the first conference terminal device is smaller than 180 degrees, the first conference terminal device is used as a reference. When it is determined that the second conference terminal device is geographically on the right side and the angle formed around the left is greater than 180 degrees, the geographically right side location with respect to the first conference terminal device It may be characterized in that it is determined that there is the third conference terminal device.

  According to these configurations, geographical location information between the second and third conference terminal devices participating in the video conference and, for example, the first conference terminal device to be controlled is acquired, and the acquired location information is used. The left and right positions when viewed from the first conference terminal device to be controlled are determined. Based on the determination, the video from the second or third conference terminal device at the right position is output to the right display device as viewed from the user among at least two display devices arranged side by side in the left-right direction. Determine the assignment. The video from the other second or third conference terminal device is assigned so as to be output to a display device in a direction that is not on the right side when viewed from the user among at least two display devices arranged side by side in the left-right direction. decide. The determined assignment is notified to the first conference terminal device to be controlled. Thereby, the image from the second or third conference terminal device that is geographically on the right side when viewed from the first conference terminal device to be controlled is, among at least two display devices arranged side by side in the left-right direction. It is displayed from the display device in the right direction as viewed from the user. The video from the second or third conference terminal device that is geographically on the left side is displayed from the left side display device as viewed from the user among at least two display devices arranged side by side in the left-right direction. Therefore, it is possible to hold a natural meeting that matches the intuitive image of the user participating in the meeting.

  Furthermore, since location information can be centrally managed on the server device, information security can be improved and management efficiency can be improved.

  Further, each of the first, second, and third conference terminal devices does not need to manage the position information of the other conference terminal devices, and may output as specified by the server device. Therefore, it is possible to simplify the processing of the first, second, and third conference terminal devices, and to reduce the cost such as the amount of memory to be used.

  Further, based on the control by the server device according to any one of claims 4 to 6, a video conference between at least three points can be performed via another second and third conference terminal devices and a network. A first conference terminal device for performing video data, a data receiving unit for receiving video data from each of the second and third conference terminal devices, and at least two display devices arranged side by side in the horizontal direction A video output unit for outputting video data received by the data receiving unit; and a notification receiving unit for receiving a notification indicating the allocation determined by the allocating unit of the server device from the server device. The unit outputs the video data received by the data receiving unit to the corresponding display device according to the assignment received by the notification receiving unit. It may be.

  According to this configuration, for example, the first conference terminal device allocates video data from which conference terminal device to which display device based on the position information of the first, second, and third conference terminal devices. A notification indicating whether or not to output is received from the server device. Thereby, the first conference terminal device does not need to manage the position information of the first, second, and third conference terminal devices, and may output them as specified by the server device. Video from the second or third conference terminal device that is geographically on the right when viewed from the first conference terminal device is displayed on the right display device among at least two display devices arranged side by side in the left-right direction. . Video from the second or third conference terminal device that is geographically on the left side can be displayed on the left display device among at least two display devices arranged side by side in the left-right direction. Therefore, it is possible not only to perform a natural meeting that matches the intuitive image of the user participating in the meeting, but also to simplify the process of the meeting terminal device. Furthermore, it is possible to reduce the cost such as the amount of memory used.

  The present invention is not only realized as an apparatus, but also realized as an integrated circuit including processing means included in such an apparatus, or realized as a method using the processing means constituting the apparatus as a step. It can also be realized as a program for causing a computer to execute.

  According to the present invention, a video from a conference terminal device as a transmission source is displayed from a direction that is intuitively viewed from a user operating a conference terminal device in a multi-point video conference system, and a natural conference can be performed. It is possible to realize a conference terminal device and a server device that can be performed.

  That is, based on the position information of the conference terminal device in the video conference system acquired at the start of communication or the like, the left and right sides of the conference terminal device of the transmission source viewed from the conference terminal device can be determined as the geographical position. Furthermore, the video data received from the conference terminal device as the transmission source can be assigned so that the right and left in the line-of-sight direction match from the user among the plurality of output destinations of the conference terminal device. Thereby, the video output of the transmission device is performed from the direction that is intuitively viewed from the user operating the conference terminal device, and a natural conference can be performed.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a video conference system according to Embodiment 1 of the present invention.

  The video conference system shown in FIG. 1 is a system for conducting a video conference between at least three points via the network 100. A conference terminal device 201, a conference terminal device 202, and a conference terminal device 203 constituting a video conference system are connected to each other via a network 100.

  Here, the network 100 is assumed to be a public network such as the Internet or a telephone network, a corporate LAN, or the like, but any network can be used as long as it can communicate with each other.

  Each of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 is a device that performs a video conference between at least three points via the network 100. At least two display devices are connected to each of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203, and an audio input device, an audio playback device, and two video input devices belonging to each of the two display devices are connected. Has been. Here, for example, the display device is a display, and the voice input device is a microphone. The video input device is a camera, and the audio reproduction device is a speaker. This will be described below using these examples.

  The conference terminal device 201 is connected to two display devices, ie, displays 2010 and 2014, and further includes a microphone 2011, a camera 2012, and a speaker 2013 belonging to the display 2010, and a microphone 2015, a camera 2016, and a speaker 2017 belonging to the display 2014. It is connected. In addition, as shown in FIG. 1, the displays 2010 and 2014 are arranged in the horizontal direction with respect to the viewing direction of the user A who uses the conference terminal device 201. The display 2010 is arranged on the left side (hereinafter referred to as the left side of the user A) in the user A's line-of-sight direction, and the display 2014 is arranged on the right side (hereinafter referred to as the right side of the user A) in the user A's line-of-sight direction. Yes.

  The conference terminal apparatus 201 encodes audio and video input by the user A via the microphones 2011 and 2015 and the cameras 2012 and 2016 and transmits the encoded audio and video to the conference terminal apparatus 202 and the conference terminal apparatus 203. That is, the conference terminal device 201 transmits the audio and video input to the microphone 2011 and the camera 2012 and the microphone 2015 and the camera 2016 to the conference terminal device 202 and the conference terminal device 203.

  The transmission destination may be selected arbitrarily, but it is desirable that the video and audio are input / output from the direction that the user A sees intuitively. That is, when the audio and video from the conference terminal device 202 are output from the speaker 2013 and the display 2010 arranged on the left side of the user A, the audio input to the microphone 2011 and the camera 2012 arranged on the left side of the user A It is desirable to transmit the video to the conference terminal device 202.

  Similarly, when the audio and video of the conference terminal device 203 are output from the speaker 2017 and the display 2014 arranged on the right side of the user A, the audio input to the microphone 2015 and the camera 2016 arranged on the right side of the user A It is desirable to transmit the video to the conference terminal device 203.

  Note that one camera or microphone may be connected to the conference terminal device 201. In that case, the same video or audio data may be transmitted to a plurality of conference terminal devices 202 and conference terminal devices 203.

  Moreover, the display connected to the conference terminal device 201 may be physically one and the display area may be divided.

  The conference terminal device 201 determines the order of the conference terminal device 202 and the conference terminal device 203 viewed from the conference terminal device 201 based on the geographical position information of each of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203. judge. Specifically, when the conference terminal device 201 determines that the conference terminal device 203 is geographically on the right side of the conference terminal device 202, for example, the conference terminal device 201 receives audio data and video data received from the conference terminal device 203. And output to the speaker 2017 and the display 2014 on the right side of the user A. When the conference terminal device 201 determines that the conference terminal device 202 is geographically on the left side of the conference terminal device 203, the conference terminal device 201 transmits the audio data and video data received from the conference terminal device 202 to the left side of the user A. Output from the speaker 2013 and the display 2010.

  Similarly, the conference terminal device 202 is connected to two displays 2020 and 2024, and further includes a microphone 2021, a camera 2022, and a speaker 2023 belonging to the display 2020, and a microphone 2025, a camera 2026, and a speaker 2027 belonging to the display 2024. Is connected. In addition, as shown in FIG. 1, the displays 2020 and 2024 are arranged in the horizontal direction with respect to the line-of-sight direction of the user B who uses the conference terminal device 202. The display 2020 is arranged on the left side (hereinafter referred to as the left side of the user B) in the user B's line-of-sight direction, and the display 2024 is arranged on the right side (hereinafter referred to as the right side of the user B) in the user B's line-of-sight direction. Yes.

  Here, similarly to the conference terminal device 201, the conference terminal device 202 is a conference viewed from the conference terminal device 202 based on the geographical location information of each of the conference terminal device 202, the conference terminal device 203, and the conference terminal device 201. The order of the terminal device 203 and the conference terminal device 201 is determined.

  In the example of FIG. 1, the conference terminal device 202 shows a case where it is determined that the conference terminal device 201 is geographically on the right side of the conference terminal device 203, and audio data and video data received from the conference terminal device 201 are shown. Is output from the speaker 2027 and the display 2024 on the right side of the user B. In addition, the conference terminal device 202 determines that the conference terminal device 203 is geographically on the left side of the conference terminal device 201, and transmits audio data and video data received from the conference terminal device 203 to the speaker 2023 on the left side of the user B. And output from the display 2020.

  Similarly, the conference terminal device 203 is connected to two displays 2030 and 2034, and further connected to a microphone 2031 belonging to the display 2030, a camera 2032 and a speaker 2033, and a microphone 2035 belonging to the display 2034, a camera 2036 and a speaker 2037. Has been. In addition, as shown in FIG. 1, the displays 2030 and 2034 are arranged in the horizontal direction with respect to the viewing direction of the user C who uses the conference terminal device 203. A display 2030 is arranged on the left side (hereinafter referred to as the left side of the user C) in the line of sight of the user C, and a display 2034 is arranged on the right side (hereinafter referred to as the right side of the user C) in the line of sight of the user C. Yes.

  Similarly to the conference terminal device 201 and the conference terminal device 202, the conference terminal device 203 is viewed from the conference terminal device 203 based on the geographical position information of each of the conference terminal device 203, the conference terminal device 201, and the conference terminal device 202. The order of the conference terminal device 201 and the conference terminal device 202 is determined.

  In the example of FIG. 1, the conference terminal device 203 outputs audio data and video data received from the conference terminal device 202 from the speaker 2037 and the display 2034 on the right side of the user C. In addition, the conference terminal device 203 outputs audio data and video data received from the conference terminal device 201 from the speaker 2033 and the display 2030 on the left side of the user C.

  FIG. 2 is a diagram showing a configuration of the conference terminal apparatus 201 according to Embodiment 1 of the present invention.

  In FIG. 2, the conference terminal device 201 includes a data reception unit 300, an audio unit 301, a video unit 302, a position information acquisition unit 303, an order determination unit 304, and an allocation unit 305. The audio unit 301 includes an audio output unit 3012 connected to the left speaker 2013 of the user A, and an audio output unit 3013 connected to the right speaker 2017 of the user A. The video unit 302 includes a video output unit 3022 connected to the display 2010 on the left side of the user A, and a video output unit 3023 connected to the display 2014 on the right side of the user A.

  The data receiving unit 300 receives audio data and video data from the conference terminal device 202 and the conference terminal device 203 via the network 100. Specifically, the data receiving unit 300 outputs audio data and video data from the conference terminal device 202 and the conference terminal device 203 to the audio unit 301 and the video unit 302.

  Specifically, the data receiving unit 300 sends the received audio data from the conference terminal device 202 and the conference terminal device 203 to the audio output unit 3012 and the audio output unit 3013, respectively, based on the allocation determined by the allocation unit 305. Output. Similarly, the data receiving unit 300 outputs the received video data from the conference terminal device 202 and the conference terminal device 203 to the video output unit 3022 and the video output unit 3023, respectively, based on the allocation determined by the allocation unit 305. .

  The order determination unit 304 uses the geographical position information acquired by the position information acquisition unit 303 to determine the order of the other conference terminal apparatuses 202 and 203 viewed from the self-conference terminal apparatus 201.

  The allocation unit 305 determines the allocation based on the order of the conference terminal device 202 and the conference terminal device 203 determined by the order determination unit 304.

  The position information acquisition unit 303 acquires the geographical position information of the conference terminal device 202 and the conference terminal device 203 that are the transmission sources of the audio data and the video data received by the data reception unit 300 via the data reception unit 300. .

  The position information may be stored in advance in a database for each conference terminal device, for example. In that case, the conference terminal device 201 acquires position information from the database using information such as addresses or telephone numbers of the conference terminal device 202 and the conference terminal device 203 as counterparts when starting communication.

  In addition, since the conference terminal device 201 generally performs call control using a call control protocol when performing a conference via the network 100, the conference terminal device 201 acquires location information by exchanging as additional information at that time. May be.

  Examples of call control protocols include SIP (Session Initiation Protocol) and H.264. H.323 is known. In the following, an example using SIP as a call control protocol is shown. Needless to say, a protocol having an equivalent function can also be implemented.

  The conference terminal devices 202 and 203 have the same configuration as the conference terminal device 202, but the description thereof is omitted.

  FIG. 3 is a diagram for explaining transmission / reception of a call control message and media according to Embodiment 1 of the present invention.

  In FIG. 3, first, when the user A of the conference terminal apparatus 201 performs a calling operation, the conference terminal apparatus 201 transmits an “INVITE” message to the conference terminal apparatus 202 (S102).

  Next, when the user B of the conference terminal device 202 performs a response operation, the conference terminal device 202 transmits a “200 OK” message (S104).

  Next, when receiving the “200 OK” message, the conference terminal apparatus 201 transmits an “ACK” message for confirmation (S106). By transmitting and receiving these call control messages, a call control session is established between the conference terminal device 201 and the conference terminal device 202.

  Further, the conference terminal apparatus 201 similarly transmits an “INVITE” message to the conference terminal apparatus 203 (S108), receives a “200 OK” message (S110), and transmits an “ACK” message (S112). Thus, a call control session is established.

  As described above, a call control session is established among the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203, and then video data and audio data are exchanged.

  The conference terminal device 201 may establish a call control session with the conference terminal device 203 without waiting for the establishment of a call control session with the conference terminal device 202. That is, the conference terminal device 201 may continuously send an “INVITE” message to the conference terminal device 202 and the conference terminal device 203 to establish a call control session in parallel.

  In the “INVITE” message and “200 OK” message, information such as the codec type, the number of media, the port number, and the IP address that can be transmitted and received by the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 is set. Is done. Using these pieces of information, the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 can transmit and receive media as shown in FIG.

  Further, position information can be added to the “INVITE” message and the “200 OK” message. Thereby, it is possible to mutually notify the positional information of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203.

  In the example illustrated in FIG. 3, a Coordinates header is added to the “INVITE” message and the “200 OK” message, and information such as longitude and latitude of the conference terminal device that is the message transmission source is notified therein.

  Further, the position information to be added and notified to the “INVITE” message and the “200 OK” message is not limited to latitude and longitude, but may be a city name, an address, a postal code, or the like. When using such information, in order to obtain the order of other conference terminal devices viewed from the own conference terminal device, information indicating the correspondence between the city name, address and postal code and latitude and longitude is managed separately. May be. Moreover, when using such information, you may hold | maintain the geographical order between those information previously.

  FIG. 4 is a diagram for explaining a method in which the order determination unit 304 according to Embodiment 1 of the present invention determines the order of other conference terminal apparatuses viewed from the self-conference terminal apparatus.

  The order determination unit 304 uses the position information acquired by the position information acquisition unit 303 to determine the order of other conference terminal devices viewed from the self-conference terminal device. Details will be described below.

  The position information acquisition unit 303 acquires, for example, latitude and longitude information as position information from the conference terminal device 202 and the conference terminal device 203 via the data reception unit 300. Here, the positional relationship among the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 is as shown in FIG.

  The order determination unit 304 in the conference terminal device 201 uses the location information acquired by the location information acquisition unit 303 to determine the order of the conference terminal device 202 and the conference terminal device 203 as viewed from the conference terminal device 201 as follows. .

  First, the order determination unit 304 selects any one of the conference terminal device 202 and the conference terminal device 203 (here, the conference terminal device 203 is selected).

  Next, the order determination unit 304 from the conference terminal device 203 to the conference terminal device 202 out of the corner formed by a line connecting the conference terminal device 203, the conference terminal device 201, and the conference terminal device 202 in order (vertically downward) (Ii) The angle α formed around the left is viewed (FIG. 4B).

  When the angle α is smaller than 180 degrees, the order determination unit 304 determines that the selected conference terminal device (here, the conference terminal device 203) is geographically on the right side when viewed from the conference terminal device 201, and the other conference terminal. It is determined that the device (here, the conference terminal device 202) is geographically on the left side.

  When the angle α is larger than 180 degrees, the order determination unit 304 determines that the other (here, the conference terminal device 202) is geographically on the right side when viewed from the conference terminal device 201, and the selected conference terminal device ( Here, it is determined that the conference terminal device 203) is geographically on the left side.

  When the angle α is equal to 180 degrees, either the conference terminal device 202 or the conference terminal device 203 may be determined to be geographically right.

  Similarly, the order of the conference terminal device 201 and the conference terminal device 203 viewed from the conference terminal device 202 can also be determined. That is, the order determination unit of the conference terminal device 202 selects the conference terminal device 203 as shown in FIG. 4C, and creates a line connecting the conference terminal device 203, the conference terminal device 202, and the conference terminal device 201 in order. Of the corners, an angle β formed counterclockwise when viewed from the sky from the conference terminal device 203 to the conference terminal device 201 (vertically downward) is calculated.

  Since the angle β is larger than 180 degrees, it is determined that the conference terminal device 201 is geographically on the right side and the conference terminal device 203 is geographically on the left side as viewed from the conference terminal device 202 (reference).

  Similarly, the order of the conference terminal device 201 and the conference terminal device 202 viewed from the conference terminal device 203 can also be determined. That is, the order determination unit in the conference terminal device 203 selects the conference terminal device 201 as shown in FIG. 4D and creates a line connecting the conference terminal device 201, the conference terminal device 203, and the conference terminal device 202 in order. Of the corners, an angle γ is calculated which is formed counterclockwise when viewed from above (vertically downward) from the conference terminal device 201 to the conference terminal device 202.

  Since the angle γ is larger than 180 degrees, it is determined that the conference terminal device 202 is geographically on the right side and the conference terminal device 201 is geographically on the left side as viewed from the conference terminal device 203 (reference).

  As described above, the order determination unit 304 uses the geographical position information acquired by the position information acquisition unit 303 to determine the order of the other conference terminal devices 202 and 203 viewed from the self-conference terminal device 201. To do.

  In the above description, the determination is made based on whether or not the angle formed in the counterclockwise direction is greater than 180 degrees, but the determination may be performed in the clockwise direction. It is clear that an equivalent determination can be made even when the clockwise angle is measured, and a description thereof will be omitted.

  FIG. 5 is a diagram for explaining a determination method for determining the order of three points when longitude (λ) and latitude (φ) are used as geographical position information.

  The determination method described above is a determination method for determining the order of other conference terminal devices viewed from the conference terminal device in the orthogonal coordinate system (XYZ), that is, the order of three points in the orthogonal coordinate system (XYZ). . It will be described below that the order of the three points can be similarly determined even when longitude (λ) and latitude (φ) are used as geographical position information.

  First, if the latitude and longitude of a certain point on the earth are known, it can be converted into coordinates in an orthogonal coordinate system (XYZ) centered on the center of gravity of the earth.

  Accordingly, when the longitude λ, the latitude φ, the equator radius a, and the polar radius b are set, the coordinates (x, y, z) in the orthogonal coordinate system are (Equation 1), (Equation 2), and (Equation 3) as follows. ).

  However, in (Equation 1), (Equation 2), and (Equation 3), they are (Equation 4) and (Equation 5).

  As described above, if the coordinates (x, y, z) in the orthogonal coordinate system shown in (Expression 1) to (Expression 5) are used, the order of three points, that is, the order of points A, B, and C is determined. can do. Specifically, among the angles formed by the lines connecting point C, point A, and point B in order shown in FIG. 5, the angle formed from the point C to the point B in the counterclockwise direction when viewed from the sky (vertically downward) is calculated. Thus, the order of the points A, B, and C can be determined. This angle calculation is a problem of angle calculation in a three-dimensional space, and there is no particular difficulty.

  In addition, without converting to a three-dimensional orthogonal coordinate system (XYZ), it may be calculated approximately using a two-dimensional orthogonal coordinate system in which longitude and latitude are X and Y, respectively. Since the proportion of the population residing in an extremely high latitude area is small, even in an approximate calculation, in many cases, the left and right determination order described above does not change.

  The above determination is not limited to a conference between three points using the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203. Further, the conference terminal device 204 can be increased, and the present invention can be applied to a conference between four points using the conference terminal device 201, the conference terminal device 202, the conference terminal device 203, and the conference terminal device 204.

  6 and 7 are diagrams for explaining a method of determining the order of conference terminal apparatuses according to Embodiment 1 of the present invention. Hereinafter, an order determination method in the case where a conference is held between four points using the four conference terminal devices 201, 202, 203, and 204 will be described with reference to FIGS.

  FIG. 6 shows a case where the conference terminal device 204 is outside the triangle formed by the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203. FIG. 7 shows a case where the conference terminal device 204 is inside a triangle formed by the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203.

  In FIG. 6, first, the order is determined based on the conference terminal device 201. Any two are selected from the conference terminal device 202, the conference terminal device 203, and the conference terminal device 204, excluding the conference terminal device 201. For example, the conference terminal device 203 and the conference terminal device 204 are selected.

  Next, by applying the above procedure using the three conference terminal devices 201 to 203 to the selected conference terminal device (the conference terminal device 203 and the conference terminal device 204), the conference terminal device 201, the conference terminal The order of the device 203 and the conference terminal device 204 is determined.

  That is, among the corners formed by the lines connecting the conference terminal device 204, the conference terminal device 201, and the conference terminal device 203 in order, the conference terminal device 204 to the conference terminal device 203 are viewed counterclockwise when viewed from the sky (vertically downward). The formed angle x is calculated.

  In the case shown in FIG. 6, since the angle x is smaller than 180 degrees, it is determined that the conference terminal device 203 is geographically on the left side and the conference terminal device 204 is geographically on the right side based on the conference terminal device 201.

  Next, by selecting any two conference terminals in another combination and performing the same procedure, the two orders can be determined. For example, when the conference terminal device 202 and the conference terminal device 203 are selected, the conference terminal device 203 to the conference terminal device out of the corners formed by lines connecting the conference terminal device 202, the conference terminal device 201, and the conference terminal device 203 in order. An angle y formed in the counterclockwise direction when viewed from the sky (vertically downward) to 202 is calculated. Since the angle y is smaller than 180 degrees, it is determined that the conference terminal device 202 is geographically on the left side and the conference terminal device 203 is geographically on the right side with respect to the conference terminal device 201.

  As described above, when any two conference terminal devices are sequentially selected in the above combination, the entire order can be determined by two processes. Therefore, it is determined that the conference terminal device 204 is the right side, the conference terminal device 203 is the center, and the conference terminal device 202 is the left side based on the conference terminal device 201.

  In FIG. 6, a case is considered in which, for example, the conference terminal device 202 and the conference terminal device 204 are first selected and then the conference terminal device 202 and the conference terminal device 203 are selected with the conference terminal device 201 as a reference. In this case, in the first determination, it is determined that the conference terminal device 202 is on the left side and the conference terminal device 202 is on the right side based on the conference terminal device 201. In the next determination, it is determined that the conference terminal device 202 is the left side and the conference terminal device 203 is the right side with respect to the conference terminal device 201. However, when any two conference terminal devices are selected twice in such a combination, it can be seen that the conference terminal device 202 is on the left side, but the order of the conference terminal device 203 and the conference terminal device 204 is unknown. Absent. In that case, the conference terminal device 203 and the conference terminal device 204 may be further selected and the same procedure may be performed. Thereby, it is possible to determine that the conference terminal device 204 is on the right side, the conference terminal device 203 is in the center, and the conference terminal device 202 is on the left side with respect to the conference terminal device 201.

  For the conference terminal device 202, the conference terminal device 203, and the conference terminal device 204, the order is determined by calculating in the same manner as a reference.

  In the case illustrated in FIG. 7, for the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203, the order of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 is calculated in the same manner based on each. It is possible to determine.

  However, as shown in FIG. 7, there is a difference with respect to the conference terminal device 201, the conference terminal device 202, and the conference terminal device 204 located at the center of the conference terminal device 203, and will be described below.

  In FIG. 7, first, any two are selected from other conference terminal devices with the conference terminal device 204 as a reference. Here, the conference terminal device 203 and the conference terminal device 202 are selected, and the same procedure as described above is performed. That is, among the corners formed by the lines connecting the conference terminal device 203, the conference terminal device 204, and the conference terminal device 202 in order, the conference terminal device 203 to the conference terminal device 202 are viewed counterclockwise as viewed from above (vertically downward). The formed angle u is calculated.

  In the case shown in FIG. 7, since the angle u is smaller than 180 degrees, it is determined that the conference terminal device 202 is the left side and the conference terminal device 203 is the right side with respect to the conference terminal device 204.

  Further, when the conference terminal device 203 and the conference terminal device 201 are selected and the same procedure is performed, the angle v is larger than 180 degrees, so the conference terminal device 201 is on the right side, the conference terminal, with the conference terminal device 204 as a reference. The device 203 is determined as the left side.

  Further, when the conference terminal device 202 and the conference terminal device 201 are selected and the same procedure is performed, the angle w is smaller than 180 degrees, so the conference terminal device 202 is on the right side, the conference terminal, with the conference terminal device 204 as a reference. The device 201 is determined as the left side.

  However, it is not possible to establish all the orders so as to satisfy the above determination result. That is, each of the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 cannot be assigned to, for example, the right side, the center, and the left side so as to satisfy the determination result.

  In such a case, that is, when the conference terminal device 204 is located at the center among the conference terminal devices 201 to 204 at the four points, the order of the other conference terminal devices 201 to 203 is arbitrarily determined in the conference terminal device 204 located at the center. May be specified. Further, the conference terminal device 204 located in the center may be provided with rules such as the right side, the center, and the left side in order from the conference terminal device having the most geographical position.

  As described above, the order of the four conference terminal devices 201 to 204 is determined.

  Next, in the conference terminal device 201, the assignment determined by the assignment unit 305 from the order information that is the result of the determination by the order determination unit 304 will be described.

  FIG. 8 is a diagram showing a case where two displays and two speakers are connected to conference terminal apparatus 201 according to Embodiment 1 of the present invention.

  A method in which the assigning unit 305 determines the output destination of the received audio data and video data based on the order information of the other conference terminal devices 202 and 203 acquired by the order determining unit 304 will be described below.

  Two speakers 2013 and 2017 and two displays 2010 and 2014 are connected to the conference terminal device 201. As shown in FIG. 8, the two speakers 2013 and 2017 and the two displays 2010 and 2014 are arranged in a horizontal direction with respect to the viewing direction of the user A who uses the conference terminal device 201.

  The allocation unit 305 determines the allocation so that the horizontal arrangement order (left and right) matches the order of the other conference terminal devices 202 and 203 viewed from the self-conference terminal device 201 determined above. That is, the allocation unit 305 determines the allocation so that the audio data and the video data received by the data receiving unit 300 are output from the audio output units 3012 and 3013 and the video output units 3022 and 3023.

  For example, the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 are positioned as shown in FIG. 1, that is, the conference terminal device 202 is on the left side and the conference terminal device 203 is on the right side as viewed from the conference terminal device 201. Suppose that In this case, the allocation unit 305 determines the allocation so that the audio data and the video data from the conference terminal device 203 are output to the speaker 2017 and the display 2014 on the right side of the user A. That is, the assigning unit 305 determines to assign the audio data from the conference terminal device 203 to the audio output unit 3013 that outputs the audio data from the audio unit 301 to the speaker 2017. Similarly, the allocation unit 305 determines to allocate the video data from the conference terminal device 203 to the video output unit 3023 that outputs the video data from the video unit 302 to the display 2014.

  The data receiving unit 300 outputs the received audio data and video data from the conference terminal device 203 to the audio output unit 3013 and the video output unit 3023, respectively, based on the allocation determined by the allocation unit 305.

  Similarly, the assigning unit 305 outputs the audio data and the video data from the conference terminal device 202 on the left side when viewed from the conference terminal device 201 to the speaker 2013 and the display 2010 on the left side of the user A. To assign to the unit 3012 and the video output unit 3022. The data receiving unit 300 outputs the received audio data and video data from the conference terminal device 202 to the audio output unit 3012 and the video output unit 3022 based on the allocation determined by the allocation unit 305, respectively.

  FIG. 9 is a diagram showing a case where three displays and three speakers are connected to conference terminal apparatus 201 according to Embodiment 1 of the present invention.

  The conference terminal device 201 is connected with three speakers 2013, 2017, and 2019 and three displays 2010, 2014, and 2018. As shown in FIG. 9, the three speakers 2013, 2017, and 2019 and the three displays 2010, 2014, and 2018 are arranged in a horizontal direction with respect to the line of sight of the user A who uses the conference terminal device 201. .

  Here, it is assumed that the conference is at four points having a positional relationship as shown in FIG. 6, and the conference terminal device 201 acquires audio data and video data from the other three conference terminal devices 202, 203 and 204. . In that case, the allocation unit 305 may determine the allocation so as to output to a display or a speaker at a position corresponding to the three conference terminal devices, as described above.

  Note that the conference terminal device 201 has audio from only the other two conference terminal devices 202 and 203 even if the three speakers 2013, 2017, and 2019 are connected to the three displays 2010, 2014, and 2018. Data and video data may be received. In that case, the assigning unit 305 assigns audio and video data from the conference terminal device 202 on the left side and the conference terminal device 203 on the right side to either (left side, right side), (left side, center), or (center, right side). The assignment may be determined so as to output to the speaker and the display in that direction.

  Furthermore, the same idea as described above may be applied to the case where there are a large number of audio output units and video output units included in the audio unit 301 and the video unit 302.

  Also, when the number of audio data and video data received by the data receiving unit 300 is smaller than the number of audio output units in the audio unit 301 or the number of video output units in the video unit 302, the audio data or video data Thus, the same idea as described above may be applied. At this time, audio data and video data received from one conference terminal device among other conference terminal devices are output to a plurality of audio and video output units included in the audio unit 301 and the video unit 302. Another method may be applied. For example, audio data synthesized from audio data received from a plurality of conference terminal devices may be output to an audio output unit having no corresponding audio data.

  Hereinafter, specific examples will be described.

  For example, it is assumed that three speakers 2013, 2017, and 2019 are connected to the conference terminal device 201, and the data receiving unit 300 receives audio data from the two conference terminal devices 202 and 203, respectively. In that case, the allocation unit 305 determines the allocation so that each of the audio output units output to the right speaker 2019 and the left speaker 2013 of the user A outputs audio data from the conference terminal device 202 and the conference terminal device 203. do it.

  Further, for example, the assignment may be determined so as to synthesize and output the audio data from the conference terminal devices 202 and 203 to the central speaker 2017 having no corresponding conference terminal device and no audio data.

  When the number of audio data and video data received by the data receiving unit 300 is larger than the number of audio output units in the audio unit 301 or the number of video output units in the video unit 302, the audio data or video data The same idea as described above may be applied. That is, video data or audio data synthesized from video data or audio data received from a plurality of conference terminal devices may be output to an audio output unit or video output unit without corresponding audio data or video data.

  A specific example will be described. For example, two speakers 2013 and 2017 and two displays 2010 and 2017 are connected to the conference terminal device 201, and data reception units receive audio data and video data from the three conference terminal devices (202, 203, and 204), respectively. Assume that 300 receives. In that case, the allocation unit 305 outputs the data of the right and left conference terminal devices viewed from the conference terminal device 201 from the speakers 2013 and 2017 and the displays 2010 and 2017 on the right and left sides of the user A, respectively. Determine the assignment. Here, the data of the central conference terminal device as viewed from the conference terminal device 201 is synthesized and output to the speakers 2013 and 2017 and the displays 2010 and 2017 on either the right side or the left side of the user A. It is good.

  FIG. 10 is a diagram illustrating a case where the display and the speaker connected to the conference terminal device 201 according to Embodiment 1 of the present invention are arranged in a shifted manner.

  FIG. 10 illustrates an example in which three speakers 2013, 2017, and 2019 and three displays 2010, 2014, and 2018 are connected to the conference terminal device 201 in an arrangement shifted from the user's line-of-sight direction. .

  Even in this case, the order of the speakers 2013, 2017, and 2019 and the displays 2010, 2014, and 2018 is determined as the left side, the center, and the right side from the user A when viewed in one dimension in the horizontal direction with respect to the user's line-of-sight direction. It is possible. The assignment unit 305 may determine assignment of the output destinations of the video data and audio data of the conference terminal device 202 and the conference terminal device 203 based on the determination order.

  As described above, according to the conference terminal device of the first embodiment, from the own conference terminal device based on the location information of the other conference terminal device and the location information of the own conference terminal device acquired at the start of communication or the like. It is possible to determine the order of other conference terminal devices that have been viewed. Furthermore, it is possible to determine that audio data and video data received from other conference terminal devices are assigned to a display in a direction that matches the determined order from among a plurality of displays that are output destinations. Thereby, the audio and video of other conference terminal devices are output from the direction intuitively viewed from the user operating the conference terminal device, and a natural conference can be performed.

  In the above description, there are a plurality of displays arranged in the left-right direction between the user and the conference terminal device in order to simplify the description of the direction that is intuitively viewed from the user operating the conference terminal device. On the assumption of the position, the left and right directions with respect to the user's line-of-sight direction are defined as the left side and the right side. However, the present invention is not limited to this, and the positions of the plurality of displays, the user, and the conference terminal device may be arbitrarily determined.

  For example, regardless of the positional relationship between the conference terminal device, the user of the conference terminal device, and the plurality of displays, the assigning unit 305 moves in the left-right direction of the plurality of displays in the direction from the plurality of displays to the other conference terminal devices. May be determined.

  Further, for example, regardless of the positional relationship between the conference terminal device, the user of the conference terminal device, and the plurality of displays, the assigning unit 305 is in the left-right direction with respect to the line-of-sight direction in which the user of the conference terminal device looks at the plurality of displays. Allocation may be determined by determining the left-right direction of a plurality of displays. That is, the assigning unit 305 determines the assignment by determining the left and right directions of the plurality of displays so as to match the order of other conference terminal devices viewed from the self-conference terminal device determined by the order determining unit 304, that is, the left and right direction. . At that time, the assigning unit 305 does not depend on the positional relationship among the conference terminal device, the user of the conference terminal device, and the plurality of displays, and the horizontal direction of the plurality of displays in the left-right direction with respect to the line-of-sight direction of the user of the conference terminal device. May be determined. Specifically, at the base where the conference terminal device is placed, the display device on the left side as viewed from the user is positioned on the left side by the determination unit regardless of the installation direction (east, west, north, and south) of the conference terminal device. Video data and audio data from the terminal device on the left determined to be assigned. Also, video data and audio data from the right terminal device determined to be in the right position by the determination unit are assigned to the right display device as viewed from the user.

  As described above, the conference terminal device, the conference terminal device user, and other conference terminal devices from the direction that is intuitively viewed from the user operating the conference terminal device, regardless of the positional relationship between the plurality of displays. Audio and video are output and a natural conference can be performed.

  Note that the positional relationship between the conference terminal device, the user of the conference terminal device, and the display may not be determined by the allocation unit 305, but the user may set the positional relationship in the allocation unit 305 in advance.

(Embodiment 2)
In the first embodiment, the order of other conference terminal devices viewed from the own conference terminal device is determined from the position information of the own conference terminal device acquired by the conference terminal device, and the allocation is performed based on the determination result. It was decided. In the second embodiment, a case will be described in which these functions are provided in the server instead of the conference terminal device.

  FIG. 11 is a diagram showing configurations of the server device 500 and the conference terminal device 501 according to Embodiment 2 of the present invention.

  In the second embodiment, the server device 500 includes a position information acquisition unit 303, an order determination unit 304, and an allocation unit 305, and further includes a notification unit 306. The server device 500 may have a conference server function and a call control server function.

  The difference between the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503, and the conference terminal device 201, the conference terminal device 202, and the conference terminal device 203 in the first embodiment is that a position information acquisition unit 303, an order determination unit 304 In addition, the receiving unit 307 is added without including the assigning unit 305.

  The position information acquisition unit 303 acquires the geographical position information of the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503, as in the first embodiment.

  The position information may be stored in advance in a database for each conference terminal device, for example. In this case, the server device 500 uses the information such as the address or telephone number of the conference terminal device 202 and the conference terminal device 203 that are counterparts when the conference terminal device 501 starts communication as a key, and uses the location information from the database. get.

  Further, the server device 500 may receive position information from the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503. In particular, when the server device 500 has the conference server function and the call control server function, the location information is received from the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503 as additional information of the call control message. May be.

  The order determination unit 304 determines the order of the conference terminal device 502 and the conference terminal device 503 viewed from the conference terminal device 501, for example, based on the location information acquired by the location information acquisition unit 303, as in the first embodiment. .

  Based on the order determined by the order determination unit 304, the allocation unit 305 determines, for example, the allocation of the audio unit 301 and the video unit 302 to be used for the audio output unit and the video output unit in the conference terminal device 501.

  At this time, it is desirable that the server device 500 acquires the number of audio output units and video output units included in the conference terminal device 501 in advance from the conference terminal device 501 or during call control.

  If it has been acquired, it is possible to determine the assignment to the audio output unit and the video output unit to be used in the same processing as in the first embodiment. If not acquired, the server apparatus 500 determines an assignment for the number of predetermined audio output units and video output units as a default. The server device 500 notifies the conference terminal device 501 of the assignment thus determined via the notification unit 306 described later. As a response to the notification, an error (including the number of audio output units and the number of video output units) may be received from the conference terminal device 501 and allocation may be determined again.

  The notification unit 306 notifies the conference terminal apparatus 501 of the allocation determined by the allocation unit 305. Here, as the notified contents, for example, audio data and video data from the conference terminal device 502 are respectively output to the audio output unit 3012 that outputs to the left speaker 2013 and the video output unit 3022 that outputs to the left display 2010. This indicates that it is assigned. In addition, the audio data and the video data from the conference terminal device 503 are assigned to the audio output unit 3013 that outputs to the right speaker 2017 and the video output unit 3023 that outputs to the right display 2014, respectively.

  Here, when the server apparatus 500 has a conference server function and a call control server function, it is desirable that the allocation determined by the allocation unit 305 is included in the call control message and notified to the conference terminal apparatus 501.

  The conference terminal device 501 receives the notification at the receiving unit 307. The conference terminal device 501 outputs the audio data and video data received by the data receiving unit 300 using the specified audio output unit 3012 and video output unit 3022 according to the content specified in the notification.

  If the number of audio and video output units does not match as described above, the conference terminal device 501 may return an error after setting the number of audio and video output units that it has when responding to the notification. Of course, some audio and video data need not be output. Also, some audio and video data may be synthesized and output.

  FIG. 12 is a diagram for explaining transmission / reception of a server call control message and media according to Embodiment 2 of the present invention.

  FIG. 12 illustrates an example of communication between the server device 500 and the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503. In this example, the server device 500 has a conference server function, and uses SIP as a call control protocol.

  In FIG. 12, first, the conference terminal device 502 and the conference terminal device 503 establish a call control session with the server device 500. That is, the conference terminal device 502 transmits an “INVITE” message to the server device 500 (S202), receives a “200 OK” message (S204), and receives an “ACK” message (S206). Thus, a call control session with the server device 500 is established. Similarly, the conference terminal apparatus 503 transmits an “INVITE” message to the server apparatus 500 (S208), receives a “200 OK” message (S210), and receives an “ACK” message (S212). Thus, a call control session with the server device 500 is established.

  Next, the conference terminal device 501 establishes a call control session with the server device 500. Further, as will be described below, the server device 500 determines the assignment by determining the order of other conference terminal devices viewed from each conference terminal device.

  First, the conference terminal device 501 sends an “INVITE” message to the server device 500 based on an operation from the user A (S214). At this time, the location information of the self-conference terminal device 501 is notified to the server device 500 using the Coordinates header included in the “INVITE” message. Further, the device header notifies the conference terminal device 501 that the number of audio output units and the number of video output units are two. Note that the number of audio output units and the number of video output units may be notified using a media line (m = line) of SDP (Session Description Protocol) set in the body of the “INVITE” message. An XML format body may be added to the “INVITE” message and notified by using it.

  Next, the server apparatus 500 acquires the position information of the conference terminal apparatus 501 by the position information acquisition unit 303. Based on the positional information of the conference terminal device 501, conference terminal device 502, and conference terminal device 503 acquired by the location information acquisition unit 303, the order determination unit 304 determines whether the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503 Determine the order. The allocation unit 305 determines allocation to the video output unit or the audio output unit of the conference terminal device 501 based on the order determined by the order determination unit 304.

  Next, the server device 500 receives the determined allocation, for example, the video data from the conference terminal device 502 on the left display 2010 of the user A, and the video data from the conference terminal device 503 on the right display 2014 of the user A. A “200 OK” message including an indication of allocation is returned (S216).

  Here, a plurality of SDP media lines are set in the “200 OK” message, and voice data from the conference terminal device 502 (IP address is 192.168.1.2) is sent to the left side (order: 2) The speaker 2013 designates that audio data from the conference terminal device 503 (IP address 192.168.1.3) is to be output to the speaker 2017 on the right side (order: 1) of the user A. Is set. Further, in the SDP media line, the same designation is made for the video data.

  Next, the conference terminal apparatus 501 sends an “ACK” message indicating that the “200 OK” message has been received to the server apparatus 500 (S218).

  Through the above message exchange, an SIP session is established among the server device 500, the conference terminal device 501, the conference terminal device 502, and the conference terminal device 503.

  Next, the conference terminal device 501 receives video data and audio data from the other conference terminal device 502 and the conference terminal device 503.

  The conference terminal device 501 refers to the transmission source of the received data, and outputs the audio data and video data from the conference terminal device 502 to the speaker 2013 and the display 2010 on the left side of the user A. Similarly, the conference terminal device 501 outputs the audio data and video data from the conference terminal device 503 to the speaker 2017 and the display 2014 on the right side of the user A.

  As described above, according to the server device 500 and the conference terminal devices 501 to 503 of the second embodiment, based on the location information of the other conference terminal devices and the location information of the own conference terminal device acquired at the start of communication or the like. In addition, the order of other conference terminal devices viewed from the self-conference terminal device can be determined. Furthermore, it is possible to determine that audio data and video data received from other conference terminal devices are assigned to a display in a direction that matches the determined order from among a plurality of displays that are output destinations.

  As a result, the audio and video of other conference terminal devices are output from the direction that is intuitively viewed from the user operating the conference terminal device, and a natural conference can be performed.

  Further, each conference terminal device 501, 502, and 503 outputs only as specified to the server device 500 without managing the position information of the other conference terminal devices. And 503 can be simplified. Furthermore, it is possible to reduce the cost such as the amount of memory used.

  Further, since the location information can be centrally managed on the server device 500, information security can be improved and management efficiency can be improved.

  As described above, the conference terminal device and the server device of the present invention have been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. .

  The present invention can be used for a conference terminal device and a server device, and in particular, can be used for a conference terminal device and a server device used for a telepresence system and a multi-screen video conference system.

It is a figure which shows the structure of the video conference system which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the conference terminal device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating transmission / reception of the call control message and medium which concern on Embodiment 1 of this invention. It is a figure for demonstrating the method in which the order determination part which concerns on Embodiment 1 of this invention determines the order of the other conference terminal device seen from the self-conference terminal device. It is a figure for demonstrating the determination method which determines the order of 3 points | pieces, when longitude ((lambda)) and latitude ((phi)) are used as geographical location information. It is a figure for demonstrating the method to determine the order of the conference terminal device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the method to determine the order of the conference terminal device which concerns on Embodiment 1 of this invention. It is a figure which shows the case where two displays and two speakers are connected to the conference terminal device which concerns on Embodiment 1 of this invention. It is a figure which shows the case where three displays and three speakers are connected to the conference terminal device which concerns on Embodiment 1 of this invention. It is a figure which shows the case where the display and speaker which are connected to the conference terminal device which concerns on Embodiment 1 of this invention have shifted and are arrange | positioned. It is a figure which shows the structure of the server apparatus and conference terminal device which concern on Embodiment 2 of this invention. It is a figure for demonstrating transmission / reception of the server call control message and medium which concern on Embodiment 2 of this invention. It is a figure for demonstrating the screen display when one display is connected to the conference terminal device in the conventional video conference system. It is a figure for demonstrating the screen display in case three display apparatuses (display) are connected to the conference terminal device in the conventional video conference system. It is a figure for demonstrating the subject in the conventional video conference system.

Explanation of symbols

100 Network 201, 202, 203, 204, 501, 502, 503, 801, 802, 803, 901, 902, 903 Conference terminal device 300 Data reception unit 301 Audio unit 302 Video unit 303 Location information acquisition unit 304 Order determination unit 305 Allocation unit 306 Notification unit 307 Reception unit 500 Server apparatus 2010, 2014, 2020, 2024, 2030, 2034 Display 2011, 2015, 2021, 2025, 2031, 2035 Microphone 2012, 2016, 2022, 2026, 2032, 2036 Camera 2013, 2017 , 2023, 2027, 2033, 2037 Speaker 3012, 3013 Audio output unit 3022, 3023 Video output unit
8010, 9010, 9011, 9012 display device

Claims (11)

A conference terminal device for conducting a video conference between at least three points via a network with the first and second terminal devices,
A data receiving unit for receiving video data from each of the first and second terminal devices;
A video output unit that outputs video data received by the data receiving unit to at least two display devices arranged side by side in the left-right direction;
A location information acquisition unit that acquires geographical location information of the conference terminal device, the first terminal device, and the second terminal device;
A determination unit that determines whether each of the first and second terminal devices is geographically left or right with respect to the conference terminal device using the position information;
Video data from the first terminal device or the second terminal device determined by the determination unit to be in the right position is displayed on the right of at least two display devices arranged side by side in the left-right direction. Video data from the first terminal device or the second terminal device that is output to the display device arranged in the direction and determined to be in the left position is arranged at least two arranged in the left-right direction. An allocation unit that determines allocation so as to output to a display device arranged in the left direction among the two display devices;
The conference terminal device, wherein the video output unit outputs the video data received by the data receiving unit to the corresponding display device according to the allocation determined by the allocation unit. The assigning unit determines the assignment by determining a left-right direction as viewed from the user in a direction of viewing the at least two display devices from a user of the conference terminal device as a left-right direction of the at least two display devices. The conference terminal device according to claim 1. The determination unit
Of the corners formed by a line connecting the first terminal device, the conference terminal device, and the second terminal device in order using the position information, the first terminal device centered on the conference terminal device. When an angle formed counterclockwise from the terminal device to the second terminal device is smaller than 180 degrees, it is determined that the first terminal device is geographically on the right side with respect to the conference terminal device, and 3. When the angle formed around the left is greater than 180 degrees, it is determined that the second terminal device is geographically on the right side with reference to the conference terminal device. Conference terminal device. First, second, and third for outputting received video data to at least two display devices arranged side by side in the left-right direction and performing a video conference between at least three points via a network. A server device for controlling the conference terminal device via a network,
A position information acquisition unit for acquiring geographical position information of the first, second and third conference terminal devices;
Using the location information, it is determined whether each of the second conference terminal device and the third conference terminal device is geographically left or right with respect to the first conference terminal device. A determination unit;
Among the at least two display devices arranged side by side in the left-right direction, the video data from the second conference terminal device or the third conference terminal device determined by the determination unit to be in the right position Output to the display device arranged in the right direction and arrange the video data from the second conference terminal device or the third conference terminal device determined to be in the left position side by side in the left-right direction An assigning unit that determines assignment so as to output to the display device arranged in the left direction among at least two display devices;
A server unit comprising: a notification unit that notifies the first conference terminal device of the allocation determined by the allocation unit. The assigning unit determines the assignment by determining a left-right direction as viewed from the user in a direction of viewing the at least two display devices from a user of the conference terminal device as a left-right direction of the at least two display devices. The server device according to claim 4, wherein: The determination unit
Of the corners formed by lines connecting the second conference terminal device, the first conference terminal device, and the third conference terminal device in order using the position information, the first conference terminal device When the angle formed in the counterclockwise direction from the second conference terminal device to the first conference terminal device is smaller than 180 degrees, it is geographically on the right side with respect to the first conference terminal device. When it is determined that there is the second conference terminal device and the angle formed around the left is greater than 180 degrees, the third conference terminal is located at a geographically right position with respect to the first conference terminal device. The server device according to claim 5, wherein it is determined that there is a device. Based on the control by the server device according to any one of claims 4 to 6, a video conference between at least three points is performed via a network with other second and third conference terminal devices. A first conference terminal device for
A data receiving unit for receiving video data from each of the second and third conference terminal devices;
A video output unit that outputs video data received by the data receiving unit to at least two display devices arranged side by side in the left-right direction;
A notification receiving unit that receives a notification indicating the allocation determined by the allocation unit of the server device from the server device;
The said video output part outputs the said video data which the said data receiving part received with respect to the said corresponding display apparatus according to the said allocation which the said notification receiving part received. The conference terminal device characterized by the above-mentioned. The conference terminal device further includes:
Its own device position information acquisition unit for acquiring geographical position information of the conference terminal device;
The conference terminal device according to claim 7, further comprising a location information notification unit that notifies the server device of the acquired location information. A conference method for a conference terminal device to perform a video conference between at least three points with a first and second terminal devices via a network,
A data receiving step in which video data is received from each of the first and second terminal devices;
A video output step in which the video data received in the data receiving step is output to at least two display devices arranged side by side in the left-right direction;
A location information acquisition step in which geographical location information of the conference terminal device, the first terminal device, and the second terminal device is acquired;
A determination step of determining whether each of the first and second terminal devices is geographically left or right with respect to the conference terminal device using the position information;
In the determination step, the video data from the first terminal device or the second terminal device determined to be in the right position is in the right direction among at least two display devices arranged side by side in the left-right direction. Video data from the first terminal device or the second terminal device determined to be in the left position is output to a display device arranged at the left and right, and at least two displays arranged side by side in the horizontal direction An assignment step in which assignment is determined to output to a display device arranged in the left direction of the device,
In the video output step, the video data received in the data receiving step is output to the corresponding display device according to the allocation determined in the allocation step. A program for a conference terminal device to perform a video conference between at least three points via a network with the first and second terminal devices,
A data receiving step in which video data is received from each of the first and second terminal devices;
A video output step in which the video data received in the data receiving step is output to at least two display devices arranged side by side in the left-right direction;
A location information acquisition step in which geographical location information of the conference terminal device, the first terminal device, and the second terminal device is acquired;
A determination step of determining whether each of the first and second terminal devices is geographically left or right with respect to the conference terminal device using the position information;
In the determination step, the video data from the first terminal device or the second terminal device determined to be in the right position is in the right direction among at least two display devices arranged side by side in the left-right direction. Video data from the first terminal device or the second terminal device determined to be in the left position is output to a display device arranged at the left and right, and at least two displays arranged side by side in the horizontal direction An assignment step in which assignment is determined to output to a display device arranged in the left direction of the device,
In the video output step, a program for causing a computer to output the video data received in the data receiving step to the corresponding display device according to the allocation determined in the allocation step. Control the first, second and third conference terminal devices for performing a video conference between at least three points via the network, and the first, second and third conference terminal devices via the network. A video conference system comprising a server device
The server device
A position information acquisition unit for acquiring geographical position information of the first, second and third conference terminal devices;
Using the location information, it is determined whether each of the second conference terminal device and the third conference terminal device is geographically left or right with respect to the first conference terminal device. A determination unit;
Video data from the second conference terminal device or the third conference terminal device determined by the determination unit to be in the right position is displayed on the right of at least two display devices arranged side by side in the left-right direction. The video data from the second conference terminal device or the third conference terminal device output to the display device arranged in the direction and determined to be in the left position is arranged side by side in the left-right direction. An assigning unit that determines assignment so as to output to a display device arranged in the left direction among at least two display devices;
A notification unit for notifying the first conference terminal device of the allocation determined by the allocation unit;
The first conference terminal device is:
A data receiving unit for receiving video data from each of the second and third conference terminal devices;
A video output unit that outputs video data received by the data receiving unit to at least two display devices arranged side by side in the left-right direction;
A notification receiving unit that receives a notification indicating the allocation determined by the allocation unit of the server device from the server device;
The video output system, wherein the video output unit outputs the video data received by the data receiving unit to the corresponding display device in accordance with the assignment received by the notification receiving unit.

Images