JP2011071822A - Terminal, communication method and communication program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately obtain a situation of a video image at a self-base displayed at other bases. <P>SOLUTION: In video conference terminals 110 which perform a conference using information to be transmitted/received among the self-base and a plurality of other bases connected via a network 150, each video conference terminal 110 picks up a self-base video image of a participant 116 at the self-base, and receives other base video information including other base video images. Each video conference terminal 110 determines whether or not the self-base video image is synthesized with the other base video images based on predetermined conditions. Each video conference terminal 110 generates self-base video information to be transmitted from the self-base video image to other bases when it is determined that the self-base video image is not synthesized with other base video images, and generates the self-base video information from the self-base video image and other base video images when it is determined that the self-base video image is synthesized with other base images. Then, each video conference terminal 110 transmits the generated self-base video information to other bases. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a terminal device, a communication method, and a communication program for transmitting / receiving information between terminal devices, and in particular, uses information transmitted / received between its own base and a plurality of other bases connected via a network. The present invention relates to a terminal device, a communication method, and a communication program that conduct a conference.

  The video conference system transmits and receives video information indicating the status of participants at each base between a plurality of terminal devices. The video conference system decodes the video information of each base transmitted / received by the terminal device and displays the video of each base. Participants of the conference at each site confirm the video displayed by the terminal device and hold the conference.

  In recent years, in order to grasp what kind of video the video information transmitted from the terminal device at the local site is displayed by the terminal device at the partner site, the terminal device at the local site has received Receives playback image quality information at the site. A communication system has been proposed in which the terminal device at its own site outputs the video information transmitted to the other site as a confirmation video with the received playback image quality (Patent Document 1).

JP 2007-150916 A

  However, in the prior art described in Patent Document 1 described above, the terminal device at the local site, in addition to decoding the video information at the partner site, based on the reproduction image quality information received from the partner site, the video information transmitted to the partner site. An example is the problem that it is necessary to perform decoding and a processing load is applied.

  In order to solve the above-described problem, the present invention accurately reduces the processing load with a simple configuration that only decodes video information from the partner site, and accurately displays the video status of the site at the partner site. It is an object to provide a terminal device, a communication method, and a communication program that can be grasped.

  In order to solve the above-described problems and achieve the object, the terminal device according to the first aspect of the present invention uses information transmitted and received between the base and a plurality of other bases connected via the network. A terminal device that performs a conference, an imaging unit that captures an image of a local site including a participant at the local site, a receiving unit that receives video information of another site including a video of another site from the other site, and a predetermined unit Based on the condition, the determination unit for determining whether or not to synthesize the own site video imaged by the imaging unit and the other site video, the own site image and the other site video by the determination unit If it is determined not to synthesize the video, the local site video information to be transmitted to the other site is generated from the local site video, and if it is determined to be combined, the local site video and the other site video From the above Generating means for generating video information, the self-location video information generated by the generating means, characterized by comprising a transmitting means for transmitting to said remote point.

  According to a second aspect of the present invention, the terminal device according to the first aspect of the present invention is characterized in that the determination means determines that the local site video and the other site video are to be combined in a predetermined cycle.

  The terminal device according to a third aspect of the invention is characterized in that, in the invention according to the first or second aspect, the determination means determines that the local base image and the other base video are combined at a predetermined timing. To do.

  The terminal device according to a fourth aspect of the present invention is the terminal device according to any one of the first to third aspects, wherein the determining means adds the own base video to the other base video included in the other base video information. Is determined, and if the local video is included, it is determined not to synthesize the local video and the other video.

  According to a fifth aspect of the present invention, in the terminal device according to any one of the first to fourth aspects, the receiving means receives the other base video information from each of the plurality of other bases, and When the other-site video information includes the own-site video, the difference between the own-site videos included in the other-site video information is calculated, and a warning is notified to the participant based on the calculation result. It is characterized by further comprising an informing means for performing.

  The communication method according to the invention of claim 6 is a communication method for conducting a conference using information transmitted and received between the local site and a plurality of other sites connected via a network, the local site An imaging process for capturing a video of the local site including the participants, a receiving process for receiving video information of the other base including the video of the other base from the other base, and an image captured by the imaging process based on a predetermined condition. A determination step of determining whether or not to synthesize the own-site video and the other-site video included in the other-site video information received by the reception step; the own-site video by the determination step; If it is determined not to synthesize another base video, the local base video information to be transmitted to the other base is generated from the local base video, and if it is determined to be combined, the local base video and the other base video are determined. Projection Generating step and said self-location video information generated by the generating process of generating the self-location video information from and to, characterized in that it comprises a transmission step of transmitting to said other hub.

  A communication program according to a seventh aspect of the invention is a communication program for a terminal device that conducts a conference using information transmitted and received between its own base and a plurality of other bases connected via a network. Based on a predetermined condition, an imaging step of capturing a video of the local site including a participant at the local site, a receiving step of receiving video information of another site including the video of another site from the other site, and the imaging A determination step for determining whether or not to synthesize the own-site video captured in the process and the other-site video; and the determination step determines that the own-site video and the other-site video are not combined. If it is determined to generate and synthesize own-site video information to be transmitted from the own-site video to the other site, the own-site video is generated from the own-site video and the other-site video. A generation step of generating information, the self-location video information is generated by the generating step, characterized in that to execute a transmission step of transmitting, to the computer to the remotely located site.

  According to the first aspect of the present invention, it is possible to transmit own-site video information obtained by combining the other-site video received from the other site with the own-site video captured at the own site. Therefore, at the other site, by confirming the own site image transmitted from the own site, the image of the other site displayed at the own site can be confirmed together with the image of the own site. Similarly, the local site can check the video of the other site together with the video of the other site by checking the video of the other site transmitted from the other site. While reducing the processing load, it is possible to grasp the state of the video displayed at the partner site with each other.

  According to the second aspect of the present invention, it is possible to synthesize the local site video and the other site video in a predetermined cycle. In other words, it is sufficient to synthesize the video of the local site displayed at the other site without synthesizing all the videos, so the load of the composition process between the local video and the other site is reduced. be able to.

  According to the invention of claim 3, since the local video and the video of the other base can be synthesized at a predetermined timing, the composite video is generated at an appropriate timing at which the video of the local base displayed at the other base can be confirmed. Processing can be optimized.

  According to the fourth aspect of the present invention, when the local video is included in the video of the other site, the video of the local site and the video of the other site are not combined. It is possible to prevent an infinite chain of video from other sites.

  According to the fifth aspect of the present invention, the video of the local site cannot be properly transmitted because the video of the local site included in the video of other sites from a plurality of sites can be compared and a warning can be notified when the difference is large. It is possible to accurately grasp other bases.

  According to the sixth aspect of the present invention, it is possible to transmit own-site video information obtained by combining the other-site video received from the other site with the own-site video captured at the own site. Therefore, at the other site, by confirming the own site image transmitted from the own site, the image of the other site displayed at the own site can be confirmed together with the image of the own site. Similarly, the local site can check the video of the other site together with the video of the other site by checking the video of the other site transmitted from the other site. While reducing the processing load, it is possible to grasp the state of the video displayed at the partner site with each other.

  According to the seventh aspect of the present invention, it is possible to transmit own-site video information obtained by combining the other-site video received from the other site with the own-site video captured at the own site. Therefore, at the other site, by confirming the own site image transmitted from the own site, the image of the other site displayed at the own site can be confirmed together with the image of the own site. Similarly, the local site can check the video of the other site together with the video of the other site by checking the video of the other site transmitted from the other site. While reducing the processing load, it is possible to grasp the state of the video displayed at the partner site with each other.

  As described above, according to the terminal device, the communication method, and the communication program according to the present invention, the processing load is reduced with a simple configuration, and the situation of the image of the local site displayed at the partner site is accurately grasped. can do.

It is explanatory drawing which shows an example of the video conference system concerning Embodiment 1 of this invention. It is explanatory drawing which shows an example of a functional structure of the video conference terminal concerning Embodiment 1 of this invention. It is explanatory drawing which shows an example of the synthetic | combination image | video displayed as a meeting image | video concerning Embodiment 1 of this invention. It is a flowchart which shows the content of the process of the video conference terminal concerning Embodiment 1 of this invention. It is a flowchart which shows the content of the production | generation process of the video packet concerning Embodiment 1 of this invention. It is explanatory drawing which shows an example of transmission / reception of the video packet concerning Embodiment 1 of this invention. It is a flowchart which shows the content of the production | generation process of the video packet of the own base concerning the modification of this invention. It is a flowchart which shows the content of the image | video comparison and alerting | reporting process concerning Embodiment 2 of this invention. It is explanatory drawing explaining the outline | summary of the video comparison concerning Embodiment 2 of this invention.

  Exemplary embodiments of a terminal device, a communication method, and a communication program according to the present invention will be explained below in detail with reference to the accompanying drawings.

(Embodiment 1)
(overall structure)
A case where the terminal device according to the first embodiment of the present invention is applied to video conference terminals installed at a plurality of bases for a video conference system that performs a video conference will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating an example of a video conference system according to the first embodiment of the present invention. In the first embodiment, the terminal device according to the present invention is realized by the video conference terminal 110 (110a, 110b, 110c, 110d) installed at each base (A, B, C, D), and the network 150 Processing of the communication method according to the present invention is executed by the video conference system 100 or the video conference terminal 110 to which the video conference terminal 110 is connected via the network, and processing is executed on the video conference terminal 110 by the communication program according to the present invention. The case where it is made to explain is demonstrated.

  In FIG. 1, the video conference system 100 is configured by connecting video conference terminals 110 a, 110 b, 110 c, and 110 d installed at each base (A, B, C, D) via a network 150. Specifically, video conference terminals 110a, 110b, 110c, and 110d installed at geographically separated locations (A, B, C, and D) are connected via a network such as the Internet, Video conference terminals 110a, 110b, 110c, and 110d installed at remote locations (A, B, C, and D) are connected via a network 150 such as a LAN (local area network). The network 150 may be configured by a public telephone line network. In FIG. 1, the video conference terminals 110a, 110b, 110c, and 110d are described as being connected to each other via a network. However, the video conference terminals 110a, 110b, 110c, and 110d are mutually connected via a management server installed at an arbitrary position on the network. It may be configured to be connected. In the following description, when the locations are not distinguished, the symbols “a”, “b”, “c”, and “d” that are symbols at the end of the symbols are omitted.

  The video conference system 100 allows each video conference terminal 110 to mutually transmit and receive video and audio of the participants 116 participating in the video conference at each site. The video conference terminal 110 is connected to a main body 111 which is a housing of a functional unit such as a CPU (Central Processing Unit). A microphone 114 that collects the voice of the participant 116 and a speaker 115 that outputs various voices are provided.

  The video conference terminal 110 captures an image of the participant 116 at the local site using the camera 113. The video conference terminal 110 transmits a video packet including the captured video to the video conference terminal 110 at another site via the network 150.

  The video conference terminal 110 collects the voice of the participant 116 at the local site using the microphone 114. The video conference terminal 110 transmits a voice packet including the voice of the collected participant 116 to the video conference terminal 110 at another site via the network 150.

  The video conference terminal 110 receives a video packet including the video of the participant 116 transmitted from the video conference terminal 110 at another base. The video conference terminal 110 decodes the received video packet and displays the conference video on the display unit 112.

  The video conference terminal 110 receives a voice packet including the voice of the participant 116 transmitted from the video conference terminal 110 at another base. The video conference terminal 110 decodes the received voice packet and outputs the conference voice through the speaker 115.

  When the video conference terminal 110 generates a video packet, the video conference terminal 110 determines whether to synthesize the video of the other site included in the video packet received from the other site with the video captured by the camera 113 of the local site. The judgment regarding the composition of the video is not performed this time when the composition was performed last time, for example, depending on whether the composition was performed on the video packet transmitted to the other site last time. That is, synthesis and non-synthesis are alternately repeated as a predetermined cycle. In the following description, the local site will be described as the A site where the video conference terminal 110a is installed, and the other site will be described as the B site where the video conference terminal 110b is installed. , C, D).

  When the video conference terminal 110a determines that the video of the B site based on the received video packet is not combined with the video of the A site, the video conference terminal 110a generates the video captured by the camera 113a as a video packet to be transmitted to the B site. That is, only the video of the A base is transmitted to the B base.

  If the video conference terminal 110a determines to synthesize the video of the B site based on the received video packet with the video of the A site, the video conference terminal 110a transmits the synthesized video obtained by synthesizing the video of the A site and the video of the B site to the B site. Generated as a video packet. Specifically, the video conference terminal 110a combines the video of the B site in the conference video based on the video packet received from the B site with the video of the A site captured by the camera 113a. The video conference terminal 110 a packetizes an image obtained by combining the image of the B site with the image of the A site, and transmits the packet to the B site via the network 150.

  The conference video displayed by the video conference terminal 110b at site B includes the video at site B displayed at site A at a predetermined rate. Therefore, the participant 116b can easily grasp how the video of the B site is displayed at the A site by confirming the video of the B site included in the displayed conference video. Similarly, the conference video displayed by the video conference terminal 110a includes the video of the A site that should have been displayed at the B site at a predetermined rate. Therefore, the participant 116a can easily grasp how the video of the A base is displayed at the B base by confirming the video of the A base included in the displayed conference video.

(Functional configuration)
The functional configuration of the video conference terminal 110 will be described with reference to FIG. FIG. 2 is an explanatory diagram illustrating an example of a functional configuration of the video conference terminal according to the first embodiment of the present invention.

  In FIG. 2, a video conference terminal 110 inputs and outputs video to / from a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, a display unit 112 and a camera 113. Communication I / F 204, audio I / F 205 that controls input / output of audio to / from the speaker 115 and microphone 114, an operation unit 206 that receives input of various information, and communication that controls communication with external devices An I / F 207 and a storage medium 208 that stores various types of information are provided. Each component of the video conference terminal 110 is connected by a bus 200.

  The CPU 201 controls the entire video conference terminal 110. The CPU 201 executes various programs read from the ROM 203 using the RAM 202 as a work area.

  The video I / F 204 displays various types of information on the display unit 112 under the control of the CPU 201. The video I / F 204 reads a video packet received from the video conference terminal 110 at another site from the storage medium 208 according to the control of the CPU 201, decodes it, and displays it on the display unit 112 as a conference video. The video I / F 204 may be configured to display a video of the local site imaged by the camera 113, an input screen for accepting an input of processing related to a video conference with another site, and the like.

  The video I / F 204 captures a video of the participant 116 at the local site by the camera 113 under the control of the CPU 201. The video I / F 204 outputs the video captured by the camera 113 to the storage medium 208 according to the control of the CPU 201.

  The sound I / F 205 causes the speaker 115 to output various sounds according to the control of the CPU 201. The audio I / F 205 reads out and decodes an audio packet received from the video conference terminal 110 at another site from the storage medium 208 according to the control of the CPU 201 and outputs the audio packet to the speaker 115 as conference audio. The voice I / F 205 may be configured to output a guidance voice of an instruction command related to a video conference with another base.

  The voice I / F 205 collects the voice of the participant 116 at the local site using the microphone 114 according to the control of the CPU 201. The sound I / F 205 outputs the sound collected by the microphone 114 to the storage medium 208 according to the control of the CPU 201.

  The operation unit 206 receives input of various information from the participant 116 and the like. The operation unit 206 includes a touch panel, operation buttons, and the like. The operation unit 206 accepts input of information regarding a video conference and outputs an input signal to the CPU 201.

  The communication I / F 207 is connected to a network 150 such as the Internet through a communication line, and is connected to other video conference terminals 110 and other external devices via the network 150. A communication I / F 207 controls an interface between the network 150 and the video conference terminal 110 and controls data input / output with respect to an external device. As the communication I / F 207, for example, a modem or a LAN adapter can be employed.

  The communication I / F 207 receives video packets and audio packets transmitted from the video conference terminal 110 at another base. The communication I / F 207 outputs the received video packet and audio packet to the recording medium 208 according to the control of the CPU 201.

  The communication I / F 207 transmits the video and audio of the local site stored in the storage medium 208 to the video conference terminal 110 at another site according to the control of the CPU 201. The communication I / F 207 transmits video packets and audio packets including video and audio at a predetermined timing under the control of the CPU 201.

  Specifically, the CPU 201 generates a video packet and an audio packet for transmission to the video conference terminal 110 at another site from the video and audio at the local site stored in the recording medium 208. The CPU 201 transmits the generated video packet and audio packet to the video conference terminal 110 at another base via the communication I / F 207.

  When generating the video packet, the CPU 201 determines whether or not to synthesize the video of the other site with the video of the local site, and generates the video packet according to the determination result. The CPU 201 determines that the video packet transmitted to the other site is combined before it is combined.

  When it is determined that the video of the local site is to be combined with the video of the local site, the CPU 201 combines the video of the local site read from the storage medium 208 with the video of the other site in the conference video displayed on the display unit 112. The video is generated as a video packet to be transmitted to another site.

  Here, the synthesized video displayed as the conference video according to the first embodiment of the present invention will be described with reference to FIG. FIG. 3 is an explanatory diagram illustrating an example of a composite video displayed as a conference video according to the first embodiment of the present invention. In FIG. 3, a composite video obtained by synthesizing the video of the A base and the video of the B base from the video conference terminal 110a of the A base is transmitted as a video packet to the video conference terminal 110b of the B base. The conference video decoded at 110b will be described.

  In FIG. 3, the video conference terminal 110b at site B decodes the video packet received from the video conference terminal 110a at site A. The video conference terminal 110b displays the conference video 300 obtained by decoding the video packet on the display unit 112b.

  On the display unit 112b, as the conference video 300, the A base image 301 of the participant 116a at the transmission source A base and the B base video 302 of the participant 116b at the B base are combined and displayed. That is, the video conference terminal 110a of the A site generates a video packet of a composite video in which the video of the B site transmitted from the B site and the video of the A site captured by the camera 113a are synthesized. The video packet is transmitted from the video conference terminal 110a at the A site to the video conference terminal 110b at the B site.

  Specifically, when generating the video packet of the conference video 300, the video conference terminal 110a at the A site decodes the video packet transmitted from the B site. The video conference terminal 110a displays the conference video obtained by decoding the video packet on the display unit 112a, and determines whether or not the video packet of the A site previously transmitted to the B site has been combined with the video of the B site. .

  When the video conferencing terminal 110a has not performed the synthesis process last time, the video conference terminal 110a generates a synthesized video obtained by synthesizing the video of the A site with the video of the A site captured by the camera 113a as a video packet to be transmitted to the B site. The video conference terminal 110a is configured to transmit the generated video packet to the video conference terminal 110b at the base B via the communication I / F 207a.

  The participant 116b at the B base confirms the B base video 302 included in the conference video 300 displayed on the display unit 112b, so that the video transmitted from the video conference terminal 110b at the B base Thus, it is possible to easily grasp whether the information is displayed without inquiring of the base A.

  Returning to FIG. 2, the storage medium 208 is an HD (hard disk), an FD (flexible disk) as an example of a removable recording medium, or the like. The storage medium 208 has respective drive devices, and various data are recorded under the control of the CPU 201. Further, data is read from the storage medium 208 according to the control of each drive device.

  If each component is described in association with each function, the function of the imaging means according to the present invention is realized by the CPU 201, the video I / F 204, and the camera 113 shown in FIG. The CPU 201 and the communication I / F 207 realize the functions of the reception unit and the transmission unit according to the present invention. The CPU 201 implements the functions of the determination unit and the generation unit according to the present invention.

(Contents of processing of the video conference terminal 110)
The contents of processing of the video conference system 100 according to the first embodiment of the present invention will be described with reference to FIGS. 4 to 6, a case where a video conference is performed by the video conference terminal 110a at the A site and the video conference terminal 110b at the B site will be described. FIG. 4 is a flowchart showing the processing contents of the video conference terminal according to the first embodiment of the present invention. FIG. 4A is a flowchart showing the contents of processing of the video conference terminal at the A site. FIG. 4B is a flowchart showing the contents of processing of the video conference terminal at site B. Note that the processing from step S451 to step S457 in FIG. 4B is substantially the same as the processing from step S401 to step S407 in FIG.

  In the flowchart of FIG. 4A, first, the CPU 201a determines whether or not a video conference is started (step S401). To start the video conference, for example, based on the operation of the operation unit 206a by the participant 116a, a connection request is made to the other video conference terminal 110a via the communication I / F 207a. This is determined by receiving a response from the other video conference terminal 110b via the communication I / F 207a.

  In step S401, the CPU 201 waits for the start of the video conference and starts the video conference (step S401: Yes), and the CPU 201 captures the image of the site A by the camera 113a via the video I / F 204a (step S402). ).

  CPU201a judges whether the video packet was received from B base by communication I / F207a (step S403).

  In step S403, when the video packet is not received from the site B (step S403: No), the CPU 201 determines whether a predetermined time has elapsed and timed out (step S404). The elapse of the predetermined time is measured, for example, with a timer (not shown) as a starting point from the start of the conference or the immediately preceding decoding.

  In step S404, if the timeout has not occurred (step S404: No), the process returns to step S403 to repeat the process. If the timeout has occurred (step S404: Yes), the process proceeds to step S405 as it is.

  In step S403, when the video packet is received from the site B (step S403: Yes), the CPU 201a stores the received video packet in the storage medium 208a, and proceeds to step S405.

  The CPU 201a controls the video I / F 204a, reads and decodes the video packet of the other site stored in the storage medium 208a, and displays the conference video on the display unit 112a (step S405). In this step S405, even if the video packet of the B site is not stored in the storage medium 208a in the unreceived state or the video packet of the B site is missing, it is decoded as it is and a black color image or a part of it is decoded. An image with missing is displayed.

  The CPU 201a performs a generation / transmission process of the video packet at the site A (step S406). Here, the generation / transmission processing of the video packet of the video conference terminal according to the first embodiment of the present invention will be described with reference to FIG. With reference to FIG. 5 (a), the video packet generation / transmission process (step S406 shown in FIG. 4 (a)) at the A site in the video conference terminal 110a according to the first embodiment of the present invention will be described. FIG. 5A is a flowchart showing the contents of the video packet generation / transmission process (step S406 shown in FIG. 4A) at the site A according to the first embodiment of the present invention. FIG. 5B is a flowchart showing the contents of the video packet generation / transmission process (step S456 shown in FIG. 4B) at the base B according to the first embodiment of the present invention. Since the processing from step S551 to step S554 in FIG. 5B is substantially the same as the processing from step S501 to step S504 in FIG.

  In the flowchart of FIG. 5A, first, the CPU 201a determines whether or not the video packet transmitted to the video conference terminal 110b at the base B before combining video is displayed in step S405 (step S405). S501).

  In step S501, if not previously synthesized (step S501: No), the CPU 201a synthesizes the video of the B base in the conference video to be displayed with the video of the A base read from the storage medium 208a (step S502). ). The CPU 201a generates a video packet to be transmitted to the site B based on the synthesized video synthesized in step S502 (step S503).

  The CPU 201a transmits the video packet generated in step S503 to the B site via the communication I / F 207a (step S504), ends the generation / transmission processing of the video packet in the A site as it is, The process returns to step S407 shown in a).

  In step S501, when combined last time (step S501: Yes), the CPU 201 proceeds to step S503, and based on the video of the local site (A site) captured in step S402, another site (B site). A video packet to be transmitted is generated (step S503).

  The CPU 201a transmits the video packet generated in step S503 to the other site via the communication I / F 207a (step S504), ends the generation / transmission processing of the video packet of the A site as it is, and the process shown in FIG. The process returns to the step S407 shown.

  Returning to FIG. 4A, the CPU 201a determines whether or not the video conference started in step S401 is ended (step S407). To end the video conference, for example, based on the operation of the operation unit 206a by the participant 116a, a disconnection request is made to the other video conference terminal 110b via the communication I / F 207a. This is determined by receiving a response from the other video conference terminal 110b via the communication I / F 207a.

  If the video conference is continued in step S407 (step S407: No), the process returns to step S402 and the process is repeated. In step S407, when the video conference is ended (step S407: Yes), the series of processes is ended as it is.

  Here, the content of the processing of the video conference system 100 in FIGS. 4 and 5 will be specifically described with reference to FIG. FIG. 6 is an explanatory diagram showing an example of transmission / reception of video packets according to the first embodiment of the present invention. 6 (a) and 5 (a) showing the processing contents of the video conference terminal 110a at the A site, and FIG. 4 (b) and FIG. 6 showing the processing contents of the video conference terminal 110b at the B site. Description will be made according to the order of the flowchart of FIG.

  In FIG. 6, when the video conference is started in step S401, the video conference terminal 110a at the site A captures an image of the site A as its own site in step S402 (T1). The video conference terminal 110b at site B performs the same processing in steps S451 and S452.

  Since the video conference terminal 110a is before receiving the video packet from the partner site, the video conference terminal 110a times out in step S404 (T2). In other words, the video conference terminal 110a times out because it is before transmitting the video packet to the B site that is the partner site in step S504. The video conference terminal 110b at site B performs the same process in step S454.

  Since the video conference terminal 110a has not received the video packet from the B site that is the partner site, in step S405, the video conference terminal 110a displays a black image indicating that there is no video at the B site (T3). The video conference terminal 110b performs the same processing in step S455.

  When the video conference terminal 110a shifts to the video packet generation / transmission processing in step S406, since the video packet was not synthesized last time (step S501: No), in step S502, the black video that is the video of the partner site is displayed. And the video captured at the local site are combined to generate a video packet to be transmitted to the partner site in step S503 (T4). The video conference terminal 110b performs the same processing in step S456, step S551, step S552, and step S553.

  When the video conference terminal 110a transmits the video packet generated in step S503 to the partner B site (T5), the video conference terminal 110a receives the video packet transmitted from the B site and displays it as a conference video (T6). The video conference terminal 110b performs the same process in step S553.

  When the conference is continued in step S407, the video conference terminal 110a newly captures an image of the local site in step S402 (T7). The video conference terminal 110b performs the same processing in step S457 and step S452.

  When the video conference terminal 110a shifts to the generation / transmission processing of the video packet in step S406, since the video packet was previously combined in T4, the video conference terminal 110a does not perform the combining and performs the step from the video of the base A as its own base. In S503, a video packet to be transmitted to the B site which is the partner site is generated (T8). The video conference terminal 110b performs the same processing in steps S456 and S553.

  When the video conference terminal 110a transmits the video packet generated in step S503 to the partner B site (T9), the video conference terminal 110a receives the video packet transmitted from the B site and displays it as a conference video (T10). The video conference terminal 110b performs the same process in step S553.

  When the conference is continued in step S407, the video conference terminal 110a newly captures an image of the site A in step S402 (T11). The video conference terminal 110b performs the same processing in steps S457 and S452.

  When the video conference terminal 110a shifts to the video packet generation / transmission process in step S406, the video conference terminal 110a is the partner site that received and displayed in T10 in step S502 because the video packet was not synthesized in T8 last time. The video of the B site and the video captured at the A site, which is its own site, are combined, and a video packet to be transmitted to the B site is generated in step S503 (T12). The video conference terminal 110b performs the same processing in step S456, step S552, and step S553.

  When the video conference terminal 110a transmits the video packet generated in step S503 to the site B (T13), the video conference terminal 110a receives the video packet transmitted from the site B and displays it as a conference video (T14). The video conference terminal 110b performs the same process in step S553.

  The video conference terminal 110a generates a video packet by repeating the synthesis of the video at the B site and the non-synthesis for the video at the A site for the duration of the conference in step S407, and transmits the video packet to the B site. Will be. Similarly, the video conference terminal 110b has a configuration in which video packets are generated by repeating the synthesis and non-synthesis of video while the conference is continued in step S457, and is transmitted to the site A.

  The communication method in each component of the present invention and each process or each function of the first embodiment of the present invention will be described in association with each other by the processes of the CPU 201, video I / F 204, and camera 113 in step S402 (S452). The process of the imaging process according to the present invention is executed. The processing of the reception process according to the present invention is executed by the processing of the CPU 201 and the communication I / F 207 in step S403 (S453) and step S404 (S454). By the processing of the CPU 201 and the storage medium 208 in step S406 (steps S501 to S503) (S456 (S551 to S553)), the determination process and the generation process according to the present invention are executed. By the processing of the CPU 201 and the communication I / F 207 in step S504 (S554), the transmission process according to the present invention is executed.

  As described above, according to the first embodiment of the present invention, it is possible to transmit own-site video information obtained by combining the other-site video received from the other site with the own-site video captured at the own site. Therefore, at the other site, by confirming the own site image transmitted from the own site, the image of the other site displayed at the own site can be confirmed together with the image of the own site.

  Similarly, by checking the video of other sites transmitted from other sites, you can check the video of your site displayed at other sites along with the video of other sites. Without decoding the image, the processing load can be reduced with a simple configuration, and the state of the video displayed at the partner site can be grasped mutually.

  Further, since the composition and the non-synthesis are alternately repeated, the synthesized video can be repeatedly synthesized to prevent the synthesized video from being chained, and a wasteful synthesis process is not performed.

(Other variations)
In Embodiment 1 of the present invention, in particular, the configuration is such that the video to be displayed mutually is transmitted and received as a video packet. That is, the data format for transmitting and receiving video may be any data format that is suitable for transmission and reception between the video conference terminals 110 that conduct a video conference. By doing in this way, the versatility can be improved about the data format transmitted / received.

  In the first embodiment of the present invention, as shown in FIG. 6, the transmission and reception of the synthesized video and the video that has not been synthesized are described at the same timing for the video at the A site and the B site. There is nothing. Specifically, the B base video packet based on the video that is not combined may be transmitted from the B base at the timing when the A base video packet based on the composite video is transmitted from the A base.

  Further, in the first embodiment of the present invention, in particular, in step S501 and step S551, it has been described that synthesis and non-synthesis are alternately repeated as a predetermined cycle, but the present invention is not limited to this. Specifically, for example, instead of repeating alternately, a configuration in which synthesis and non-synthesis are repeated a plurality of times may be employed. That is, non-synthesis may be repeated a plurality of times after one synthesis, and the synthesis processing load can be optimized. Further, the synthesis may be determined by detecting a predetermined timing instead of the predetermined cycle.

  Here, the generation / transmission process of the video packet of the local site according to the present modification will be described with reference to FIG. FIG. 7 is a flowchart showing the contents of the video packet generation process (step S406 shown in FIG. 4A or step S456 shown in FIG. 4B) according to the modification of the present invention. In the flowchart of FIG. 7, first, the CPU 201 determines whether or not a predetermined timing has come (step S701). The predetermined timing is a timing such as when the video conference terminal 110 is activated, when the network 150 environment changes, when the video conference terminal 110 is newly connected to or disconnected from the video conference.

  In step S701, when the predetermined timing is reached (step S701: Yes), the CPU 201 synthesizes the video of the other site in the conference video to be displayed with the video of the local site read from the storage medium 208 (step S702). ).

  The CPU 201 generates a video packet to be transmitted to another site based on the synthesized video obtained by synthesizing the video of the other site in the conference video with the video of its own site in step S702 (step S703).

  The CPU 201 transmits the video packet generated in step S703 to the other site via the communication I / F 207 (step S704), ends the generation / transmission processing of the video packet of its own site as it is, and FIG. The process returns to step S407 shown in a) (or step S457 shown in FIG. 4B).

  In step S701, when the predetermined timing is not reached (step S701: No), the CPU 201 proceeds to step S703, and transmits a video packet to another site based on the video of the local site imaged in step S402. Is generated (step S703).

  The CPU 201 transmits the video packet generated in step S703 to the other site via the communication I / F 207 (step S704), ends the generation / transmission processing of the video packet of its own site as it is, and FIG. The process returns to step S407 shown in a) (or step S457 shown in FIG. 4B).

  In this way, by determining whether or not it is necessary to synthesize at a predetermined timing without using the conference video, an infinite chain of multiple local video and other site video enters the video to be displayed with a simple configuration. Can be prevented. Further, it is possible to optimize the composite video generation process by determining whether video synthesis is necessary at a predetermined timing.

  Further, in the first embodiment of the present invention, it has been particularly described that the composition and non-synthesis of the video are repeated in the Australian language as the predetermined period elapses. However, the present invention is not limited to this. Specifically, based on whether or not the video of the local site is included in the conference video, it may be determined that the video of the local site is combined with the video of the local site. That is, if the conference video obtained by decoding the video packet transmitted from another site does not include the video of the local site, it is determined that the video is synthesized.

  If the conference video based on the video packet sent from another site contains the own site video, the local site video and the other site video will not be combined, so multiple local site videos will be included in the video to be displayed. Can be prevented, and a smooth video conference can be performed with an optimal conference video.

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described. In the second embodiment of the present invention, a case will be described in which when the conference video described in the first embodiment is received from a plurality of other bases, the other base having a display defect is notified. In addition, about the outline | summary of the video conference system 100 concerning Embodiment 2 of this invention, the functional structure of the video conference terminal 110, and the content of the process of the video conference terminal 110, FIG. 1, FIG. Since it is substantially the same as FIG. 4 and FIG. 5, description is abbreviate | omitted.

  In Embodiment 2 of the present invention, video packets received from a plurality of other sites during steps S404 to S407 shown in FIG. 4A or steps S454 to S457 shown in FIG. It is the structure which performs a comparison and alerting | reporting process. That is, the communication I / F 207 receives video packets transmitted from the video conference terminals 110 at a plurality of other bases and outputs them to the recording medium 208.

  The CPU 201 determines whether or not the video of the local site is included in the conference video obtained by decoding the video packets transmitted from a plurality of other bases. If the video is included, the CPU 201 calculates the difference between the videos of the local site. To do. When the difference is equal to or greater than a certain value, the CPU 201 is configured to notify the participant 116 of that fact by the display unit 112 or the speaker 115.

(Contents of video comparison / notification process)
Here, the video comparison / informing process of the conference video by the video conference terminal 110 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart showing the contents of the video comparison / notification process according to the second embodiment of the present invention. In the flowchart of FIG. 8, first, the CPU 201 determines whether or not there is a video of the local site in the conference video based on video packets received from a plurality of other bases (step S801).

  In step S801, when there is no video of the local site (step S801: No), the series of video comparison / notification processing is terminated as it is, and the process returns to the flowchart of FIG. In step S1301, when there is a video of the local site (step S801: Yes), the CPU 201 calculates a difference between pixels of the video of the local site included in the conference video (step S802).

  The CPU 201 determines whether or not the difference calculated in step S802 is greater than or equal to a certain value (step S803). The determination of whether or not the difference is greater than or equal to a certain value is, for example, determining that the video of the local site is not being displayed appropriately.

  Here, with reference to FIG. 9, the video conference terminal 110b at the B base is the video conference terminals 110a, 110c at the A, C, and D bases for an overview of the video comparison by the video conference terminal 110 according to the second embodiment of the present invention. An example in which a video packet of a conference video including a video of the site A is received from 110d will be described. FIG. 9 is an explanatory diagram for explaining an overview of video comparison according to the second embodiment of the present invention.

  In FIG. 9, the B conference video conference terminal 110b receives video packets from the video conference terminals 110a, 110c, and 110d. The video conference terminal 110b decodes the video packets received from the A, C, and D sites as a conference video 900 at the A site, a conference video 910 at the C site, and a conference video 920 at the D site, respectively.

  In the conference video 900, the A base image 901 of the participant 116a and the B base video 902 of the participant 116b displayed at the A base are combined. The conference video 910 is composed of the C base video 911 of the participant 116c and the B base video 912 of the participant 116b displayed at the C base. In the conference video 920, the D base video 911 of the participant 116d and the B base video 922 of the participant 116b displayed at the C base are combined.

  In the B base image 922 in the conference video 920, a missing part 950 exists, and when the difference between the pixels of the B base images 902, 912, and 922 as the own base of the B base is calculated, A certain difference or more is calculated.

  The video conference terminal 110b determines that the video packet of the base B, which is its own base, has not been properly transmitted to the base D of the transmission source of the conference video 920 including the base B video 922 that has a certain difference or more. , To that effect. The notification displays, for example, a message 960 such as “Not successfully reproduced at D site” for the participants 116b at B site.

  Returning to FIG. 8, if there is no difference greater than or equal to a certain value in step S803 (step S803: No), the series of video comparison / notification processing is terminated as it is, and the flow returns to the flowchart of FIG.

  In step S803, when there is a certain difference or more (step S803: Yes), the CPU 201 controls the video I / F 204 to display that there is a certain difference or more on the display unit 112 (step S803). S804), the series of video comparison / notification processing is terminated, and the process returns to the flowchart of FIG.

  As described above, according to the second embodiment of the present invention, it is possible to compare the video of the local site included in the video of the other site and notify the warning when the difference is large. It is possible to accurately grasp other sites that could not be transmitted.

(Other variations)
In Embodiment 2 of the present invention, in particular, the other site where the video of the local site has not been properly transmitted is notified to the participant 116 of the local site, but the present invention is not limited to this. Specifically, in the example of FIG. 9, for example, the video conference terminal 110 b may notify other sites A, C, and D between sites where video is not properly transmitted. By notifying each other that communication has not been properly established, the participant 116 can identify between bases where communication cannot be established.

  In the second embodiment of the present invention, the difference between the pixels of the B base images 902, 912, and 922 in the conference images 900, 910, and 920 has been described. When the B base images 902, 912, and 922 are transmitted with the stamp attached, the difference may be calculated when the time stamp of the same time is attached. By using the time stamp, it is possible to accurately grasp a communication failure.

  In the second embodiment of the present invention, the difference between the pixels of the B base images 902, 912, and 922 in the conference images 900, 910, and 920 has been described. However, the present invention is not limited to this. Specifically, for example, the video conference terminal 110b can temporarily hold the B base images 902, 912, and 922 transmitted by the own device. In the configuration in which the difference between each B base image 902, 912, 922 in each conference video 900, 910, 920 and the B base image 902, 912, 922 that is temporarily held (and transmitted by the device itself) is calculated. Good. In this way, it is possible to accurately grasp a communication failure.

  In the second embodiment of the present invention, the message 960 is displayed to indicate that there is a certain difference or more. However, the present invention is not limited to this. Specifically, the conference video 920 from the D site having a certain difference or more may be highlighted or a voice notification may be performed. Thus, the convenience of the video conference terminal 110 can be improved by applying various notification forms.

  In the second embodiment of the present invention, the message 960 is displayed in particular as a video packet is not properly transmitted to the site D. However, the present invention is not limited to this. Specifically, a message such as “There is a problem in transmission from the D site” or “There is a problem in communication with the D site” regarding a problem in the video packet transmitted from the D site may be used.

  In the above description, the first embodiment, the second embodiment, and some modified examples have been described as separate examples, but the present invention is not limited thereto. That is, the methods according to the first embodiment, the second embodiment, and some modifications may be used in appropriate combination.

  Note that the communication methods described in the first and second embodiments and the modifications of the present invention can be realized by executing a communication program prepared in advance on a computer such as a personal computer or a workstation. The communication program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The program may be a transmission medium that can be distributed via a network such as the Internet.

100 Video conference system 110 (110a, 110b, 110c, 110d) Video conference terminal 111 (111a, 111b, 111c, 111d) Housing 112 (112a, 112b, 112c, 112d) Display unit 113 (113a, 113b, 113c, 113d) ) Camera 114 (114a, 114b, 114c, 114d) Microphone 115 (115a, 115b, 115c, 115d) Speaker 116 (116a, 116b, 116c, 116d) Participant 150 Network 200 Bus 201 CPU
202 RAM
203 ROM
204 Video I / F
205 Voice I / F
206 Operation unit 207 Communication I / F
208 storage media

Claims (7)

A terminal device that conducts a conference using information transmitted and received between its own base and a plurality of other bases connected via a network,
Imaging means for imaging a video of the local site including participants at the local site;
Receiving means for receiving other site video information including other site video from the other site;
A determination means for determining whether to synthesize the own-site video imaged by the imaging means and the other-site video based on a predetermined condition;
When it is determined by the determining means that the local site video and the other site video are not to be combined, it is determined that the local site video information to be transmitted from the local site video to the other site is generated and combined. A generating means for generating the own-site video information from the own-site video and the other-site video;
Transmitting means for transmitting the local video information generated by the generating means to the other base;
A terminal device comprising:   The terminal device according to claim 1, wherein the determination unit determines to synthesize the local site video and the other site video in a predetermined cycle.   The terminal device according to claim 1, wherein the determination unit determines to synthesize the local site video and the other site video at a predetermined timing.   The determination means determines whether or not the own-site video is included in the other-site video included in the other-site video information, and when the own-site video is included, The terminal device according to claim 1, wherein the terminal device is determined not to be combined with the other-site video. The receiving means receives the other-site video information from each of a plurality of other sites,
When the local site video information is included in a plurality of the other site video information, the difference between the local site videos included in the other site video information is calculated, and the participant is warned based on the calculation result The terminal device according to any one of claims 1 to 4, further comprising notification means for notifying the user. A communication method in which a conference is performed using information transmitted and received between the own base and a plurality of other bases connected via a network,
An imaging step of capturing an image of the local site including the participants of the local site;
A receiving step of receiving other site video information including other site video from the other site;
Judgment whether or not to synthesize the own-site video imaged in the imaging step and the other-site video included in the other-site video information received in the receiving step based on a predetermined condition Process,
If it is determined in the determination step that the local site video and the other site video are not combined, it is determined that the local site video information to be transmitted from the local site video to the other site is generated and combined. A generating step of generating the local video information from the local video and the other video;
A transmitting step of transmitting the local site video information generated by the generating step to the other site;
A communication method comprising: A communication program for a terminal device that conducts a conference using information transmitted and received between its own base and a plurality of other bases connected via a network,
An imaging step of capturing an image of the local site including the participants of the local site;
A receiving step of receiving other site video information including other site video from the other site,
A determination step of determining whether to synthesize the own site video imaged in the imaging step and the other site video based on a predetermined condition;
If it is determined by the determination step that the local site video and the other site video are not to be combined, it is determined that the local site video information to be transmitted from the local site video to the other site is generated and combined. A generating step of generating the local site video information from the local site video and the other site video;
A transmitting step of transmitting the local site video information generated by the generating step to the other site;
A communication program for a terminal device, characterized in that a computer is executed.