JP6481937B2 - Communication device for video conference - Google Patents

Description

  The present invention relates to a video conference communication apparatus that is provided at a host site and can be connected simultaneously with partner devices at a plurality of sites different from the host site.

  2. Description of the Related Art Conventionally, there has been known a video conference system capable of performing a video conference by connecting simultaneously with counterpart devices at a plurality of bases (for example, Patent Document 1). In such a video conference system, the data transmitted from each partner device is decoded for each partner device, the video data is synthesized every time decoding is completed, and the composite video data is displayed.

  In the video conference communication device, it takes a predetermined time until video data is received from all the partner devices and preparation for the start of the video conference is completed. In particular, when a partner device having a specification that can be connected only using a low bandwidth is included, the time until the video data transmitted from the partner device is received becomes longer.

JP 2014-241525 A

  However, in the conventional apparatus, recording of video data and audio data is started regardless of whether or not video data of all bases is displayed, so that some bases before the start of the video conference are started. Unnecessary video data and audio data such as chats of viewers are also recorded, and the corresponding memory capacity is wasted.

  An object of the present invention is to provide a video conference communication device that can prevent unnecessary video data and audio data from being recorded and can effectively use the memory capacity.

The video conference communication device according to the present invention is a video conference communication device that is provided at a host site and can be connected simultaneously with a partner device at a plurality of sites different from the host site, and the network device and the partner device at the plurality of sites. Received from the communication control unit in each area of the divided control screen divided into the total number of bases for the video conference and the communication control unit that receives the video data and audio data transmitted from the partner device A display control unit that sequentially displays the video data, and synthesized video data that is generated by synthesizing video data of all the bases that perform the video conference, and is synthesized voice data of all the bases that perform the video conference Receives video data from the video / sound synthesizer that generates the data and the partner devices at all bases that conduct the video conference, and the synthesized video data is generated. In timing, it employs a configuration having a, and recordings controller for starting recording of the composite video data and the synthesized speech data.

  According to the present invention, since recording of video data and audio data can be started at the timing when video data transmitted from all the counterpart devices is displayed, unnecessary video data and audio data are recorded. Can be prevented and the memory capacity can be used effectively.

1 is a block diagram showing a configuration of a video conference communication device according to Embodiment 1 of the present invention. The block diagram which shows the structure of the video / audio output control part which concerns on Embodiment 1 of this invention. State transition diagram showing state transition of the video conference system according to Embodiment 1 of the present invention The flowchart which shows operation | movement of the video conference communication apparatus which concerns on Embodiment 1 of this invention. Schematic diagram showing an example of an image displayed on the screen according to the first embodiment of the present invention. The block diagram which shows the structure of the video conference system which concerns on Embodiment 2 of this invention. The flowchart which shows the operation | movement of the video conference communication apparatus which concerns on Embodiment 2 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(Embodiment 1)
<Configuration and connection of video conference communication device>
First, the configuration and connection relationship of the video conference communication apparatus 1 according to Embodiment 1 of the present invention will be described in detail with reference to FIG.

  The video conference communication device 1 and the counterpart device (conference participation base) 2, 3, and 4 are installed at different bases. The video conference communication device 1 can be connected to and disconnected from the partner devices 2, 3, 4 via the network 5. Note that the number of counterpart devices to which the video conference communication device 1 is connected is not limited to three, and can be changed to two or four or more as required.

  The video conference communication device 1 transmits / receives audio data and video data via the network 5 after the connection is completed between the partner devices 2, 3, and 4. The network 5 is typically the Internet.

  The video conference communication device 1 is connected to a user operation input device 6, a video input device 7, an audio input device 8, and a display device 9.

  The user operation input device 6 detects an operation of selecting a site by the user, and transmits a signal including selected site information indicating the selected site to the video conference communication device 1 by wire or wirelessly. The user operation input device 6 is typically a remote control or a touch panel.

  The video input device 7 outputs video data and the like obtained by photographing a base where the video conference communication device 1 is installed (hereinafter referred to as “host base”) to the video conference communication device 1. The video data output from the video input device 7 includes video data stored on a PC, video data reproduced by a DVD player, and the like in addition to video data captured by the camera.

  The voice input device 8 outputs voice data and the like obtained by collecting the voice of the host site to the video conference communication device 1. The audio data output from the audio input device 8 includes audio data collected by a microphone, video data stored in a PC and audio data set, video data reproduced by a DVD player, and audio data set. There are data.

  The display device 9 displays the video data output from the video conference communication device 1 on a screen and outputs the audio data output from the video conference communication device 1 from a speaker (not shown).

  The video conference communication device 1 includes a user instruction receiving unit 101, a conference control unit 102, a communication control unit 103, a video / audio encoding / decoding unit 104, a video / audio synthesis unit 105, and a video / audio output. It mainly includes a control unit 106, a still image holding unit 107, a video input control unit 108, and an audio input control unit 109. The video / audio synthesis unit 105 and the video / audio output control unit 106 constitute a display control unit.

  The user instruction receiving unit 101 receives a signal transmitted from the user operation input device 6, extracts selected base information included in the received signal, and outputs the selected base information to the conference control unit 102.

  The conference control unit 102 receives data from the communication control unit 103, the video / audio encoding / decoding unit 104, and the video / audio synthesis unit 105 based on the selected base information input from the user instruction receiving unit 101. Control output timing. In addition, the conference control unit 102 outputs the selected base information input from the user instruction receiving unit 101 to the video / voice synthesis unit 105. In addition, the conference control unit 102 controls the transmission processing and call establishment processing in the communication control unit 103 based on the selected base information, and monitors whether video data has been received.

  The communication control unit 103 operates at a timing according to the control of the conference control unit 102. The communication control unit 103 establishes a call with the partner apparatuses 2, 3, 4. Then, after establishing the call, the communication control unit 103 receives the video data and audio data transmitted from each partner device 2, 3, 4 via the network 5, and sends them to the video / audio encoding / decoding unit 104. Output. In addition, after establishing the call, the communication control unit 103 transmits the video data and audio data input from the video / audio encoding / decoding unit 104 to each partner device 2, 3, 4 via the network 5. The communication control unit 103 operates according to a predetermined communication protocol. The communication protocol is typically SIP or H.264. H.323.

  The video / audio encoding / decoding unit 104 operates at a timing according to the control of the conference control unit 102. The video / audio encoding / decoding unit 104 encodes the video data input from the video / audio synthesis unit 105 and the audio data input from the audio input control unit 109 and outputs the encoded data to the communication control unit 103. Also, the video / audio encoding / decoding unit 104 decodes the video data and audio data from the counterpart devices 2, 3, and 4 input from the communication control unit 103 and outputs them to the video / audio synthesis unit 105.

  The video / audio synthesis unit 105 operates at a timing according to the control of the conference control unit 102. The video / audio synthesizing unit 105 includes video data input from the video / audio encoding / decoding unit 104, video data input from the video input control unit 108, and still image data read from the still image holding unit 107. Based on the number of bases of the selected base information input from the conference control unit 102, composite video data for generating and displaying video is generated and output to the video / audio output control unit 106. Also, the video / sound synthesis unit 105 synthesizes the synthesized voice obtained by synthesizing the voice data input from the video / audio encoding / decoding unit 104 and the voice data input from the voice input control unit 109. Data is generated and output to the video / audio output control unit 106. Further, the video / audio synthesis unit 105 outputs the video data input from the video input control unit 108 to the video / audio encoding / decoding unit 104. Details of the configuration of the video / sound synthesizer 105 will be described later.

  The video / audio output controller 106 displays the synthesized video data input from the video / audio synthesizer 105 on the screen of the display device 9 and outputs the synthesized audio data as audio from the speaker of the display device 9.

  The still image holding unit 107 holds in advance still image data for displaying a predetermined still image on the screen of the display device 9. The predetermined still image is an image for informing the current communication status with the counterpart device until the video data is received from the counterpart devices 2, 3, and 4.

  The video input control unit 108 outputs the video data input from the video input device 7 to the video / audio synthesis unit 105.

  The audio input control unit 109 outputs the audio data input from the audio input device 8 to the video / audio encoding / decoding unit 104 and the video / audio synthesis unit 105.

<Configuration of video / audio synthesis unit>
Next, the internal configuration of the video / audio synthesis unit 105 will be described with reference to FIG. As illustrated in FIG. 2, the video / audio synthesis unit 105 includes a screen layout unit 201, a duplication unit 202, and a data allocation unit 203.

  The screen layout unit 201 calculates the total value of the number of bases (partner devices) of the selected base information input from the conference control unit 102 and outputs the total value to the duplication unit 202. In addition, the screen layout unit 201 executes a layout setting process for setting a screen layout for displaying a divided screen divided into the total number of bases for the video conference in which “1” (host base) is added to the total number of bases. The screen layout information indicating the screen layout is output to the data allocation unit 203. The layout setting process includes a process of determining a screen area for displaying video data from each counterpart device.

  At the time of activation, the duplication unit 202 duplicates the same number of still image data read from the still image holding unit 107 as the total value input from the screen layout unit 201 and outputs the duplicated data to the data allocation unit 203.

  The data allocating unit 203 allocates the still image data input from the duplicating unit 202 to the screen layout of the screen layout information input from the screen layout unit 201 to generate composite video data, and sends it to the video / audio output control unit 106. Output. The data allocating unit 203 switches the still image data to the video data every time video data from the partner device 2, 3, 4 is input from the video / audio encoding / decoding unit 104, and generates new synthesized video data. Is generated. At this time, the data allocation unit 203 updates the composite video data without changing the screen layout.

<Transition of connection status of video conference communication device>
Next, the transition of the connection state of the video conference communication device 1 will be described in detail with reference to FIG.

  The video conference communication device 1 starts operating when the power is turned on.

  The video conference communication device 1 is in a non-communication state immediately after being powered on (S1).

  When the video conference communication device 1 is connected to one partner device 2 in the non-communication state (S 1), the video conference communication device 1 makes a call to the partner device 2 and receives a call from the partner device 2. One communication state (1: 1 communication state) is entered (S2). Then, in the 1: 1 communication state (S2), the video conference communication device 1 enters a non-communication state when the connection with the partner device 2 in the communication state is disconnected (S1). The video conference communication device 1 ends the operation when the power is turned off in the non-communication state (S1).

  In addition, when the video conference communication device 1 is further connected to another partner device 3 from the 1: 1 communication state (S2), by making a call to the partner device 3 and receiving a call from the partner device 3, A communication state (MCU communication state) is established with the partner device 2 and the partner device 3 at a plurality of bases (S3).

  Then, the video conference communication device 1 is in a one-to-one communication state if the connection with the partner device 3 in the communication state is disconnected (S2), and if the connection with the partner device 2 in the communication state is further disconnected. A non-communication state is entered (S1).

  Further, the video conference communication device 1 makes a communication state (MCU communication state) with the partner devices 2, 3, 4 by simultaneously transmitting to the partner devices 2, 3, 4 in a non-communication state (S 1) (S3). Then, the video conference communication device 1 is brought into a non-communication state if the connections with the partner devices 2, 3, and 4 are disconnected all at once (S1).

  As a method of transmitting all at once, a method in which a user manually inputs a destination to the video conference communication device 1 at the time of outgoing or a plurality of destination information stored in advance in the video conference communication device 1 is registered. A method of allowing the user to select a list can be considered. Further, as a method for specifying the destination, an IP address, a telephone number, an identification code, or the like can be considered.

<Operation of video conference communication device>
Next, the operation of the video conference communication apparatus 1 will be described in detail with reference to FIGS. FIG. 5 shows an example in which the video conference communication device 1 transmits all of the other devices 2, 3, and 4 at the same time.

  First, the user instruction receiving unit 101 receives a signal from the user operation input device 6, extracts selected base information included in the received signal, and outputs it to the conference control unit 102.

  The conference control unit 102 controls the communication control unit 103 so as to perform simultaneous transmission to the partner devices 2, 3, and 4 at N (N is an integer of 2 or more, N = 3 in FIG. 5) of the selected site information. Thereby, the communication control part 103 performs simultaneous transmission to the other apparatuses 2, 3, and 4 (S11). Specifically, when the selected base information is information of one list in which a plurality of destination information is recorded, the conference control unit 102 determines the number of bases based on the number of destinations of the destination information registered in the list. N can be recognized.

  Further, the video / audio synthesis unit 105 determines a screen layout to be (N + 1) divided screens based on the selected base information input from the conference control unit 102 (S12).

  Next, the video / audio synthesis unit 105 reads the still image data from the still image holding unit 107 and controls the video / audio output control unit 106 to display the read still image data on each of the divided screens ( S13). When all the sites are not responding, the video / audio synthesis unit 105 displays the video of the host site on the divided screen B1, as shown in FIG. 5A, and the remaining divided screens B2, B3, B4. Displays a still-calling still image.

  If the meeting control unit 102 satisfies the condition that there is a base that has not received video data, the conference control unit 102 sends a call to the partner device at the n-th base (n is an integer equal to or less than N) that has not received video data. The communication control unit 103 is controlled to perform. As a result, the communication control unit 103 makes a call to the partner device at the nth base (S14).

  Next, the conference control unit 102 determines whether or not the communication control unit 103 has established a call (S15).

  When the communication control unit 103 has not established a call (S15: NO), for example, because the partner device at the nth base is powered off, the conference control unit 102 displays a still image for the video / sound synthesis unit 105. Control to display. As a result, the video / audio synthesis unit 105 reads still image data from the still image holding unit 107. Then, the video / sound synthesizer 105 performs control for displaying a still image “in preparation for connection” on the divided screen B2 of the display device 9 as shown in FIG. 5B (S16). Thereafter, the flow returns to the process of S14.

  On the other hand, when the communication control unit 103 has established a call (S15: YES), the conference control unit 102 starts encoding and decoding of video data and audio data to the video / audio encoding / decoding unit 104. Then, control is performed to wait for reception of video data (S17).

  Next, the conference control unit 102 determines whether or not video data has been received from the counterpart device at the nth base (S18).

  When the conference control unit 102 does not receive video data from the counterpart device at the nth base due to packet loss due to the low bandwidth (S18: NO), the conference control unit 102 Controls the display of still images. As a result, the video / audio synthesis unit 105 reads still image data from the still image holding unit 107. Then, as shown in FIG. 5C, the video / audio synthesizing unit 105 performs control for displaying the still image “in preparation for video” on the divided screens B3 and B4 of the display device 9 (S19). Thereafter, the flow returns to the process of S14.

  On the other hand, when the conference control unit 102 receives video data from the partner device at the n-th base (S18: YES), the video data from the partner device at the n-th base is sent to the video / audio synthesis unit 105. Is controlled (S20). As a result, the video / audio synthesis unit 105 performs control to display video on the divided screens B3 and B4 of the display device 9, as shown in FIG.

  The conference control unit 102, the communication control unit 103, the video / audio encoding / decoding unit 104, and the video / audio synthesis unit 105 repeat the process of S100 (S14 to S20) until the conditions for performing the process of S100 are not satisfied.

  Thus, when video data from all bases is received, video is displayed on all the divided screens B1, B2, B3, and B4 as shown in FIG. 5 (e).

  As described above, according to the present embodiment, the divided screen divided into the total number of bases for the video conference is displayed before receiving the video data, and the received video data is sequentially displayed on each of the divided screens. Thus, it is possible to display the composite video without changing the screen layout from the start. This can prevent the screen from becoming unsightly for the viewer.

  In addition, according to the present embodiment, until the video data is received from each partner apparatus, the viewer displays the video of each site by displaying the still image informing the current communication status on each of the divided screens. You can recognize the current state until.

(Embodiment 2)
<Configuration and connection of video conference communication device>
Next, the configuration of the video conference communication device 1a according to Embodiment 2 of the present invention will be described in detail below with reference to FIG. In the video conference communication device 1a shown in FIG. 6, the same components as those in the video conference communication device 1 shown in FIG.

  The video conference communication device 1a shown in FIG. 6 adopts a configuration in which a recording / recording control unit 301 is added to the video conference communication device 1 shown in FIG. Further, in the video conference communication device 1a shown in FIG. 6, a video / audio synthesizer 105a is added instead of the video / audio synthesizer 105 of FIG.

  The video conference communication device 1 a is connected to the recording / recording device 10 in the recording / recording control unit 301.

  The recording / recording device 10 records audio data and video data input from the video conference communication device 1a (recording / recording control unit 301).

  The video / audio synthesizing unit 105a receives video data from the partner apparatuses 2, 3, and 4 at all bases and generates synthesized video data at the timing when the synthesized video data is generated. Output data. Other functions of the video / audio synthesis unit 105 a are the same as those of the video / audio synthesis unit 105.

  The recording / recording control unit 301 causes the recording / recording apparatus 10 to record the synthesized video data and synthesized voice data input from the video / sound synthesis unit 105a.

<Operation of video conference communication device>
Next, the operation of the video conference communication device 1a will be described in detail with reference to FIG. In FIG. 7, steps that are the same as those in FIG. 2 are given the same reference numerals, and descriptions thereof are omitted.

  FIG. 7 employs a configuration in which step S21 is added after step S100, as compared with FIG.

  At the timing when the processing up to S100 is completed and the synthesized video data of the video data received from the counterpart devices 2, 3, and 4 at all bases is generated, the recording / recording control unit 301 starts from the video / sound synthesis unit 105a. Recording of the input synthesized video data and synthesized voice data is started by the recording / recording apparatus 10 (S21).

  As described above, according to the present embodiment, since the recording of the composite video data and the composite audio data can be started at the timing when all the videos of the plurality of bases are displayed, unnecessary video data, audio Data can be prevented from being recorded, and the memory capacity can be effectively utilized.

  In the present embodiment, as in the first embodiment, the video data is not limited to the case where the video data is displayed on the divided screen of the display device every time the video data is received from the partner device without changing the screen layout. The above-described effect can be obtained even when the video data is synthesized every time it is received and the synthesized video data is displayed while changing the layout of the screen every time the video data is received.

  In this embodiment, the timing of starting recording of video data, such as the time when decoding of video data of all bases is completed or the time when processing in the layout setting process is completed to a predetermined ratio (for example, 80%). Recording of audio data may be started a little before. In the present embodiment, the user may arbitrarily set the timing for starting recording of audio data.

  The present invention is not limited to the above-described embodiments in terms of the type, arrangement, number, etc. of the members, and departs from the gist of the invention, such as appropriately replacing the constituent elements with those having the same operational effects. Of course, it can be appropriately changed within the range not to be.

  Specifically, in Embodiment 1 and Embodiment 2 described above, the case where a still image is displayed before video is displayed has been described, but the present invention is not limited to this, and characters such as messages other than still images are displayed. Information may be displayed.

  In the first embodiment and the second embodiment, the case where the divided screens divided into the same size are displayed has been described. However, the present invention is not limited to this, and if the screen layout is not changed, different sizes are displayed. You may make it display the divided screen divided | segmented further or the divided screen partially overlapped. In the present invention, a split screen may be displayed on a part of the screen, and an area to which no split screen is assigned may be a black screen, or a still image may be displayed on an area to which no split screen is assigned on the display device screen. Also good.

  In the first embodiment and the second embodiment, the still image is changed in the order of “during transmission”, “in preparation for connection”, “in preparation for video”, but the present invention is not limited to this, and the video is not limited to this. All the same still images may be displayed until they are displayed. Further, in the present invention, the content to be notified as a still image may be a still image other than the above as long as information indicating the current communication status is included.

  The present invention is suitable for use in a video conference communication apparatus that is provided at a host site and can be connected simultaneously with partner devices at a plurality of sites different from the host site.

1, 1a Communication device for video conference 2, 3, 4 Partner device (conference participation base)
6 User operation input device 7 Video input device 8 Audio input device 9 Display device 10 Recording / recording device 101 User instruction receiving unit 102 Conference control unit 103 Communication control unit 104 Video / audio encoding / decoding unit 105, 105a Video / audio synthesis Unit 106 video / audio output control unit 107 still image holding unit 108 video input control unit 109 audio input control unit 201 screen layout unit 202 replication unit 203 data allocation unit 301 recording / recording control unit

Claims (2)

A teleconferencing communication device that is provided at a host site and can be connected simultaneously with a partner device at a plurality of sites different from the host site,
A communication control unit that connects the partner devices of the plurality of bases via a network and receives video data and audio data transmitted from the partner device;
A display control unit for sequentially displaying the video data received from the communication control unit in each area of the divided screen divided into the total number of bases for the video conference;
A video / sound synthesizer that generates synthesized video data by synthesizing video data of all bases that perform the video conference, and generates synthesized audio data that synthesizes audio data of all bases that perform the video conference;
A recording / recording control unit that receives video data from partner devices at all bases that conduct the video conference and starts recording the synthesized video data and the synthesized audio data at a timing when the synthesized video data is generated When,
A teleconferencing communication apparatus comprising: The display control unit
Until the video data is received from the counterpart device, a still image that informs the current communication status with the counterpart device is displayed in each area of the divided screen.
The video conference communication apparatus according to claim 1.

Images