WO2007080788A1

WO2007080788A1 - Teleconference control device and teleconference control method

Info

Publication number: WO2007080788A1
Application number: PCT/JP2006/326033
Authority: WO
Inventors: Yoshimasa Honda
Original assignee: Matsushita Electric Industrial Co., Ltd.
Priority date: 2006-01-12
Filing date: 2006-12-27
Publication date: 2007-07-19
Also published as: JP2009076952A

Abstract

Provided is a device capable of continuously performing transmission with a delay amount based on the conference state while synchronizing audio with video even in a best effort type network. A teleconference control device controls a teleconference device (101) communicating video data and audio data via a transmission path. The teleconference control device includes: a low delay priority decision unit (105) for deciding the low delay priority indicating the degree for suppressing the delay of the video data and the audio data to a higher value as the frequency of switching between transmission and reception of the audio data as a sound detected by the teleconference device (101) increases; a delay amount decision unit (106) for deciding a smaller delay amount as the low delay priority increases; and a parameter control unit (107) for deciding an encoding parameter or packet priority used in the teleconference device (101) in accordance with the decided delay amount.

Description

Specification

Video conference control device and video conference control method

Technical field

The present invention relates to a τν conference control device that controls a τν conference device.

Background art

[0002] In recent years, ADSL (Asymmetric Digital Subscriber Line) and optical fiber networks have rapidly spread and low-cost and high-speed Internet connections are becoming available. In addition, by using such a low-cost high-speed Internet, video and audio data (hereinafter referred to as “video / audio data”) is transmitted bidirectionally between multiple remote sites. It has become possible to build

[0003] However, the IP network represented by the current Internet is a best-f-auto type network that guarantees nothing about the effective bandwidth in which data can be transmitted without data loss. Therefore, for example, if data in a band that exceeds the effective bandwidth that can be transmitted is transmitted, a congestion state occurs in the network, a transmission delay of video and audio data occurs, and a congestion state that exceeds the noise in the network device Then, data loss occurs.

[0004] In general, in video conferences, in order to facilitate smooth communication, it is very important to transmit video and audio with a low delay without interruption. However, as mentioned above, in a low-cost best F auto network, the bandwidth is not guaranteed, so it is very difficult to always realize low-delay transmission. On the other hand, in order to realize low-latency transmission, it is possible to use a dedicated line service, which is a bandwidth-guaranteed network that is not the best F-automatic network, but the monthly fee is approximately 10 million yen. It is necessary and very expensive.

[0005] As a conventional method for solving this problem, for example, in Patent Document 1, a video conference is synchronized by inserting a delay into an audio in accordance with a delay of an image having a large bandwidth, usually in a video conference. When both voices are input, the voice delay is reduced by stopping the voice delay insertion to improve the response performance. FIG. 9 is a diagram showing a conventional video conference apparatus. In FIG. 9, the video conference apparatus 9 10 includes a video code Z decoding unit 901 that performs video code Z decoding, and audio coding Z decoding that performs audio code Z decoding. 902, transmission delay circuit 903 for delaying voice input, transmission switcher 904 for switching delay input, and voice input on the transmission / reception side are monitored, and when both are input simultaneously, the delay time is shortened. An audio input monitoring unit 905 that performs reception switching, a reception delay circuit 907 that performs delay input on received audio, and a multiplexing Z separation unit that performs video and audio multiplexing Z separation processing 908 And is connected to the multipoint connection control device 909, and performs two-way video and audio communication between the TV conference devices.

[0007] As described above, the TV conference device in Patent Document 1 monitors the audio input on the transmission / reception side in the audio input monitoring unit 905, and shortens the delay time when both are input simultaneously. Thus, low-delay transmission of voice is enabled.

Patent Document 1: JP-A-8-317362

Disclosure of the invention

Problems to be solved by the invention

However, Patent Document 1 has a problem in that only audio data is transmitted with low delay, so that the video data is displayed without being synchronized with the audio data.

[0009] For example, after a speaker's voice is played back, a speech video is played back with a delay.

, Unnatural video 'sound playback will be a big discomfort in video conferences

. Also, for example, when a meeting is held while pointing to a model, synchronization between voice and movement pointing to an object is essential for smooth communication.

In view of such points, the present invention determines the degree of necessity of low-delay transmission of video / audio data using the frequency of audio data transmission / reception as a low-delay priority, By controlling the video / audio encoding parameters or transmission parameters so that the lower the delay, the higher the priority, the synchronization between audio and video is always possible even in a best-f-automatic network. The purpose is to provide a device capable of seamless transmission with a delay amount according to the conference state.

Means for solving the problem [0011] The TV conference control device of the present invention is a TV conference control device that controls a TV conference device that transmits video data and audio data through a transmission line, and is a sound that is detected by the TV conference device. A low-delay priority determination means for determining a low-delay priority indicating a degree to which delay of video data and audio data should be suppressed as the frequency of switching between transmission and reception of certain audio data increases, and the low-delay priority Higher !, smaller! /, Delay amount determining means for determining the delay amount, and determining the parameter or packet priority of the code key used in the video conference apparatus according to the determined delay amount Parameter control means.

[0012] According to this configuration, the higher the frequency of switching between transmission and reception of audio data, the higher the low delay priority of audio and video is set, and the delay amount corresponding thereto is determined. Furthermore, the code key parameter or the packet priority is determined according to the delay amount.

[0013] The parameter of the code key is a parameter used for the compression code key, and thereby the amount of data after the code key and the data amount used for the decoding key can be controlled. Data with a high packet priority is processed with priority over data with a low packet priority by a router on the transmission path, and is transferred first on the transmission path.

[0014] Therefore, by determining the code key parameter or packet priority, it is possible to communicate video and audio with a lower delay for a meeting that is actively discussed while synchronizing audio and video. Become.

[0015] Preferably, the low-delay priority determining unit measures the frequency based on the number of times within a predetermined time when transmission and reception of voice data, which is sound detected by the TV conference device, are switched.

[0016] According to this configuration, the frequency of switching between transmission and reception of audio data within a predetermined time is used as an index of the speed of switching between transmission and reception of audio data. Therefore, it is possible to determine the low delay priority with a simple process of monitoring the transmission / reception timing.

[0017] More preferably, the low delay priority determination means compares the frequency with a threshold, and when the frequency is greater than the threshold, the low delay priority is higher than when the frequency is equal to or less than the threshold. To decide.

[0018] According to this configuration, when the low delay priority is determined, the audio data transmission 'reception frequency and The low delay priority can be determined by a simple process of comparing preset threshold values.

[0019] More preferably, the low-delay priority determining unit measures the frequency based on a difference between a transmission time and a reception time of audio data detected by the TV conference device, and a difference between the transmission time and the reception time. The lower the priority, the higher the low delay priority is determined.

[0020] According to this configuration, the difference between the transmission time and the reception time of the audio data is used as an index of the speed of switching between transmission and reception of the audio data. Therefore, it is possible to determine the low-latency priority with a simple process that monitors the transmission / reception time and the difference.

[0021] More preferably, the low delay priority determination means compares the difference between the transmission time and the reception time with a threshold value, and when the difference between the transmission time and the reception time is smaller than the threshold value, The low delay priority that is higher than when the difference between the time and the reception time is equal to or greater than the threshold is determined.

[0022] According to this configuration, when the low delay priority is determined, it is possible to determine the low delay priority by a simple process of comparing a difference between audio data transmission / reception times and a preset threshold value. .

[0023] More preferably, the parameter control means decodes the video data and audio data in a TV conference device that receives the video data and audio data as the compression code amount parameter. Determine the receive buffer capacity.

[0024] According to this configuration, by dynamically changing the buffer amount of the video / audio decoding key, the smaller the reception buffer capacity, the smaller the reception waiting time. Therefore, the delay is less than the requested delay value. It becomes possible to control the amount.

[0025] More preferably, the parameter control means determines a maximum generated code amount for encoding the video data and audio data in the video conference device as the compression code amount parameter.

[0026] Since the maximum generated code amount of video / audio encoding is smaller, the transmission delay becomes smaller and low-delay transmission is performed, so that it can be controlled to be equal to or less than the requested delay amount.

[0027] Further, the TV conference control device of the present invention is a TV conference control device that controls a plurality of TV conference devices that communicate video data and audio data of a plurality of conferences through a common transmission path. The frequency of switching between transmission and reception of voice data that is detected by each TV conference device is high! The lower the delay priority determining means for determining the higher the lower delay priority indicating the degree to which the delay of the video data and the audio data should be suppressed, and the higher the lower delay priority, the more used for each conference. A delay amount determining means for determining a small delay amount for the TV conference device group, and parameter control for determining a parameter of a code or a packet priority used in the TV conference device according to the determined delay amount Means.

[0028] According to this configuration, since the low delay priority is determined as compared with other TV conferences, the higher the request for low delay among the plurality of TV conferences using a common transmission path, the lower the delay. It becomes possible to. In addition, it is possible to communicate video and audio with a lower delay in a conference where active discussions are made while synchronizing audio and video.

It should be noted that the present invention can be realized not only as such a TV conference control apparatus but also as an integrated circuit and a TV conference apparatus including characteristic means included in the TV conference control apparatus. The present invention can also be realized as a TV conference system including a TV conference control device and a TV conference device. Furthermore, the present invention can also be realized by a TV conference control method in which characteristic means included in the TV conference control apparatus is a step.

The invention's effect

[0030] According to the present invention, the parameter for controlling the delay amount common to video and audio is determined according to the speed of switching between transmission and reception of audio data. For this reason, as the need for low delay is high, such as in TV conferences where active discussions are taking place, the delay is low and the video and audio are synchronized, so the conference can proceed smoothly. become.

Brief Description of Drawings

FIG. 1 is a block diagram showing a configuration of a video conference apparatus according to Embodiment 1.

FIG. 2 is a flowchart showing processing executed when the TV conference apparatus according to Embodiment 1 performs a TV conference.

FIG. 3 is a flowchart showing details of delay control processing S204 shown in FIG.

[FIG. 4] FIG. 4 is a conceptual diagram showing audio data transmission'reception frequency according to the first embodiment. FIG. 5 is a block diagram showing a configuration of a video conference system according to Embodiment 2 of the present invention.

FIG. 6 is a diagram showing a relationship between a plurality of video conference apparatuses and a delay control server according to Embodiment 2.

FIG. 7 is a flowchart showing processing executed by the video conference apparatus according to Embodiment 2.

FIG. 8 is a flowchart showing processing executed by the delay control server according to the second embodiment.

FIG. 9 is a diagram showing a configuration of a conventional video conference terminal device.

Explanation of symbols

101, 501, 601, 602, 603, 604 Video conferencing equipment

102 Video / audio input section

103 Video / audio encoding unit

104 Transmitter

105 Low-latency priority determination unit

106 Delay amount determination unit

107, 503 Parameter control section

108 Video / audio output

109 Video audio decoder

110 Receiver

111 Transmission line

502 transceiver

504, 607 Delay control server

505 Low latency priority receiver

506 Delay amount determination unit

507 Delay amount transmitter

901 Video encoding / decoding unit

902 Speech coding Z decoding unit 903 Transmission delay circuit

904 Transmitter switch

905 Voice input monitoring unit

906 Reception switching unit

907 Receive delay circuit

908 Multiplex Z separator

909 Multipoint connection controller

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Embodiment 1]

FIG. 1 is a block diagram showing the configuration of the video conference apparatus according to Embodiment 1 of the present invention. The video conference apparatus of this embodiment is placed under a conference participant and communicates audio and video. Each of the video conference apparatuses according to the present embodiment dynamically changes the delay amount from the input of the other party's audio and video to the output of the other audio and video to the other TV conference apparatus according to the state of the conference. A video conference control device is provided.

A video conference apparatus 101 shown in FIG. 1 includes a video / audio input unit 102 connected to a camera and a microphone and inputs video and audio, and video data and audio data (hereinafter referred to as “audio / video data”). A video / audio code unit 103 that performs compression coding, a transmission unit 104 that is connected to the transmission path 111 and transmits encoded video / audio data, and a low-delay priority that determines low delay priority using video / audio input. A delay priority determination unit 105, a delay amount determination unit 106 that determines a delay amount using a low delay priority, a parameter control unit 107 that changes a coding parameter using the delay amount, and a transmission path 111. A receiving unit 110 that receives encoded video / audio data transmitted from a video conference terminal that is a connected communication partner, a video / audio decoding unit 109 that decodes the encoded video / audio data, and a monitor And connected to the speaker, And a video / audio output unit 108 for outputting the decoded video / audio data to a monitor and a speaker.

Here, the code key parameter is a parameter used for the code key in the video / audio code key unit 103 and the decoding key in the video / audio decoding key unit 109. Same It is.

[0037] The video / audio input unit 102 outputs uncompressed video data and audio data input by the camera and microphone power to the video / audio encoding unit 103 and the low delay priority determination unit 105 in units of frames.

[0038] The video / audio encoding unit 103 compresses the video / audio data input from the video / audio input unit 102 using a coding parameter input from the parameter control unit 107, such as MPEG-2. The encoded video / audio data is output to the transmission unit 104.

[0039] Transmitting section 104 uses another transmission parameter input from parameter control section 107 for the encoded video / audio data input from video / audio encoding section 103 to another TV conference apparatus that is a communication partner. Send to.

[0040] The low-delay priority determination unit 105 detects a voiced portion included in the audio data from the video / audio data input from the video / audio input unit 102, and the low-delay priority is determined based on the detected voiced portion. The low delay priority is output to the delay amount determination unit 106.

Here, the voiced portion is detected by, for example, whether or not the sound volume included in the audio data is greater than or equal to a threshold value. A method for determining the low delay priority will be described later.

The delay amount determination unit 106 determines a delay amount using the low delay priority input from the low delay priority determination unit 105, and outputs the delay amount to the parameter control unit 107.

The meter control unit 107 changes the code key parameter using the delay amount input from the delay amount determination unit 106, and outputs the code key parameter to the video / audio code key unit 103.

Note that the parameter control unit 107 may change the transmission parameter instead of the sign key parameter, and output the transmission parameter to the transmission unit 104. This transmission parameter is, for example, packet priority.

[0045] Receiving section 110 receives the encoded video / audio data transmitted from the video conference terminal as the communication partner through transmission path 111, and outputs the received data to video / audio decoding section 109.

[0046] The video / audio decoding unit 109 performs a decoding process on the encoded video / audio data input from the receiving unit 110 in accordance with the encoded encoding method. The audio data is output to the audio / video output unit 108, and the audio data is assigned a low delay priority. Output to fixed part 105.

The video / audio output unit 108 outputs the video / audio data input from the video / audio decoding unit 109 to the connected monitor and speaker.

Next, the operation of the video conference apparatus 101 according to the present embodiment will be described using the flowcharts shown in FIG. 2 and FIG.

FIG. 2 is a flowchart of processing executed when the TV conference apparatus 101 according to Embodiment 1 performs a TV conference. Each process shown in FIG. 2 is stored as a control program in a storage device (not shown) such as a ROM or flash memory of the TV conference apparatus 101, and is controlled by the CPU (not shown).

[0050] <Step S201: Video / audio input processing>

First, the video / audio input unit 102 inputs uncompressed video data and audio data in units of frames from the connected camera and microphone, and outputs the video data and audio data to the video / audio encoding unit 103 for audio. The data is output to the low delay priority determination unit 105 (step S201).

[0051] <Step S202: Data reception processing>

The receiving unit 110 receives the encoded video / audio data transmitted from the video conference terminal as a communication partner through the transmission path 111, and outputs the received encoded data to the video / audio decoding unit 109 (step S1). S 202).

[0052] <Step S203: Video 'Audio Decoding and Display Output Processing>

The video / audio decoding unit 109 performs decoding processing on the encoded video / audio data input from the receiving unit 110 in accordance with the encoded encoding method, and the decoded video / audio data. Is output to the video / audio output unit 108, and the audio data is output to the low delay priority determination unit 105. The video / audio output unit 108 connects the video / audio data input from the video / audio decoding unit 109 to the video / audio output unit 108. Is output to the monitor and speaker (step S203).

[0053] <Step S204: Delay control processing>

The low delay priority determination unit 105 determines the low delay priority, the delay amount determination unit 106 determines the delay amount according to the determined low delay priority, and the parameter control unit 107 uses the delay amount to code.匕 Determine the parameters (step S204). Details of this delay control process This will be described with reference to the drawings.

[0054] Here, parameter control section 107 may determine the transmission parameter using the delay amount instead of the sign key parameter.

FIG. 3 is a flowchart showing details of the delay control process (step S204) shown in FIG.

[0056] <Step S301: Voice packet interval calculation process>

First, the delay priority determination unit 105 transmits audio data using the transmission audio data input from the video / audio input unit 102 and the reception audio stream input from the video / audio decoding unit 109. And the frequency of reception is calculated (step S301).

Here, the frequency of transmission and reception of voice data represents the degree of conversation activity. In other words, the higher the frequency of voice data transmission and reception, the more active the discussion is, and the greater the delay effect, the lower the delay transmission is required.

[0058] [Equation 1]

N (t) = N s (t) + N r (t) (Equation 1)

(Equation 1) shows an example of a calculation formula for the frequency of transmission and reception of audio data. In (Equation 1), N (t) represents the frequency of audio data transmission and reception at time t, Ns (t) represents the number of times audio data was transmitted during the past T hours, and Nr (t) Indicates the number of audio data received during the past T hours from time t.

Here, it is assumed that the number of transmissions and the number of receptions are sound data with sound, and the delay priority determination unit 105 performs sound determination. In other words, the frequency of transmission and reception of such voice data is an example of the frequency at which transmission and reception of voice data that is sound is switched, and transmission and reception of voice data that is sound during a certain period of time. It is indicated by the number of times to switch.

[0061] (Equation 1) is an example of a method for determining the transmission / reception frequency of audio data, and any method can be used as long as it is a calculation method representing the transmission / reception frequency of audio data.

[0062] Strictly speaking, the transmission data and the reception data may be out of time at the time t due to the effect of the transmission delay. Therefore, the transmission delay at the time t of the transmission data in consideration of the transmission delay amount. It is also possible to calculate by adding.

FIG. 4 is a conceptual diagram showing the frequency of audio data transmission / reception according to the first embodiment. With reference to this figure, a method for calculating the frequency of audio data transmission / reception between two TV conference devices will be described.

[0064] In Fig. 4, a hatched section 401 written as S indicates a voice data transmission state, and a white section 402 written as a scale indicates a voice data reception state. However, if there is an input for both transmission and reception, priority is given to the louder volume.

[0065] When the transmission / reception frequency of audio data is calculated according to (Equation 1), for example, at time tl, N (tl)

= 4 + 4 = 8, and at time t2, N (t2) = 2 + 2 = 4, and at time t2, the frequency of audio data transmission / reception is higher.

[0066] That is, in the state of Fig. 4 (a) at time t2, voice data is transmitted and received at a higher frequency, and it can be said that a live discussion is being conducted as a TV conference. Therefore, Fig. 4 (a) is more affected by delay compared to Fig. 4 (b), and the need for low-delay transmission is high, and it is necessary to increase the low-delay priority.

Next, the low delay priority determination unit 105 calculates the audio data transmission / reception frequency, and then determines the low delay priority to be higher as the transmission / reception frequency is higher. For example, the low delay priority P (t) is determined as in (Equation 2) (step S302).

[0068] [Equation 2]

0 (when N (t) is TH1)

PMAXxfN (t) -TH 1)

P (t) (when TH 1 <N (t) <TH2)

(TH2-TH 1)

PMAX (when N (t)> TH2)

[0069] In (Equation 2), P (t) is the low delay priority at time t, N (t) is the frequency of audio data transmission and reception at time t, and TH1 and TH2 are It is a predetermined threshold (where TH 1 <TH2), and PMAX is a predetermined maximum priority. (Equation 2) is an example of a method for determining the low delay priority. Any method can be used as long as the delay priority increases as the frequency of audio data transmission / reception increases. [0070] <Step S303: Delay Amount Calculation Processing>

Next, the delay amount determination unit 106 calculates a delay amount using the low delay priority calculated as described above (step S303).

Here, (Equation 3) is an example showing a method of calculating the delay amount. In (Equation 3), Delay

(t) is a delay amount at time t, DMAX is a predetermined maximum delay amount, and P (t) is a low delay priority at time t.

[0072] [Equation 3] t) = 0)

D e l a y (t)

(When (t) ≠ 0)

P (t) + l

[0073] According to (Equation 3), the delay amount determination unit 106 determines the delay amount so that the value decreases as the low delay priority increases. Note that (Equation 3) is an example of a method for determining the delay amount, and any method can be used as long as the value becomes lower as the low delay priority is higher.

[0074] Step S304: Parameter calculation 'Update process>

Further, parameter control section 107 calculates an encoding parameter using the delay amount calculated in the delay amount calculation process (step S303), and outputs the calculated encoding parameter to video / audio encoding section 103. .

Here, a case will be described in which the fluctuation width of the bit rate (the amount of code generated per unit time), which is one of the sign key parameters, is targeted as a parameter to be calculated. In order to perform low-delay transmission, it is important to perform fixed bit rate control in the video code. In other words, the target code amount per frame, which is uniquely determined from the bit rate and frame rate (screen update frequency), is set as the target for video code, and the generated code amount per frame is kept below the target. is there.

[0076] Here, the maximum generated code amount is defined as the maximum generated code amount per frame. For example, if the generated code amount per frame is N times the average code amount, it takes N times the normal amount of data to transmit the encoded data for that frame because the data amount is N times. become. Therefore, if the frame rate is 30fps, the delay amount is NZ30 (second ).

[0077] [Equation 4]

BITS MA X = De ^{1 ay} (t) x BITLATE

1 0 0 0

[0078] (Expression 4) is a mathematical expression showing a method of calculating the maximum generated code amount. In (Equation 4), BITS MAX is the maximum generated code amount (bits) per frame, Delay (t) is the delay amount (ms) at time t, and BITRATE is the video code This is the bit rate (bitsZ seconds), which is the amount of generated code per second.

In this embodiment, an example has been shown in which the maximum amount of generated code is calculated for the delay amount power, but as a parameter, the TOS (TYPE OF SE RVICE) value, which is the packet priority in TCPZIP communication, is changed. Any parameter can be used as long as the delay amount can be controlled relatively.

Note that the delay control process described above is performed for each frame, but can be performed at predetermined intervals to reduce the processing amount.

[0081] Note that the parameter control unit 107 calculates a transmission parameter using the delay amount calculated in the delay amount calculation process (step S303) instead of calculating the encoding parameter, and calculates the calculated transmission parameter. You may output to the transmission part 104. FIG.

When using the packet priority, the transmission side increases the packet priority included in the packet as the low delay priority increases. Since routers on the transmission path preferentially process packets with higher packet priority, packets with higher packet priority reach the TV conference device on the receiving side earlier. For this reason, packets with a higher packet priority are processed earlier at the router on the transmission path than packets with a lower packet priority, and low-delay transmission is realized.

[0083] <Step S205: Video / Audio Coding and Transmission Processing>

The audio / video encoding unit 103 performs compression encoding such as MPEG-2 on the audio / video data input from the audio / video input unit 102 using the encoding parameters input from the noram control unit 107, The encoded video / audio data is output to transmitting section 104 (step S 205). [0084] For example, when the maximum generated code amount is input from the parameter control unit 107 in the video / audio code input unit 103, the generated code amount per frame is set to be equal to or less than the input value and per unit time. The bit rate is controlled so that the amount of generated codes is less than a certain value.

However, the encoding method in the video / audio encoding unit 103 is not limited to MPEG, and any encoding method can be used.

[0086] Furthermore, the transmission unit 104 uses the predetermined transmission parameter for the encoded video / audio data input from the video / audio code unit 103, and transmits to another TV as a communication partner. Data is transmitted to the conference device. For example, here, IPZUDPZRTP is used as the transmission method, but any method can be used as long as it is a method capable of transmitting video and audio data through the transmission path.

[0087] When parameter control unit 107 calculates the packet priority instead of the sign key parameter, transmission unit 104 uses another transmission parameter input from parameter control unit 107 as another TV as a communication partner. Data is transmitted to the conference device.

[0088] <Step S206: End determination>

The video / audio input unit 102 determines that the process has ended when the input of the video / audio data is completed, or when a preset time has elapsed (Yes in step S206), and ends the process. In other cases (No in step S206), the video conference apparatus 101 continues the video / audio input process (step S201) and the data reception process (step S202).

The above is the description of the processing executed by the TV conference device 101 of the present embodiment.

[0090] As described above, in the present embodiment, the low delay priority determination unit 105 determines a low delay priority, the delay amount determination unit 106 determines a delay amount using the low delay priority, The parameter control unit 107 changes the video / audio code parameter or the transmission parameter according to the delay amount.

[0091] With this, it is possible to dynamically determine the low delay priority during the TV conference, change the encoding parameter or the transmission parameter, and control the delay amount. The amount of delay can also be controlled by dynamically determining the low delay priority during a video conference and changing the transmission parameters. Further, in the present embodiment, the low-delay priority determination unit 105 uses the frequency of transmission and reception of voice data that is voiced to transmit voice data. Determine the degree high.

[0093] This makes it possible to increase the priority of low-delay as the state of active discussion that audio transmission / reception frequency is high and low-latency is important is important. is there.

Further, in the present embodiment, the low delay priority determination unit 105 uses the frequency of transmission and reception of voice data that is voiced to transmit voice data. When determining a high degree, the low delay priority is determined by comparing with a preset threshold.

According to this configuration, when determining the low delay priority, it is possible to determine the low delay priority with a simple process of comparing the audio data transmission frequency with a preset threshold value. is there.

In this embodiment, the low delay priority determination unit 105 determines the low delay priority using the audio data transmission / reception frequency. However, the transmission time of the audio data and the voice data of the voice data are determined. The lower delay priority may be determined as the difference between the transmission time and the reception time is smaller! /.

According to this, it is possible to reduce the delay amount in a state where the difference between the transmission time and the reception time is small, that is, in a state where the speech timings overlap each other in the video conference.

[0098] In the present embodiment, low delay priority determination section 105 determines low delay priority by comparing the frequency of transmission and reception of voice data that is voiced with a preset threshold value. However, the difference between the transmission time and the reception time of the voice data may be compared with a preset threshold value, and the lower the delay priority, the higher the low delay priority may be determined.

According to this, when determining the low delay priority, it is possible to determine the low delay priority by simple threshold processing.

[0100] Also, in the present embodiment, the delay amount determination unit 106 performs a predetermined threshold process using the determined low delay priority, and the delay amount is set to a smaller value as the priority is higher. decide. [0101] According to this configuration, since the delay amount is determined by threshold processing using the low delay priority, the delay amount can be determined by simple processing.

[0102] Also, in the present embodiment, parameter control section 107 changes the maximum generated code amount of video / audio encoding to be small so as to be equal to or less than the determined delay amount.

[0103] According to this, as the maximum generated code amount of the video / audio encoding is smaller! /, The maximum value of the transmission delay can be made less than the delay amount, and the low-delay transmission with less than the designated delay amount. Can be performed.

In the present embodiment, parameter control unit 107 may change the buffer capacity of the video / audio decoding key so as to be equal to or less than the determined delay amount.

[0105] According to this, it is possible to reduce the maximum amount of delay waiting for reception by dynamically reducing the buffer capacity set on the receiving side, and therefore control it to be equal to or less than the specified delay value. Is possible.

In the present embodiment, parameter control section 107 may set the packet priority of transmission data higher as the delay amount is smaller in accordance with the determined delay amount.

According to this, since data with higher packet priority is transferred earlier in the transmission path, it is possible to realize low-delay transmission as the delay amount is set smaller.

[Embodiment 2]

In the present embodiment, an example will be described in which the delay control servers connected via transmission lines determine the delay amounts of a plurality of video conference apparatuses in an integrated manner.

FIG. 5 is a block diagram showing a configuration of the video conference system according to Embodiment 2 of the present invention. The TV conference system according to the present embodiment is placed under a conference participant and communicates with a plurality of TV conference devices that communicate audio and video, and a delay in a TV conference using the plurality of TV conference devices. A delay control server 504 for determining and controlling the quantity. The plurality of video conference apparatuses and the delay control server 504 are configured by being connected through a common transmission path 111. A video conference apparatus 501 shown in the figure shows one of a plurality of video conference apparatuses connected to the transmission path 111.

[0110] The video conference apparatus 501 shown in Fig. 5 includes a video / audio input unit 102 that is connected to a camera and a microphone and receives video / audio, a video / audio encoding unit 103 that encodes video / audio, and a transmission A transmission unit 104 connected to the channel 111 for transmitting the encoded video / audio data, a low delay priority determining unit 105 for determining the low delay priority using the video / audio input, and the low delay priority A transmission / reception unit 502 that transmits to the delay control server 504 and receives the delay amount, a parameter control unit 503 that changes the sign key parameter using the delay amount, and a video conference that is connected to the transmission path 111 and is the communication partner A receiving unit 110 that receives the encoded video / audio data transmitted from the terminal camera, a video / audio decoding unit 109 that decodes the encoded video / audio data, and a monitor and speaker connected to the decoding unit And a video / audio output unit 108 for outputting the video / audio data thus output to a monitor and a speaker.

Further, the delay control server 504 includes a low delay priority receiving unit 505 that receives a low delay priority through a transmission line, a delay amount determining unit 506 that determines a delay amount using the low delay priority, a delay A delay amount transmitting unit 507 for transmitting the amount.

That is, in this embodiment, the delay amount determination unit 506 provided in the delay control server 504, the low delay priority determination unit 105 and the parameter control unit 503 provided in the TV conference device 501 correspond to the TV conference control device. To do. Further, the communication between the delay control server 504 and the TV conference device 501 is an example realized by the low delay priority receiving unit 505, the delay amount transmitting unit 507, and the transmitting / receiving unit 502.

In FIG. 5, the processing units having the same operation contents as those of the first embodiment are given the same numbers as those in FIG. 1, and the description of the operations is omitted. Therefore, the processing units that differ in operation content from Embodiment 1 are the transmission / reception unit 502, the parameter control unit 503, and the delay control server 504 in the TV conference device 501.

The transmission / reception unit 502 transmits the low delay priority input from the low delay priority determination unit 105 to the delay control server 504 through the transmission path 111.

The low delay priority receiving unit 505 receives the low delay priority transmitted from the transmission / reception unit 502 through the transmission path 111 and outputs the low delay priority to the delay amount determination unit 106. The delay amount determination unit 106 determines a delay amount using the low delay priority input from the low delay priority reception unit 505, and outputs the delay amount to the delay amount transmission unit 507. The delay amount transmitting unit 507 transmits the delay amount input from the delay amount determining unit 506 to the TV conference device 501 through the transmission path 111.

[0116] FIG. 6 shows the relationship between a plurality of video conference apparatuses and a delay control server according to Embodiment 2. FIG. For example, the six TV conference devices shown in the figure each have three conferences separately, and the delay control server 607 controls the delay amount of these TV conference devices. In FIG. 6, TV conference apparatuses 601 to 606 have the same function as the TV conference apparatus 501 in FIG. 5, and the delay control server 607 has the same function as the delay control server 504 in FIG.

Next, operations of the TV conference device 501 and the delay control server 504 having the above-described configurations will be described using the flowcharts shown in FIG. 7 and FIG.

[0118] The operations of the flowcharts shown in FIGS. 7 and 8 are stored as control programs in a storage device (for example, a ROM or a flash memory), not shown in the video conference device 501 and the delay control server 504. Controlled by a CPU (not shown).

FIG. 7 is a flowchart showing processing executed by the video conference apparatus 501 according to Embodiment 2. In FIG. 7, steps having the same processing contents as those in the first embodiment are given the same numbers as those in FIG. 2 and will not be described.

That is, in the present embodiment, the video conference apparatus 501 performs the video / audio input process (step S201), the data reception process (step S202), and the video / audio decoding output process (step S203) of the first embodiment. ) And then the following processing is executed.

[0121] <Step S701: Low-latency priority calculation processing>

The low delay priority determination unit 105 calculates the low delay priority using the audio data transmission / reception frequency, and outputs the low delay priority to the transmission / reception unit 502 (step S701). Here, the low-delay priority determination unit 105 obtains the low-delay priority through the same processes as the audio data transmission / reception frequency calculation process (step S301) and the low-delay priority calculation process (step S302) of the first embodiment. calculate.

[0122] <Step S702: Low delay priority transmission processing>

Next, the transmission / reception unit 502 transmits the low delay priority input from the low delay priority determination unit 105 to the delay control server 504 through the transmission path 111 (step S702).

[0123] <Step S703: Receive delay amount, update parameter>

Next, the transmission / reception unit 502 receives the delay amount transmitted from the delay control server 504 through the transmission path 111, and the parameter control unit 503 further performs the delay amount calculation process of the first embodiment. (Step S303) and parameter calculation / update processing By executing the same processing as (Step S304), the delay amount is calculated, the encoding parameter is determined, and the encoding parameter is input to the video / audio encoding unit 103. Output. Here, parameter control section 503 may determine a transmission parameter based on the calculated delay amount, and output the transmission parameter to transmission section 104.

Furthermore, TV conference device 501 executes the processing of the video and audio encoding transmission processing (step S205) and the end determination processing (step S206) of the first embodiment, and ends the processing.

[0125] The process executed in each of the TV conference apparatuses 601 to 606 shown in FIG. 6 is the same as the process executed by the TV conference apparatus 501 described with reference to FIG.

[0126] Next, the operation of the delay control server 504 will be described with reference to FIG.

FIG. 8 is a flowchart showing a process executed by the delay control server according to the second embodiment. Here, it is assumed that three video conference sessions are established as shown in FIG. 6, and the case where all video conference device capabilities low delay priorities are transmitted to the delay control server will be described. That is, in FIG. 6, the TV conference devices 601 and 602, the TV conference devices 6 03 and 604, and the TV conference devices 605 and 606 are holding separate TV conferences. The case where each TV conference device transmits a low delay priority to the delay control server 607 will be described.

First, the low-delay priority receiving unit 505 receives the low-delay priority transmitted from the transmission / reception unit 502 of the video conference apparatus 501 via the transmission path 111 and outputs the low-delay priority to the delay amount determination unit 506 (step S801).

Next, the delay amount determination unit 506 receives six low delay priorities from the TV conference terminals 601 to 606 in FIG. 6, determines individual delay amounts for each of the six TV conference devices, and sets the delay amount. Output to delay amount transmission section 507 (step S802).

[0130] [Equation 5]

D e 1 a y (t, χ) = D AV E— (Ρ (t, χ)-P A V E (t)) χ Κ

(Formula 5) [0131] (Formula 5) shows an example of a delay amount calculation formula. In (Equation 5), Delay (t, x) and P (t, χ) are the amount of delay and low delay priority of the TV conference device χ at time t, and DAVE is the average value of the predetermined amount of delay. PAVE (t) is an average value of low delay priorities in all TV conference devices at time t, and K is a predetermined delay adjustment parameter.

[0132] It is assumed that time t is synchronized with all terminals, and in calculating the delay amount, the delay amount is calculated from the low delay priority at the same time t.

[0133] As described above, in a plurality of TV conference devices, the delay amount can be reduced as the relatively low delay priority is higher than that of other TV conference devices.

[0134] Note that (Equation 5) is an example of a method for calculating the delay amount, and any method can be used as long as the lower delay priority is higher and the delay amount can be calculated smaller.

[0135] <Step S803: Delay amount transmission processing>

Next, the delay amount transmission unit 507 transmits the delay amount for each of the plurality of TV terminals input from the delay amount determination unit 506 to the respective TV conference apparatuses 601 to 606 through the transmission path 111 (step S803).

[0136] As described above, in the present embodiment, the transmission / reception unit 502 performs low-delay priority transmission processing and delay amount reception processing between the delay control servers 607, and the delay control server 607 includes a plurality of delay control servers 607. Using the low delay priority of the TV conference device, the amount of delay is determined so that the TV conference device having a relatively high low delay priority has a higher value.

[0137] As a result, the delay control server centrally manages the amount of delay, so it is easy to grasp the low delay priority of multiple TV conferences. Or transmission is possible.

[0138] Also, when a plurality of TV conferences use a common transmission path, the bandwidth is used so that a conference in which conversations are frequently exchanged proceeds smoothly. Therefore, it is possible to effectively use the limited bandwidth of the transmission path.

[0139] In the present embodiment, transmission / reception section 502 transmits the low delay priority to the delay control server, but it can also be transmitted to other video conference apparatuses, and the delay amount is determined by the delay control. It is also possible to do it in the video conferencing device. [0140] With this, a delay control server is unnecessary, the delay amount is determined by comparing with other TV conference devices, and it is possible to set a smaller delay amount for a TV conference that requires a lower delay. is there.

[0141] Note that in this embodiment, the transmission / reception unit 502 transmits the low delay priority to the delay control server, but instead of the low delay priority, the transmission / reception frequency of the audio data is exchanged with other TV conference devices. The low-delay priority determination unit 105 can also determine the delay priority as the frequency is higher than the transmission / reception frequency of other TV conference devices received by the transmission / reception unit 502. Is possible.

[0142] According to this, since the low-latency priority is determined in comparison with other video conferences, the necessity of low delay is high among a plurality of video conferences, and it is possible to reduce the delay as the number of video conferences increases. is there.

[0143] In the present embodiment, the transmission / reception unit 502 transmits the low delay priority to the delay control server. However, instead of the low delay priority, the transmission / reception time difference of the audio data is changed to another TV conference device. The low-latency priority determination unit 105 can reduce the delay amount as the difference is smaller compared to the difference in the transmission / reception time of other TV conference devices received by the transmission / reception unit 502. It is also possible to determine a smaller value.

[0144] Thus, when determining the low-latency priority, the difference between the transmission time and the reception time is determined by comparison with other TV conferences. Therefore, a TV conference with a higher request for lower delay can have a lower delay. is there.

Industrial applicability

[0145] The video conference apparatus according to the present invention uses the frequency of transmission and reception of audio data, and the frequency is high! Decide the amount of delay so that the amount of delay decreases as you go, and follow the determined amount of delay! TV By dynamically controlling the encoding or transmission parameters, the amount of delay becomes smaller in the TV conference state where the frequency of audio data transmission / reception is high and the need for low delay is low, and the frequency of audio data transmission / reception is low By increasing the delay amount for video conferencing conditions that do not require a network, it is possible to set an optimal delay amount on a transmission path with limited bandwidth, especially for the best-f-auto Internet. This is particularly useful in video conference systems that require low-delay transmission.

Claims

The scope of the claims

[1] A TV conference control device for controlling a TV conference device that communicates video data and audio data through a transmission line,

The higher the frequency of switching between voice data transmission and reception detected by the video conference device, the lower the delay priority that indicates the degree to which the delay of video data and audio data should be suppressed. Delay priority determination means;

The lower delay priority is higher !, the smaller is! /, A delay amount determination means for determining a delay amount, and an encoding parameter or a coding parameter used in the TV conference device according to the determined delay amount, or Parameter control means for determining packet priority

A video conference control device characterized by that.

[2] The low-delay priority determining means measures the frequency based on the number of times within a fixed time at which transmission and reception of voice data that is sound detected by the video conference device is switched. The video conference control device according to claim 1.

[3] The low-delay priority determining unit compares the frequency with a threshold value, and determines the low-delay priority level that is higher than when the frequency is less than or equal to the threshold value when the frequency is greater than the threshold value.

The video conference apparatus according to claim 2, wherein

[4] The low-latency priority determination means measures the frequency based on a difference between the transmission time and the reception time of the audio data detected by the TV conference device, and the difference between the transmission time and the reception time is small. The higher the low latency priority is determined

The video conference control device according to claim 1, wherein:

[5] The low delay priority determining means compares the difference between the transmission time and the reception time with a threshold value, and if the difference between the transmission time and the reception time is smaller than the threshold value, the transmission time and the reception time are determined. 5. The video conference apparatus according to claim 4, wherein the low-latency priority is determined to be higher than when the difference from time is equal to or greater than the threshold.

[6] The parameter control means determines a reception buffer capacity for decoding the video data and audio data in the video conference device that receives the video data and audio data as the compression code amount parameter. Do The video conference control device according to claim 1, wherein:

[7] The parameter control means determines, as the compression code amount parameter, a maximum generated code amount for encoding the video data and audio data in the video conference apparatus.

The video conference control device according to claim 1, wherein:

[8] A TV conference control device that controls a plurality of TV conference devices that communicate video data and audio data of a plurality of conferences through a common transmission line.

The higher the frequency of switching between voice data transmission and reception detected by each video conference device, the higher the low delay priority that indicates the degree to which the delay of video data and audio data should be suppressed. Low-delay priority determination means for determining;

Delay amount determining means for determining a smaller V ヽ delay amount for the TV conference device group used for each conference as the low delay priority is higher;

Parameter control means for determining encoding parameters or packet priorities used in the video conference device according to the determined delay amount.

A video conference control device characterized by that.

[9] A TV conference control method for controlling a TV conference device that communicates video data and audio data through a transmission line,

The higher the frequency of switching between voice data transmission and reception detected by the video conference device, the lower the delay priority that indicates the degree to which the delay of video data and audio data should be suppressed. A delay priority determination step;

A delay amount determining step for determining a low / low delay amount as the low delay priority is higher and lower, and according to the determined delay amount, an encoding parameter used in the TV conference device or Parameter control step for determining packet priority

A video conference control method characterized by the above.

[10] An integrated circuit for controlling a TV conference device that communicates video data and audio data through a transmission line,

The higher the frequency of switching between voice data transmission and reception detected by the video conference device, the lower the degree to which the delay of video data and audio data should be suppressed. Low delay priority determination means for determining a high delay priority;

The low delay priority is high !, the smaller is !, delay amount determining means for determining a delay amount, and an encoding parameter or packet used in the TV conference device according to the determined delay amount Parameter control means for determining priority

An integrated circuit characterized by that.

[11] A TV conference device that communicates video data and audio data through a transmission line, and an input means for inputting video data and audio data including video and audio, and encodes the input video data and audio data Encoding means for transmitting, transmitting means for transmitting the encoded video data and audio data to another TV conference apparatus through the transmission path,

Receiving means for receiving encoded video data and audio data from another TV conference device through the transmission path;

Decoding means for decoding received video data and audio data; output means for outputting video and audio included in the decoded video data and audio data;

The voiced portion of the voice data transmitted by the transmitting unit and the voiced portion of the voice data received by the receiving unit are detected, and transmission and reception of the voice data including the detected voiced portion is performed. A low-delay priority determination means for determining a low-delay priority that indicates a degree to which delay of video data and audio data should be suppressed as the frequency of switching is higher;

The lower delay priority is higher !, the smaller is! /, The delay amount determining means for determining the delay amount, and the encoding parameters used by the encoding means or the parameter according to the determined delay amount And a norm control means for determining a packet priority included in the data transmitted by the transmission means.

A video conference apparatus characterized by that.

[12] A TV conference system comprising: a plurality of TV conference devices that communicate video data and audio data including a plurality of conferences through a common transmission path; and a TV conference control device that controls the plurality of TV conference devices. There, Each of the video conference devices

Input means for inputting video data and audio data including video and audio; encoding means for encoding the input video data and audio data;

Transmitting means for transmitting the encoded video data and audio data to the other video conference apparatus through the transmission line;

Receiving means for receiving encoded video data and audio data from the other video conference device;

Decoding means for decoding received video data and audio data;

Output means for outputting video and audio included in the decoded video data and audio data,

The video conference control device

A video conference system characterized by this.