TWI410137B - Video frame flow control device and method for controlling video frame - Google Patents

Abstract

A video frame flow control device is used for capturing a plurality of video frames from a camera, and controls the video frame flow in accordance with a plurality of audio signals captured from a microphone by an audio capture unit. The video frame flow control device includes a video capture unit, a first calculating unit, a second calculating unit and an analysis unit. The video capture unit successively defines the captured video frames as, {Sn}, n=1, 2,..., N, N being a positive integer and there is a fist time interval between the every two neighboring video frames. Furthermore, the first video frame is captured right after the second video frame. The first calculating unit is used for calculating the first time interval. The second calculating unit is used for calculating a second time interval, wherein the second time interval is taken to finish sending the second video frame. When the first video frame is captured, the analysis unit will compare the first time interval with the second time interval to determine whether the first video frame is to be kept or not.

Description

Video frame flow control device and video frame control method

The present invention relates to a frame flow control device and a frame control method, and more particularly to a video frame flow control device and a video frame control method.

Due to the rapid development of the communication technology industry, it not only greatly enhances the convenience of life, but also causes compression in time and space, so that everyone in modern times no longer restricts the boundaries of geography, but can make each other Closer interaction and the exchange of a large amount of information knowledge make it possible to pursue the best interests of the common interests. Among them, the popularization of 3G networks makes information transmission and knowledge exchange more convenient and unimpeded.

When the complete 3G network system is built, in addition to providing voice communication functions, the most important thing is that 3G will open a door for data services, enabling users to obtain seamless services through their mobile devices anytime, anywhere. . Among them, the demand for 3G mobile phones with video phones is increasing day by day, and it is favored by more and more users, because it not only provides the traditional voice call function, but also increases the video sharing function, so that users can make voice calls. At the same time, share instant camera images or video to enrich your visual experience and add fun.

At present, the system processing flow of the 3G mobile phone is as follows: First, after the video call is connected, the handshake operation of the 3G-324M layer is performed, and then after the handshake action of the protocol layer is completed, if the two parties successfully negotiate the video call. Then, the audio and video will be retrieved by the corresponding hardware devices, and the data compression processing of the format is performed according to the handshake negotiation result, and then the sub-protocol H.223 of the 3G-324M protocol is combined into a single binary data. The stream is finally sent to the other party via the wireless network device. The processing flow at the receiving end will be a reverse process. The receiving end first performs the decomposed processing of the received binary data stream by the H.223 protocol, decomposes it into audio and video data, and decompresses it to the corresponding hard. Play and display.

However, in the above data processing flow, the video phone mainly has a video delay problem during the call, and the reason may include: a network delay factor, a delay caused by a video frame in a data processing process such as capturing and transmitting. Among them, the network delay will be effectively solved with the performance improvement of the 3G network system. In addition, the processing delay of the video frame is mainly due to the compression of the video frame after the capture and the 3G-324M H.223 composite protocol composite. It can only be sent out over the wireless network. Since the frequency bandwidth of the current video telephone is fixed at 64K, the frequency of capturing the video frame of the camera generally tends to fluctuate as the system load changes. Therefore, the number of video frames captured per unit time will not be fixed. Therefore, when the system load is large, the video frames captured from the camera cannot be compressed in time and combined according to the H.223 composite protocol. This will cause a large amount of post-compositing to stay in the transmit queue, causing delays in the video frame at the receiving end. Among them, compared with the video, after the audio is compressed, the amount of data is extremely small, which basically does not affect.

Therefore, the present invention provides a video frame flow control device, which can adjust the video frame amount in a timely manner in the video frame processing flow to improve the delay problem, thereby improving performance and stability.

An aspect of the present invention provides a video frame flow control apparatus that can actually adjust a video frame amount to reduce a delay caused by processing a video frame.

According to an embodiment of the present invention, a video frame flow control device is configured to capture a plurality of video frames from a camera, and control video frame traffic according to a plurality of audios captured by the audio capture unit from the microphone. The video frame flow control device includes a video frame capture unit, a first calculation unit, a second calculation unit, and an analysis unit. The video frame capturing unit is configured to sequentially define the captured video frames as the first video frame to the Nth video frame, and the first video frame is captured after the second video frame, where any two consecutive videos are captured. The frame has a first time interval. The first calculation unit is then used to calculate the first time interval. The second calculating unit is configured to calculate a second time interval according to the transmission bandwidth and the captured audio, wherein the second time interval is an estimated time required for the second video frame to complete the transmission. The analyzing unit is configured to compare the first time interval with the second time interval when the video frame capturing unit captures the first video frame to decide to discard or retain the first video frame.

Another aspect of the present invention is to provide a method for controlling a video frame, which can avoid the stagnation of a video frame, thereby improving the processing efficiency of the video.

According to another embodiment of the present invention, a method for controlling a video frame includes the following steps. In this embodiment, first, a plurality of video frames are sequentially captured from a camera, and are respectively defined as a first video frame to an Nth video frame, and the first video frame is captured after the second video frame. Any two consecutive video frames have a first time interval. Then, calculating a first time interval according to the captured video frame; and calculating a second time interval according to a transmission bandwidth and a plurality of audios taken in a unit time, wherein the second time interval is an estimation The time required for the second video frame to complete transmission. Finally, the first time interval and the second time interval are compared to decide to discard or retain the first video frame.

In this way, the video frame flow control device can filter the excess video frames, reducing the problem of video frame delay, thereby improving the processing efficiency of the video.

In order to make the description of the present invention more complete and complete, reference is made to the accompanying drawings and the accompanying drawings. On the other hand, well-known elements and steps are not described in the embodiments to avoid unnecessarily limiting the invention.

FIG. 1 is a functional block diagram of a video frame flow control apparatus according to an embodiment of the present invention. As shown, a video frame flow control device 100 is configured to capture a plurality of video frames from the camera 150 and control the video frame traffic according to the plurality of audios captured by the audio capture unit 160 from the microphone 170. The video frame stream control apparatus 100 includes a video frame capturing unit 110, a first calculating unit 120, a second calculating unit 130, and an analyzing unit 140. The video frame capturing unit 110 sequentially defines the captured video frames as the first video frame to the Nth video frame, and the first video frame is captured after the second video frame, where any two consecutive video frames are captured. There is a first time interval T1. The first calculating unit 120 is configured to calculate the first time interval T1 according to the captured video frame. The second calculating unit 130 is configured to calculate a second time interval T2 according to the transmission bandwidth and the number of bits of the plurality of audios that are captured, wherein the second time interval T2 is used to estimate that the second video frame is completely transmitted. time. The analyzing unit 140 is configured to compare the first time interval T1 and the second time interval T2 when the video frame capturing unit 110 captures the first video frame to decide to discard or retain the first video frame.

In an embodiment, the first calculating unit 120 calculates the first time interval T1 by providing an average compression time according to the second to sixth frames of video frames, and provides the same to the analyzing unit 140. When the number of video frames captured by the video frame capturing unit 110 is less than six frames, each of the video frames is directly subjected to format compression combining. However, the calculation method of the first time interval T1 is not limited thereto. For example, in other embodiments, the average compression time of the second to eighth frames of video frames may also be calculated as the first time interval T1, and When the number of captured video frames does not reach eight frames, each of the video frames is directly subjected to format compression composite.

In addition, the above transmission bandwidth is 64K, and the audio capture unit 160 extracts audio from the microphone 170. Therefore, the second calculating unit 130 will calculate the second time interval T2 according to the actual video and audio data size and the transmission bandwidth, and then provide to the analyzing unit 140 to perform comparative analysis with the first time interval T1.

In addition, the second calculating unit 130 further includes a speed calculating module 132 and a time interval calculating mode resistance 134. The speed calculation module 132 calculates the average transmission speed of the video frame by subtracting the transmission bandwidth from the number of compression bits of the audio captured by the audio extraction unit 160 per unit time. For example, in an embodiment, if the transmission bandwidth is 64K, and the number of compression bits of the audio captured in a unit time is 8K, the average transmission speed of the video frame is 56K.

The time interval calculation modulo 134 is calculated according to the average transmission speed, and is transmitted to the analysis unit 140, wherein the second time interval T2 is the number of bits of the second video frame divided by the average transmission speed. . For example, in an embodiment, if the transmission bandwidth is 64K, the number of compressed bits of the audio captured in a unit time is 8K, and the number of bits of the second video frame is 4K, according to the above disclosure, The average transmission speed of the video frame is 56K, so that the second time interval T2 can be calculated, and the number of bits of the second video frame is 4K divided by the average transmission speed 56K, which is 0.071 seconds.

Therefore, if the first time interval T1 is less than or equal to the second time interval T2, it means that when the first video frame is captured, the second video frame has not been sent yet, so the analyzing unit 140 will discard the first video frame at this time. To reduce unnecessary video frame compression processing, thereby reducing system load.

However, if the first time interval T1 is greater than the second time interval T2, it means that when the first video frame is captured, the second video frame has been transmitted, so the analyzing unit 140 will retain the first video frame and then format. Compressed and transmitted to the encoder 180.

In this manner, the video frame stream control apparatus 100 can determine whether to filter the unnecessary video frames according to the first time interval T1 and the second time interval T2, so as to reduce the video frame delay problem, thereby improving the image call quality.

2 is a flow chart showing a method for controlling a video frame according to another embodiment of the present invention. The control method of the video frame in this embodiment is achieved, for example, by the video frame flow control device shown in FIG. 1. As shown in FIG. 2, the video frame control method 200 includes the following steps 210-270. First, step 210 is to sequentially capture a plurality of video frames from a camera, and sequentially define the first video frame to the Nth video frame, and the first video frame is captured after the second video frame. Two consecutive video frames have a first time interval T1. Step 220 is to detect whether the total number of captured video frames is greater than five frames. When the total number of video frames is less than or equal to five frames, step 270 is directly performed, that is, the first video frame directly enters format compression, composite, and Transfer to the wireless network to send. In addition, when the total number of captured video frames is greater than five frames, step 230 is performed.

Step 230 is to calculate the average compression time of the second to sixth frames of video frames as the first time interval T1. Next, in step 240, a second time interval T2 is calculated according to the transmission bandwidth and the number of bits of the audio captured in the unit time, and the second time interval T2 is the time required for the second video frame to complete the transmission. Then, step 250 is to determine whether the first time interval T1 is greater than the second time interval T2.

However, when the first time interval T1 and the second time interval T2 are to be compared according to the above steps, the transmission bandwidth of the step 250 is 64K, and the audio is extracted from the microphone by the audio capturing unit, thereby calculating the 撷The average transmission speed of one of the video frames is taken, that is, the transmission bandwidth minus the number of compression bits of the audio captured in a unit time. The number of bits of the second video frame is then divided by the average transmission speed, and a second time interval T2 is obtained.

Therefore, when the first time interval T1 is less than or equal to the second time interval T2, step 260 is performed. Step 260 is to directly discard the first video frame to avoid the retention of the video frame, thereby improving the processing efficiency of the video. On the other hand, when the first time interval T1 is greater than the second time interval T2, step 270 is performed. In step 270, the first video frame is reserved, and then the format is compressed and composited.

In this way, the video frame flow control device can discard the excess video frame by calculating the first time interval T1 and the second time interval T2 to reduce the system load, thereby effectively solving the time delay and providing a stable communication quality.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and the present invention can be modified and modified without departing from the spirit and scope of the present invention. The scope is subject to the definition of the scope of the patent application attached.

100. . . Video frame flow control device

110. . . Video frame capture unit

120. . . First computing unit

130. . . Second computing unit

132. . . Speed calculation module

134. . . Time interval calculation module

140. . . Analysis unit

150. . . camera

160. . . Audio capture unit

170. . . microphone

180. . . Encoder

200. . . Method of operation

210~270. . . step

The above and other objects, features, advantages and embodiments of the present invention will become more apparent and understood.

FIG. 1 is a functional block diagram of a video frame flow control apparatus according to an embodiment of the present invention.

2 is a flow chart showing a method for controlling a video frame according to another embodiment of the present invention.

100. . . Video frame flow control device

110. . . Video frame capture unit

120. . . First computing unit

130. . . Second computing unit

132. . . Speed calculation module

134. . . Time interval calculation mode

140. . . Analysis unit

150. . . camera

160. . . Audio capture unit

170. . . microphone

180. . . Encoder

Claims (12)

A video frame flow control device is configured to extract N video frames from a camera, wherein N is a positive integer, and according to an audio capture unit, a plurality of audios captured from a microphone, thereby controlling the video frames The video frame flow control device includes: a video frame capturing unit, configured to sequentially define the captured N video frames as a first video frame, a second video frame, ... An Nth video frame, wherein the first video frame is captured after the second video frame, wherein any two consecutive video frames have a first time interval; a first computing unit is configured to use the N Calculating the first time interval by using a part of the video frames in the video frame; a second calculating unit, configured to calculate a second time interval according to a transmission bandwidth and the captured audio, wherein the second time interval The time interval is an estimated time required for the second video frame to complete the transmission; and an analyzing unit is configured to compare the first time interval with the second time when the video frame capturing unit captures the first video frame Interval, to decide to discard or retain the first video . The video frame flow control device of claim 1, wherein the first calculating unit, when N is greater than 5, is configured to use the second video frame and the third video frame in the N video frames. The average compression time of the sixth frame video frame is calculated, and the first time interval is calculated. For example, the video frame flow control device described in claim 1 is The second computing unit further includes: a speed calculation module, configured to subtract the number of compression bits of the audio captured in the unit time from the transmission bandwidth to calculate one of the N video frames An average transmission speed; and a time interval calculation module for dividing the number of bits of the second video frame by the average transmission speed to calculate the second time interval. The video frame flow control device of claim 1, wherein the transmission bandwidth is 64K. The video frame flow control device of claim 1, wherein the analyzing unit is configured to discard the first video frame when the first time interval is less than or equal to the second time interval. The video frame flow control device of claim 1, wherein the analyzing unit is configured to reserve the first video frame when the first time interval is greater than the second time interval. A video frame control method includes: capturing N video frames, where N is a positive integer, and sequentially defining the N video frames as a first video frame, a second video frame, ... An Nth video frame, and the first video frame is captured after the second video frame, wherein any two consecutive video frames have a first time interval; according to some of the N video frames Frame, calculate the first time Interval; calculating a second time interval according to a transmission bandwidth and the plurality of audios retrieved, wherein the second time interval is estimated to be a time required for the second video frame to complete transmission; and when the video frame is When the unit captures the first video frame, the first time interval and the second time interval are compared to decide to discard or retain the first video frame. The method for controlling a video frame according to claim 7, wherein in the step of calculating the first time interval, when N is greater than 5, according to the second video frame in the N video frames, The third video frame, . . . , the average compression time of the sixth frame video frame, calculates the first time interval. The method for controlling a video frame as described in claim 7, wherein the transmission bandwidth is 64K. The method for controlling a video frame according to claim 9, wherein the step of calculating the second time interval comprises: subtracting the transmission bandwidth from the number of compressed bits of the audio captured in a unit time. And calculating an average transmission speed of the one of the N video frames; and dividing the number of bits of the second video frame by the average transmission speed to calculate the second time interval. The method for controlling a video frame according to claim 7, wherein in the step of deciding to discard or retain the first video frame, when the first time interval is less than or equal to the second time interval, discarding The first video frame. The method for controlling a video frame according to claim 7, wherein in the step of deciding to discard or retain the first video frame, when the first time interval is greater than the second time interval, the first A video frame.