TWI539798B - Method for adaptively adjusting video transmission frequency and terminal device thereof and video transmission system - Google Patents

Description

Method for adaptively adjusting video transmission bandwidth and terminal device and video transmission system thereof

The present invention relates to a video transmission system, and more particularly to a video transmission system using a method of adaptively adjusting a video transmission bandwidth and a terminal device thereof.

With the rapid development and advancement of the Internet, the advancement of audio and video compression technology and communication protocols, and the popularity of computer hardware devices, video conferencing systems have become an indispensable tool for companies with cheap beauty. Through the connection of the Internet and the use of video and voice input and output devices, people who are separated from each other can quickly and surely achieve the purpose of information exchange and communication through time and space constraints. Therefore, the video conferencing system brings a lot of convenience and has a very important need in life.

Some multi-person video conferencing software for personal computers, such as Co-life, ooVoo, Skype, etc., have been launched in the market. These softwares have multi-person video conferencing functions, which can be used not only on personal computers, but also in current smart models. Mobile phones and smart TVs also support similar video conferencing capabilities, so multiplayer video will be used extensively in video conferencing, distance learning, and even remote political presentations or long-distance weddings.

However, the transmission of multi-person video conferencing often requires a very large bandwidth, and thus the bandwidth is a major problem in multi-person video conferencing. In multi-person video conferencing, since the picture comes from a number of different locations, the required bandwidth must be doubled to avoid image delays. In addition, the conventional video conferencing system has a fixed server, and each terminal device from different locations must first log in and connect to the fixed server, and the multi-person video conference can be smoothly carried out through the management and control of the fixed server. However, when the fixed server is down or not working properly due to other factors such as the network, the entire video conferencing system will not be able to continue.

In view of the above, the present invention provides a video transmission system using a method for adapting the video transmission bandwidth and a terminal device thereof, and adopts the concept of a dynamic host to connect each terminal device and dynamically adjusts The transmission bandwidth is used to save bandwidth.

The invention provides a method for adaptively adjusting the bandwidth of a video transmission, which is suitable for a terminal device, which is communicatively connected with one or more target terminal devices. This method includes the following steps. The first video content including the image or the sound is captured by the terminal device according to the first transmission bandwidth, and the first video content is transmitted to the one or more target terminal devices through the network. And downloading, by the terminal device, the second video content from the one or more target terminal devices from the network according to the second transmission bandwidth. Then, the image or sound change of the first and second video content is automatically detected, and the bandwidth adjustment information is determined according to the bandwidth adjustment information, and the settings of the first and second transmission bandwidths are adjusted according to the bandwidth adjustment information. Finally, the first and the second video content are played according to the adjusted settings of the first and second transmission bandwidths.

In an embodiment of the present invention, when the terminal device is used as the main control terminal, the method further includes: transmitting the bandwidth adjustment information to the target terminal device, so that the target terminal device adjusts the information according to the bandwidth adjustment bandwidth. Adjustment.

In an embodiment of the present invention, when the terminal device is used as the master terminal, the method further includes sending a control message to specify one of the target terminal devices as the master terminal.

In an embodiment of the present invention, when the terminal device is not used as the master terminal, the method further includes receiving another bandwidth adjustment information from the target terminal device through the network, thereby adjusting the first and second transmission frequencies. Wide setting.

In an embodiment of the present invention, the step of playing the first and second video content according to the adjusted first and second transmission bandwidth settings includes adjusting image resolution, screen update rate, and image compression. Ratio, image compression mode or stop playing the image to meet the adjusted settings of the first and second transmission bandwidths, thereby playing the first and second video content.

The invention further provides a terminal device for adaptively adjusting a video transmission bandwidth, which comprises a controller, a video capture module, an upload/download module, a detection module and a play module. The controller is used to set the transmission bandwidth. The video capture module captures the first video content including the image or sound according to the first transmission bandwidth set by the controller. The uploading/downloading module is coupled to the video capturing module and the controller, and transmits the first video content to the one or more target terminal devices through the network, and the second transmission bandwidth is set according to the second transmission bandwidth of the controller. Downloading one or more second video content from the target terminal device. The detection module is coupled to the video capture module and the upload/download module, and automatically detects image or sound changes of the first and second video content, and transmits the detection result to the controller, and the controller determines The bandwidth adjustment information is adjusted correspondingly to the settings of the first and second transmission bandwidths. The playing module plays the first and second video content according to the adjusted settings of the first and second transmission bandwidths.

The present invention further provides a video transmission system including one or more target terminal devices and a terminal device. The terminal device and one or more target terminal devices are in communication with each other. The terminal device includes a controller, a video capture module, an upload/download module, a detection module, and a play module. The controller is used to set the transmission bandwidth. The video capture module captures the first video content including the image or sound according to the first transmission bandwidth set by the controller. The uploading/downloading module is coupled to the video capturing module and the controller, and transmits the first video content to the one or more target terminal devices through the network, according to the second transmission bandwidth set by the controller. Downloading one or more second video content from the target terminal device. The detection module is coupled to the video capture module and the upload/download module, and automatically detects image or sound changes of the first and second video content, and transmits the detection result to the controller, and the controller generates The bandwidth adjustment information is adjusted correspondingly to the settings of the first and second transmission bandwidths. The playing module plays the first and second video content according to the adjusted settings of the first and second transmission bandwidths.

Based on the above, the method for adapting the bandwidth of the video transmission provided by the present invention and the video transmission system of the terminal device thereof can be mastered according to the performance of the network in the video transmission system and the size of the bandwidth resource. Hand over to maintain the quality of video transmission. And by automatically detecting changes in video content, the transmission bandwidth can be dynamically adjusted to save bandwidth.

The above described features and advantages of the present invention will be more apparent from the following description.

FIG. 1 is a block diagram of a terminal device for adaptively adjusting a video transmission bandwidth according to an embodiment of the invention. Referring to FIG. 1, the terminal device 100 of the present embodiment is, for example, a television, a computer, a notebook computer, a tablet computer, or a smart phone having a video function. The terminal device 100 includes a controller 110, a video capture module 120, an upload/download module 130, a detection module 140, and a play module 150. The functions of each module are described below: The controller 110 can be a functional module implemented by hardware and/or software. The hardware may include a central processing unit (CPU), a chipset, a microprocessor, and the like, or a combination of the hardware devices having the computing function, and the software may be a driver, an application software, or the like. The controller 110 can be used to set the transmission bandwidth, transmit or receive various control signals.

The video capture module 120 is coupled to the controller 110 and the upload/download module 130. The video capture module 120 includes various hardware devices, such as a lens, a microphone, and the like, for capturing user images and sounds, and may include The video content of the user's image and sound is transmitted to the network N through the upload/download module 130.

The upload/download module 130 must have a communication network interface that can be connected to the network to upload or download video content. The terminal device 100 of the embodiment can be connected to the network through a wired or wireless connection. Therefore, the communication network interface of the upload/download module 130 can support, for example, an asymmetric digital subscriber line (ADSL) network. The network interface of the Cable MODEM network; or the network interface that supports the Wireless Fidelity (WiFi) network and the Long Term Evolution (LTE) network, not limited to the above .

In an embodiment, the detection module 140 may include a function module implemented by a hardware and/or a software such as an image recognition unit or a voice recognition unit, and may be used to detect an image or sound change of the video content, and the detection result may be detected. It is transmitted to the controller 110. In another embodiment, the detection module 140 can also detect a signal input by a user through an input unit (eg, a keyboard, a touch screen, etc.).

The play module 150 is coupled to the controller 110 and the upload/download module 130, and corresponds to the video bandwidth downloaded by the play upload/download module 130 and the video capture module 120 according to the transmission bandwidth set by the controller. Video content.

FIG. 2 is a flow chart of a method for adaptively adjusting a video transmission bandwidth according to an embodiment of the invention. The method of this embodiment is applicable to the terminal device 100 of FIG. The operation flow of the terminal device 100 of the present embodiment will be described in detail below with reference to FIGS. 1 and 2.

In step S210, the video capture module 120 of the terminal device 100 captures the first video content including the image or the sound according to the first transmission bandwidth, and transmits the first video content to the one or more targets through the network N. Terminal device.

Here, the connection relationship between the terminal device 100 and the plurality of target terminal devices must be described. FIG. 3A illustrates a connection between the terminal device 100 and a plurality of target terminal devices according to an embodiment of the invention. Schematic diagram of the architecture. Referring to FIG. 3A, the target terminal device in this embodiment is exemplified by four, wherein the target terminal devices 200, 300, 400, and 500 and the terminal device 100 may be the same or similar designs, and the terminal device 100 and the target terminal device 200, 300, 400, 500 are connected to each other. In this embodiment, the terminal device 100 wants to transmit the first video content to the target terminal device 200, 300, 400, 500, except that it can be transmitted to the target terminal device 200, 300 through the data streams f1, f2, f3, and f4, respectively. In addition to 400 and 500, the terminal device 100 of this embodiment may also select to directly or indirectly transmit the first video content.

For example, FIG. 3B is a schematic diagram of transmitting video content in a split manner according to an embodiment of the invention. Referring to FIG. 3B, it is assumed that the terminal device 100 cannot transmit the first video content to the target terminal device 200, 300, 400, 500, respectively, due to factors such as poor network performance or insufficient bandwidth. At this time, the terminal device 100 can select the target terminal device having a larger bandwidth to perform the operation of the offload transmission. For example, the terminal device 100 selects to transmit the first video content to the target terminal device 300 having a larger bandwidth, and then the target terminal device 300 transmits the received first video content to the target terminal through the data streams f5, f6, and f7, respectively. Devices 200, 400, 500. In this way, the bandwidth required for the terminal device 100 to transmit the first video content can be reduced. The video transmission system 30 formed by the terminal device 100 and the target terminal devices 200, 300, 400, and 500 can effectively reduce transmission failure or image delay due to factors such as unstable network performance or insufficient bandwidth. And so on.

Referring back to FIG. 1 and FIG. 2, in step S220, the uploading/downloading module 130 of the terminal device 100 can download the second video from the target terminal device 200, 300, 400, 500 from the network according to the second transmission bandwidth. content. The first transmission bandwidth and the second transmission bandwidth may be the same or different settings.

After receiving the first and second video content, the detecting module 140 of the terminal device 100 automatically detects the image or sound change of the first and second video content, and transmits the detection result to the control. The controller 110 determines the bandwidth adjustment information according to the detection result, and adjusts the settings of the first and second transmission bandwidths in real time.

In detail, the video transmission system formed by the terminal device 100 and the target terminal devices 200, 300, 400, and 500 of the present embodiment is mostly applied to video conferencing, distance teaching, and the like. For example, in video conferencing, the most important thing is the host and speaker of the video conference; if it is used in distance education, the most important thing is the teacher or the speaker. Therefore, the detection module 140 can distinguish, by the image recognition unit or the voice recognition unit, the first video content and the plurality of second video content from the target terminal devices 200, 300, 400, and 500 respectively as the speaking end or There is an image of a special action such as raising a hand, and the detection result is further transmitted to the controller 110.

The controller 110 can then determine the importance order of the plurality of video content according to the detection result and generate the bandwidth adjustment information. The terminal device belonging to the speaking end has the highest importance, so it can be matched with a larger bandwidth; the terminal device not belonging to the speaking terminal is less important, so it can be matched with a lower bandwidth. In other words, the controller 110 adjusts the first transmission bandwidth used by the video capture module 120 according to the bandwidth adjustment information, and adjusts the first use of the upload/download module 130 to download video content from other target terminal devices. Two transmission bandwidth.

Finally, in step S240, the playing module 150 plays the first video content and the plurality of second video content according to the adjusted first transmission bandwidth and the second transmission bandwidth. In order to cooperate with the adjusted transmission bandwidth setting, the playback module 150 can process and play the first video content and the plurality of second video by adjusting the image resolution, the screen update rate, the image compression ratio, and the image compression mode. content. For example, the amount of data with a resolution of 800×600 pixels is small; the amount of data with a resolution of 1024×768 pixels is large. If the screen update rate is 10 frames per second (frame per second, fps), the amount of data is small; if the screen update rate is 30 frames per second, the amount of data is large. The video content of the higher compression ratio is smaller; the video content of the lower compression ratio is larger. Video content with a relatively small amount of data can be captured at a lower bandwidth; conversely, video content with a larger amount of data must be captured with a larger bandwidth. Even when the network performance is poor, the controller 110 can control the playing module 150 to play only the video content of the highest importance, and the rest of the video content stops playing the screen, so that the required bandwidth can be greatly reduced.

Based on the above, the terminal device 100 of the present embodiment can initially capture or download the video content according to the preset transmission bandwidth, but the terminal device 100 can automatically detect which is the speaking end or the importance as time changes. The higher video content is appropriately allocated with higher bandwidth resources, and the bandwidth required for capturing the remaining video content is reduced in time, so that the terminal device 100 can achieve the purpose of saving bandwidth.

Furthermore, the terminal device 100 can control the target terminal devices 200, 300, 400, and 500 to adjust the bandwidth resources in addition to adapting the bandwidth resources of the video content. Specifically, if the terminal device 100 functions as a master, the terminal device 100 has the mastership of the entire video transmission system. That is, the terminal device 100 can transmit the bandwidth adjustment information to the target terminal devices 200, 300, 400, and 500, and the target terminal devices 200, 300, 400, and 500 adjust the bandwidth resources according to the bandwidth adjustment information.

4A and FIG. 4B are schematic diagrams showing an application scenario of a video transmission system according to an embodiment of the invention. This embodiment is applied to a video transmission system for distance learning. The video transmission system includes nine terminal devices, which are used by the teacher T and the students S1 to S8, respectively. Since the teacher T is teaching, the terminal device used by the teacher T captures the video content including the image and sound of the teacher T with a larger bandwidth; the rest of the video content of the student S1~S8 is downloaded at a lower bandwidth. Similarly, the terminal devices used by the students S1~S8 download the video content of the teacher T at a larger bandwidth. As shown in the capture screen 40 of FIG. 4A, since the video content of the teacher T is downloaded with a larger bandwidth, the resolution is the highest, and the resolution of the video content of the remaining students S1 to S8 is low.

It is assumed here that the teacher T is the master, that is, the terminal device used by the teacher T has the mastership, so when the teacher T wants to click on one of the students S1 to S8 to speak, the teacher T can use the terminal device to send A bandwidth adjustment information is given to the students S1~S8, and the student who will speak next is informed, and the terminal device used by the students S1~S8 receives the bandwidth adjustment information, and then adjusts the bandwidth resource accordingly. As shown in the screen 40' of FIG. 4B, it is assumed that the teacher T clicks on the student S5 to make a speech. At this time, each terminal device in the system changes the video content of the student S5 with a larger bandwidth. At this time, the student S5 The resolution of the video content is the highest, and the remaining terminal devices are captured with a smaller bandwidth. Finally, when the student S5 has stopped speaking, the transmission bandwidth can be re-adjusted, and the video content of the teacher T can be retrieved with a larger bandwidth.

In summary, the generation of the bandwidth adjustment information has three control modes: one is to use the detection module in the terminal device for automatic detection, for example, when the detection module uses the image or sound to recognize that the student S5 has stopped speaking, The bandwidth adjustment information is automatically generated. The other is to use the detection module to detect the bandwidth adjustment information input by the user through the input unit. For example, the teacher T can send the bandwidth adjustment information through an input tool such as a keyboard or a mouse. The teacher T and the students S1~S8 each have autonomy and control the bandwidth resources of their terminal devices.

It must be noted that the terminal device as the master terminal must record the Internet address of all the remaining terminal devices, and once the terminal device as the master terminal finds that it cannot be used as the master due to network problems or other factors. At the end, the control message can be sent to the remaining terminal devices, one of which is designated as the new master, and the Internet address of all the terminal devices is transmitted to the new host to complete the mastership. The action is followed by the new master to set the bandwidth adjustment information of the video content transmission mode or format.

In summary, the video transmission system provided by the present invention uses the concept of a dynamic host to connect each terminal device, and does not need to be connected to a fixed server as described in the prior art. Greatly reduce the cost of the architecture video transmission system. Each terminal device in the video transmission system of the present invention has an opportunity to serve as a master terminal, and can perform mastership transfer according to the performance of the network in the video transmission system and the size of the bandwidth resource to maintain the video. The quality of the transmission. More importantly, the terminal device in the video transmission system can dynamically adjust the transmission bandwidth to save bandwidth by automatically detecting changes in video content.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

100. . . Terminal device

110. . . Controller

120. . . Video capture module

130. . . Upload/download module

140. . . Detection module

150. . . Playback module

200, 300, 400, 500. . . Target terminal device

30. . . Video transmission system

40, 40’. . . Capture picture

F1~f7. . . Data flow

N. . . network

T. . . teacher

S1~S8. . . student

S210~S240. . . Steps to adaptively adjust the bandwidth of the video transmission

FIG. 1 is a block diagram of a terminal device for adaptively adjusting a video transmission bandwidth according to an embodiment of the invention.

FIG. 2 is a flow chart of a method for adaptively adjusting a video transmission bandwidth according to an embodiment of the invention.

FIG. 3A is a schematic diagram showing a connection architecture of a terminal device 100 and a plurality of target terminal devices according to an embodiment of the invention.

FIG. 3B is a schematic diagram of transmitting video content in a split manner according to an embodiment of the invention.

4A and 4B are schematic diagrams showing an application scenario of a video transmission system according to an embodiment of the invention.

S210~S240. . . Steps to adaptively adjust the bandwidth of the video transmission

Claims (15)

A method for adaptively adjusting a video transmission bandwidth, which is applicable to a terminal device, wherein the terminal device is in communication with at least one target terminal device, the method comprising: capturing, by the terminal device, an image according to a first transmission bandwidth or a first video content of the sound, and the first video content is transmitted to the at least one target terminal device through a network; and the terminal device downloads the at least one target from the network according to the at least one second transmission bandwidth At least one second video content of the terminal device; automatically detecting an image or sound change of the first and the at least one second video content, thereby determining a bandwidth adjustment information, and adjusting the first and the second according to the bandwidth adjustment information Setting the at least one second transmission bandwidth; and playing the first and the at least one second video content according to the adjusted settings of the first and the at least one second transmission bandwidth. The method for adaptively adjusting the video transmission bandwidth according to the first aspect of the invention, wherein the terminal device as a master terminal further includes: transmitting the bandwidth adjustment information to the at least one target terminal device, And causing the at least one target terminal device to adjust the transmission bandwidth according to the bandwidth adjustment information. The method for adaptively adjusting a video transmission bandwidth as described in claim 2, wherein when the terminal device functions as the master, the method further includes: transmitting a control message to designate one of the at least one target terminal device as The master end, and complete the handover of the master control. Adaptation of video transmission as described in item 2 of the patent application scope a method for transmitting a bandwidth, wherein when the terminal device is not the master, the method further includes: receiving, by the network, a bandwidth adjustment information from the at least one target terminal device, thereby adjusting the first and the at least one The setting of the second transmission bandwidth. The method for adaptively adjusting a video transmission bandwidth according to the first aspect of the patent application, wherein the first and second video content are played according to the adjusted setting of the first and the at least one second transmission bandwidth The step includes: playing the first by adjusting the image resolution, the image update rate, the image compression ratio, the image compression mode, or stopping the playback of the image to meet the adjusted settings of the first and the at least one second transmission bandwidth. And the at least one second video content. A terminal device for adaptively adjusting a video transmission bandwidth, the terminal device comprising: a controller for setting a transmission bandwidth; a video capture module, according to a first transmission bandwidth set by the controller a first video content including an image or a sound; an uploading/downloading module coupled to the video capturing module and the controller, and transmitting the first video content to the at least one target terminal device through a network, Downloading at least one second video content from the at least one target terminal device from the network according to the at least one second transmission bandwidth set by the controller; a detection module coupled to the video capture module and The upload/download module The group automatically detects image or sound changes of the first and the at least one second video content, and transmits the detection result to the controller, where the controller generates a bandwidth adjustment information and correspondingly adjusts the first And the setting of the at least one second transmission bandwidth; and a playing module, playing the first and the at least one second video content according to the adjusted settings of the first and the at least one second transmission bandwidth. The terminal device for adapting the video transmission bandwidth according to the sixth aspect of the invention, wherein the terminal device serves as a master terminal: the detecting module further includes transmitting the bandwidth adjustment information to the at least one And the target terminal device, so that the at least one target terminal device adjusts the transmission bandwidth according to the bandwidth adjustment information. The terminal device for adaptively adjusting a video transmission bandwidth according to claim 7, wherein when the terminal device functions as the master terminal, the controller further includes transmitting a control message to specify the at least one target terminal device. One of them is the master, and the master is handed over to the designated target terminal device. The terminal device for adapting the video transmission bandwidth according to claim 7, wherein when the terminal device is not the master terminal, the controller further comprises receiving, by the network, the at least one target terminal device. A bandwidth adjustment information is used to adjust the settings of the first and the at least one second transmission bandwidth. The terminal device for adaptively adjusting the video transmission bandwidth according to claim 6 of the patent application, wherein: the playback module adjusts image resolution, screen update rate, and image The compression ratio, the image compression mode, or the stop of playing the image to meet the adjusted settings of the first and the at least one second transmission bandwidth. A video transmission system comprising: at least one target terminal device; and a terminal device communicatively coupled to the at least one target terminal device, wherein the terminal device comprises: a controller for setting a transmission bandwidth; and a video capture module a first video content including an image or a sound according to a first transmission bandwidth set by the controller; an upload/download module coupled to the video capture module and the controller, The first video content is transmitted to the at least one target terminal device through a network, and at least one second video content from the at least one target terminal device is downloaded from the network according to the at least one second transmission bandwidth set by the controller. a detection module coupled to the video capture module and the upload/download module, automatically detecting image or sound changes of the first and the at least one second video content, and transmitting the detection result Giving the controller, the controller determining a bandwidth adjustment information and correspondingly adjusting the settings of the first and the at least one second transmission bandwidth; and a playing module, according to the adjusted The transmission bandwidth of a second set corresponding to at least the first player and at least one second video content. The video transmission system of claim 11, wherein: The terminal device serves as a master terminal and transmits the bandwidth adjustment information to the at least one target terminal device, and the at least one target terminal device respectively receives the bandwidth adjustment information, and accordingly adjusts the transmission bandwidth. The video transmission system of claim 12, wherein when the terminal device functions as the master terminal, the terminal device further includes transmitting a control message to each of the target terminal devices to specify the at least one target terminal device. One as the master. The video transmission system of claim 13, wherein: the target terminal device designated as the master terminal transmits another bandwidth adjustment message to the terminal device and the remaining target terminal device, and the terminal device And the remaining target terminal device adjusts the setting of the transmission bandwidth according to the another bandwidth adjustment message. The video transmission system of claim 12, wherein the terminal device and each of the target terminal devices adjust image resolution, picture update rate, image compression ratio, and image compression ratio of the first and the at least one second video content, Image compression mode or stop playing the image to match the adjusted transmission bandwidth setting.