WO2013123839A1 - 一种多人视频通信中的视频编码方法以及终端 - Google Patents

一种多人视频通信中的视频编码方法以及终端 Download PDF

Info

Publication number
WO2013123839A1
WO2013123839A1 PCT/CN2013/070833 CN2013070833W WO2013123839A1 WO 2013123839 A1 WO2013123839 A1 WO 2013123839A1 CN 2013070833 W CN2013070833 W CN 2013070833W WO 2013123839 A1 WO2013123839 A1 WO 2013123839A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
focus
encoding
terminal
frame rate
Prior art date
Application number
PCT/CN2013/070833
Other languages
English (en)
French (fr)
Inventor
谷沉沉
Original Assignee
腾讯科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技(深圳)有限公司 filed Critical 腾讯科技(深圳)有限公司
Priority to RU2014138120/07A priority Critical patent/RU2570355C1/ru
Priority to CA2865045A priority patent/CA2865045C/en
Priority to EP13752176.1A priority patent/EP2819405B1/en
Priority to KR1020147025280A priority patent/KR101629226B1/ko
Priority to BR112014020225-7A priority patent/BR112014020225B1/pt
Priority to AP2014007931A priority patent/AP2014007931A0/xx
Priority to NZ630215A priority patent/NZ630215A/en
Priority to US14/378,091 priority patent/US9332267B2/en
Priority to SG11201404904VA priority patent/SG11201404904VA/en
Publication of WO2013123839A1 publication Critical patent/WO2013123839A1/zh
Priority to PH12014501820A priority patent/PH12014501820A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/127Prioritisation of hardware or computational resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/179Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scene or a shot
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems
    • H04N7/152Multipoint control units therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems
    • H04N7/155Conference systems involving storage of or access to video conference sessions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162User input

Definitions

  • the present invention relates to multi-person video communication technology, and in particular to a video coding method in multi-person video communication and a terminal to which the video coding method is applied.
  • Multi-person video communication refers to real-time video communication involving 3 or more people at the same time.
  • a terminal participating in multi-person video communication usually includes a plurality of video encoders.
  • the computational complexity of video coding is usually high, and the computational complexity of multi-channel video coding is higher. Therefore, it is necessary to set the coding parameters of each video encoder reasonably in the case of limited processing capability of the terminal. Reasonably control the purpose of computational complexity.
  • uniform coding parameters such as the highest encoding frame rate, etc.
  • uniform coding parameters are set for each video encoder on each terminal according to the processing power of the average or most terminals, such as the highest encoding frame rate of each video encoder.
  • Video encoding is performed and the encoded data is sent.
  • the terminal involved in multi-person video communication may be a traditional terminal such as a personal computer (PC), or a mobile electronic device, such as a new terminal such as a mobile phone or a palmtop computer, and different types of terminals in terms of processing capabilities and the like.
  • PC personal computer
  • a mobile electronic device such as a new terminal such as a mobile phone or a palmtop computer
  • different types of terminals in terms of processing capabilities and the like.
  • the present invention provides a video encoding method and a terminal in multi-person video communication, Can improve the overall video communication quality.
  • a video encoding method in multi-person video communication when any terminal X participates in a multi-person video communication, performs the following processing:
  • the coding parameters of the focus video and the coding parameters of the common video are respectively determined, and each enabled focus video encoder and the ordinary video encoder are respectively notified according to the latest determined coding.
  • the parameters are video encoded.
  • a terminal includes: a focus video encoder, a normal video encoder, and an adaptive adjustment module; and an adaptive adjustment module, configured to separately calculate an enabled focus video in the terminal after the terminal participates in a multi-person video communication
  • the coding parameters of the focus video and the coding parameters of the common video are respectively determined, and each enabled focus video encoder and the ordinary video encoder are respectively notified according to the newly determined coding parameters.
  • a computer storage medium storing a computer executable program for performing a video encoding method in multi-person video communication provided by the present invention.
  • each terminal participating in the multi-person video communication can be specifically determined according to its processing capability, the number of enabled focus video encoders, the number of common video encoders, and the like.
  • the coding parameters of the focus video and the coding parameters of the common video enable the computing resources in the terminal to be more rationally utilized, thereby improving the overall video communication quality.
  • FIG. 1 is a flow chart of an embodiment of a video encoding method in multi-person video communication according to the present invention.
  • FIG. 2 is a schematic diagram of a process for determining the highest encoded frame rate of a focus video and the highest encoded frame rate of a normal video by the terminal X of the present invention.
  • FIG. 3 is a schematic structural diagram of a terminal embodiment of the present invention. Detailed ways
  • FIG. 1 is a flow chart of an embodiment of a video encoding method in multi-person video communication according to the present invention.
  • any terminal X participates in a multi-person video communication, it is processed according to the flow shown in FIG. As shown in Figure 1, the following processes are included:
  • Step 11 Terminal X separately counts the number of focus video encoders and common video encoders that are enabled.
  • the terminal X is used to represent any terminal, and the terminal X may be a PC, a mobile phone, a palmtop computer or the like.
  • the video is divided into a focus video and a normal video according to the degree of attention
  • the focus video refers to a video with a high degree of attention, that is, a video that requires attention
  • the ordinary video refers to a video with a low degree of attention, usually, a focus.
  • the resolution of the video is higher, while the resolution of the normal video is lower.
  • the normal video or the focus video is automatically distinguished, for example, by whether the degree of attention reaches a predetermined threshold, or is distinguished by the user of the terminal, and may be distinguished by any other means, which does not constitute a limitation of the present invention.
  • a video connection needs to be established separately from each of the other remote terminals participating in the multi-person video communication; the user (user) of the terminal X can freely according to his own needs. Select whether to use each remote video as a focus video or a normal video, and switch between the communication process, that is, switch the focus video to normal video, or switch the normal video to focus video, for example, for a certain remote video, before Don't pay much attention to it, just use it as a normal video, but if you need to pay attention to it for some reason, you can switch to the focus video, you can switch through the buttons displayed on the communication interface; likewise, participate in the multiplayer video.
  • the user of each of the other remote terminals in the communication can also freely select whether the video generated by the terminal X is used as a focus video or a normal video, and can be switched during communication.
  • terminal X respectively enables a video encoder, In some cases, it is also possible to share a video encoder with multiple remote terminals.
  • Terminal X needs to enable ⁇ «) focus video encoders, and if there are m remote users to use the video generated by the terminal X as the normal video, then Terminal X needs to enable w c (m. ⁇ m) common video encoders.
  • terminal X separately counts the number of focus video encoders and the number of ordinary video encoders that have been enabled.
  • Step 12 The terminal X determines the coding parameters of the focus video and the coding parameters of the common video according to the processing capability of the terminal and the latest statistical result, and notifies each of the enabled focus video encoder and the ordinary video encoder to be determined according to the latest.
  • the encoded parameters are video encoded.
  • the coding parameter mainly refers to the highest coding frame rate.
  • other parameters may also be included. Specifically, which parameters can be determined according to actual needs.
  • the highest encoding frame rate of the focus video can be appropriately increased, so that limited computing resources and the like are concentrated on the focused video with high attention, so that the focused video has higher quality, thereby improving the subjective experience of the user on the video quality.
  • the resolution of the focus video is high, and the highest encoding frame rate of the focus video is increased, which causes the computing resource consumption of the terminal to rise sharply. Therefore, while the highest encoding frame rate of the focus video is increased, the calculation of the control terminal is also required. Resource consumption and the like are within the processing capability of the terminal.
  • the coding parameters of the focus video and the coding parameters of the ordinary video are determined in a targeted manner, so that the computing resources in the terminal can be more rationally utilized, thereby improving the overall video communication quality.
  • FIG. 2 is a schematic diagram of a process in which the terminal X determines the highest encoded frame rate of the focus video and the highest encoded frame rate of the normal video. As shown in Figure 2, it includes:
  • Step 21 Initialize the highest encoded frame rate f b of the focus video and the highest encoded frame rate of the normal video according to its own processing capability and the latest statistical result:
  • r represents the scale factor for calculating the resource consumption when encoding the focus video with a resolution of 3 ⁇ 4> ⁇ and the normal video with a resolution of w s xh s , which can be calculated simply by the resolution ratio:
  • F denotes the processing capability of terminal X, which is represented by the highest encoding frame rate allowed for encoding a video with a resolution of ⁇ , which can be calculated as follows:
  • F fps enc -w usage ; (3 ) where fps enc indicates the encoding speed of the processor of terminal X when the video of one resolution is full-loaded (represented by the average number of frames encoded per second), which can be passed in advance.
  • Step 22 Increase and decrease.
  • the specific value of ⁇ / can be determined according to actual needs, usually 1.
  • Step 23 Determine whether the newly obtained / 6 and /: whether the predetermined condition is satisfied, if yes, repeat step 22, and if no, perform step 24.
  • step 22 If the newly obtained ⁇ and f s , that is, the / A and f s after the processing of the step 22 satisfy both the conditions (a) and (b), the step 22 is repeated, and if not, the step 24 is executed.
  • the absolute value of the difference between the MOS (Mean Opinion Score) score of the focus video and the MOS score of the normal video is less than a predetermined threshold A aS raa
  • Theshold specific value can be determined according to actual needs, such as 0.5;
  • the MOS score is a measure of the subjective quality of the video and can be used by different observers.
  • the subjective scores obtained from the video quality evaluation are averaged, and a scoring method with a perfect score of 5 is generally adopted;
  • the MOS score of the focus video can be determined according to the latest obtained / 6 , and the MOS score of the ordinary video is determined according to the latest obtained f s ;
  • MOS plus (/) 5_ln (/ (5) where "and ? are calculation coefficients;
  • / max represents the highest possible encoding frame rate in video communication
  • the MOS score corresponding to each coding frame rate that may be used in video communication may be calculated in advance according to formula (4) or (5), and stored in the form of a table, and then when needed, Obtain the newly obtained MOS scores corresponding to f b and f s by looking up the table;
  • the newly obtained MOS scores corresponding to f b and f s may be directly calculated according to formula (4) or (5).
  • Step 24 Determine the highest encoding frame rate of the focus video based on the latest y, and determine the highest encoding frame rate of the normal video according to the newly obtained /; that is: use the newly obtained ⁇ - ⁇ / as the true The highest encoded frame rate of the determined focus video, using the newly obtained y; + ⁇ / as the highest encoded frame rate of the determined normal video.
  • the highest encoding frame rate of the focus video is improved, so that the focus video has higher quality, and the subjective quality difference between the focus video and the ordinary video is within a reasonable range, and the calculation will be performed. Resource consumption is controlled within the processing power of the terminal.
  • the method may further include the following steps: The terminal X determines that at least one of the enabled focus video encoder number and the normal video encoder number has occurred. Change, repeat steps 11 and 12.
  • At least one of the number of enabled focus video encoders and the number of normal video encoders may be changed, which may include the following cases: The total number of terminals participating in multi-person video communication is increased or decreased (ie, terminals are joined or logged out), or the total number does not occur. Change, but switching between the focus video and the normal video occurs (for example, a remote terminal switches the video generated by the terminal X from the focus video to the normal video), or any combination of the above various conditions.
  • the adjustment response can be quickly and efficiently determined according to the change of the number of focus video encoders and the number of ordinary video encoders, and the coding parameters of the focus video and the coding parameters of the ordinary video are re-determined, thereby further improving the overall video communication quality.
  • FIG. 3 is a schematic structural diagram of a terminal embodiment of the present invention. As shown in FIG. 3, it includes: a focus video encoder (such as ESI ⁇ EBn shown in FIG. 3), a normal video encoder (as shown in FIG. 3 ⁇ Esm), and an adaptive adjustment module 31.
  • a focus video encoder such as ESI ⁇ EBn shown in FIG. 3
  • a normal video encoder as shown in FIG. 3 ⁇ Esm
  • an adaptive adjustment module 31 an adaptive adjustment module
  • the adaptive adjustment module 31 is configured to: when the terminal participates in a multi-person video communication, Do not count the number of focus video encoders and the number of normal video encoders that have been enabled in the terminal;
  • the adaptive adjustment module 31 is further configured to determine, according to the processing capability of the terminal and the latest statistical result, the coding parameters of the focus video and the coding parameters of the common video, and notify each enabled focus video encoder and the common video encoder.
  • the video coding is performed according to the newly determined coding parameters.
  • the encoding parameters of the focus video and the encoding parameters of the normal video are determined in a targeted manner according to the processing capability thereof, the number of the focus video encoders enabled, the number of ordinary video encoders, and the like.
  • the computing resources in the terminal can be more rationally utilized, thereby improving the overall video communication quality.
  • the adaptive adjustment module is further operable to repeatedly perform its own function when at least one of the enabled number of focus video encoders and the number of normal video encoders changes.
  • the encoding parameters of the focus video and the encoding parameters of the normal video can be re-determined, thereby further improving the overall video communication quality.
  • the encoding parameters include: a highest encoded frame rate.
  • the adaptive adjustment module 31 may specifically include (to simplify the drawing, not shown):
  • a first processing unit configured to separately count the number of the focus video encoders and the number of the normal video encoders that are enabled in the terminal, and send the statistics to the second processing unit;
  • a second processing unit configured to initialize a highest encoded frame rate ⁇ of the focus video and a highest encoded frame rate f s of the normal video according to the processing capability of the terminal and the latest statistical result, and send the highest encoded frame rate f s to the third processing unit;
  • a third processing unit configured to increase / 4 and decrease / s , and determine whether the newly obtained ⁇ sum satisfies a predetermined condition, and if so, repeatedly perform its own function, and if not, determine the focus video according to the latest obtained
  • the highest encoding frame rate determining the highest encoding frame rate of the normal video according to the latest obtained / s , and notifying each enabled focus video encoder and the ordinary video encoder to perform video encoding according to the latest determined highest encoding frame rate
  • a fourth processing unit configured to use the number of focus video encoders and the number of normal video encoders When at least one of the changes occurs, the first processing unit is notified to repeatedly perform its own function.
  • the second processing unit may further include:
  • n represents the latest count of the number of ordinary video encoders
  • n the latest counted number of focus video encoders
  • the encoding speed ratio when the focus video with the resolution of 3 ⁇ 4> ⁇ and the normal video with the resolution of ⁇ xh s are respectively encoded, and the obtained encoding speed ratio is taken as ⁇ ;
  • a second processing subunit configured to send the calculated ⁇ and /; to the third processing unit.
  • the third processing unit may further include:
  • a fourth processing sub-unit configured to determine a MOS score of the focus video according to the newly obtained / 6, and determine a MOS score of the ordinary video according to the latest obtained result
  • the absolute value of the difference between the MOS and MOS scores focus video general video less than a predetermined threshold value ⁇ 3 ⁇ 4 3 ⁇ 4, lm-f s + n -f b -r ⁇ F, ⁇ ⁇ f b, f s ⁇ f, / max represents the highest possible encoding frame rate in video communication, and then determines the newly obtained / ft and satisfies the predetermined condition, notifying the third processing subunit to repeatedly perform its own function, otherwise, using the newly obtained as the highest of the determined focus video Encoding frame rate, using the newly obtained + ⁇ / as the highest encoding frame rate of the determined normal video, and notifying each enabled focus video encoder and the normal video encoder to perform video according to the latest determined highest encoding frame rate coding.
  • the fourth processing sub-unit separately calculates the MOS score corresponding to each encoding frame rate that may be used in the video communication according to a predetermined formula, and stores it in the form of a table, and obtains the latest obtained f b by looking up the table.
  • the fourth processing sub-unit directly calculates the newly obtained / ⁇ and the corresponding MOS score according to a predetermined formula;
  • the terminal shown in Fig. 3 further includes some other components, such as a capture module 32, a video decoder (D1 ⁇ DN as shown in Fig. 3), and a display module 33.
  • the capture module 32 may further include: a focus video source capture unit 321 and a normal video source capture unit 322; wherein the focus video source capture unit 321 sends the captured video data to each enabled focus video encoder, a common video source.
  • the capture unit 322 transmits the captured video data to each enabled normal video encoder; the focus video source capture unit 321 and the normal video source capture unit 322 capture the same video content, but with different resolutions.
  • a video decoder is required for each remote terminal, but a code stream generated by a video encoder code can be sent to one or more remote terminals.
  • the terminal (terminal 0) shown in Figure 3 is enabled for each remote terminal, that is, a video encoder and a video decoder are respectively enabled for the terminals 1 to N. Then, a total of N video encoders and N need to be enabled in the terminal 0. Video decoders.
  • the display module 33 displays the focus video and the normal video decoded by each video decoder.
  • the display module 33 displays the focus video and the normal video decoded by each video decoder.
  • ROM Read Only Memory
  • RAM Ramdom Access Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明公开了一种多人视频通信中的视频编码方法以及终端。该方法包括:当任一终端X参与到一多人视频通信中后,进行以下处理:A、分别统计出自身已启用的焦点视频编码器数和普通视频编码器数;B、根据自身的处理能力以及最新的统计结果,分别确定出焦点视频的编码参数以及普通视频的编码参数,并通知各已启用的焦点视频编码器和普通视频编码器分别按照最新确定出的编码参数进行视频编码。本发明同时公开了一种终端。应用本发明所述方案,能够提高整体视频通信质量。

Description

一种多人视频通信中的视频编码方法以及终端
技术领域
本发明涉及多人视频通信技术, 特别涉及一种多人视频通信中的视频编 码方法以及一种应用该视频编码方法的终端。 背景技术
随着互联网技术和无线通信技术的发展和普及, 群体视频聊天、 视频会 议, 以及网络视频游戏等多人视频通信形式, 使得人与人之间的沟通交流变 得更加方便快捷, 娱乐方式更加丰富直观, 因此越来越受到人们的欢迎。 多 人视频通信是指 3人或 3人以上同时参与的即时视频通信。
由于多人视频通信中需要处理多路视频数据, 为适应多路不同的网络状 况, 参与多人视频通信的终端中通常包括多个视频编码器。 视频编码的计算 复杂度通常较高, 同时进行多路视频编码的计算复杂度会更高, 因此, 需要 在终端的处理能力有限的情况下, 合理地设置各视频编码器的编码参数, 以 便达到合理地控制计算复杂度的目的。
在实际应用中, 通常会按照平均或者多数终端的处理能力为各终端上的 各视频编码器设置统一的编码参数, 如最高编码帧率等, 各视频编码器按照 所设置的最高编码帧率等进行视频编码, 并将编码后的数据发送。
但是, 参与到多人视频通信中的终端可能是个人计算机 ( PC, Personal Computer )等传统终端, 也可能是移动电子设备, 如手机或掌上电脑等新型 终端, 不同类型的终端在处理能力等方面存在着^^大的差别, 这样, 如果针 对各终端中的各视频编码器设置统一的编码参数, 则会导致以下问题: 对于 处理能力较强的终端, 会带来计算资源等的浪费, 从而不能达到最好的视频 通信质量, 而对于处理能力较弱的终端, 则会造成视频通信质量下降等。
无论出现上述哪种情况, 均会导致终端中的计算资源等不能被合理地利 用, 从而影响了整体视频通信质量。 发明内容
有鉴于此,本发明提供了一种多人视频通信中的视频编码方法以及终端 , 能够提高整体视频通信质量。
为达到上述目的, 本发明的技术方案是这样实现的:
一种多人视频通信中的视频编码方法, 当任一终端 X参与到一多人视频 通信中后, 进行以下处理:
A、 分别统计出自身已启用的焦点视频编码器数和普通视频编码器数;
B、根据自身的处理能力以及最新的统计结果,分别确定出焦点视频的编 码参数以及普通视频的编码参数, 并通知各已启用的焦点视频编码器和普通 视频编码器分别按照最新确定出的编码参数进行视频编码。
一种终端, 包括: 焦点视频编码器、 普通视频编码器、 自适应调节模块; 自适应调节模块, 用于当终端参与到一多人视频通信中后, 分别统计出 终端中已启用的焦点视频编码器数和普通视频编码器数;
根据终端的处理能力以及最新的统计结果, 分别确定出焦点视频的编码 参数以及普通视频的编码参数, 并通知各已启用的焦点视频编码器和普通视 频编码器分别按照最新确定出的编码参数进行视频编码。
一种计算机存储介质, 存储有计算机可执行程序, 所述计算机可执行程 序用于执行本发明提供的多人视频通信中的视频编码方法。
可见, 采用本发明所述方案, 可针对参与到多人视频通信中的每个终端, 分别根据其处理能力以及所启用的焦点视频编码器数和普通视频编码器数 等, 有针对性地确定其焦点视频的编码参数以及普通视频的编码参数, 从而 使得终端中的计算资源等能够得到更为合理的利用, 进而提高了整体视频通 信质量。 附图说明
为了更清楚地说明本发明实施例中的技术方案, 下面将对实施例或现有 技术描述中所需要使用的附图作简单地介绍, 显而易见地, 下面描述中的附 图仅仅是本发明的一些实施例, 对于本领域普通技术人员来讲, 在不付出创 造性劳动性的前提下, 还可以根据这些附图获得其他的附图。
图 1为本发明多人视频通信中的视频编码方法实施例的流程图。
图 2为本发明终端 X确定焦点视频的最高编码帧率以及普通视频的最高 编码帧率的过程示意图。 图 3为本发明终端实施例的组成结构示意图。 具体实施方式
针对现有技术中存在的问题, 本发明中提出一种改进后的多人视频通信 中的视频编码方案。
为使本发明的技术方案更加清楚、 明白, 以下参照附图并举实施例, 对 本发明所述方案作进一步地详细说明。
图 1为本发明多人视频通信中的视频编码方法实施例的流程图。 当任一 终端 X参与到一多人视频通信中后, 按照图 1所示流程进行处理。 如图 1所 示, 包括以下流程:
步骤 11 :终端 X分别统计出自身已启用的焦点视频编码器数和普通视频 编码器数。
为便于表述, 本实施例中, 用终端 X来代表任一终端, 终端 X可为 PC、 手机、 掌上电脑等。
本发明中, 根据关注度将视频区分为焦点视频和普通视频, 焦点视频是 指关注度较高的视频, 即需要重点注意的视频, 普通视频则是指关注度较低 的视频, 通常, 焦点视频的分辨率较高, 而普通视频的分辨率则较低。 其中, 普通视频或焦点视频例如通过关注度是否达到一预设阈值来自动区分, 或者 由终端的使用者 由选择进行区分, 也可以釆用任意其它方式进行区分, 其 不构成对本发明的限制。
当终端 X加入到一多人视频通信中后, 需要与参与到该多人视频通信中 的其它每个远程终端之间分别建立视频连接; 终端 X的使用者(用户)可以 根据自身需求, 自由选择将各路远端视频作为焦点视频还是普通视频, 并可 在通信过程中进行切换, 即将焦点视频切换为普通视频, 或将普通视频切换 为焦点视频, 比如, 对于某一路远端视频, 之前不太关注, 只将其作为普通 视频, 但由于某种原因需要对其进行关注, 则可切换为焦点视频, 可通过通 信界面上显示的按钮等来实现切换; 同样, 参与到该多人视频通信中的其它 每个远程终端的使用者也可自由选择将终端 X产生的视频作为焦点视频还是 普通视频, 并可在通信过程中进行切换。
在一实施方式中,针对每个远程终端,终端 X分别启用一个视频编码器, 在某些情况下, 也可以多个远程终端共用一个视频编码器。
这样, 如果有 n个远程用户将终端 X产生的视频作为焦点视频, 那么终 端 X就需要启用 ≤«)个焦点视频编码器,如果有 m个远程用户将终端 X 产生的视频作为普通视频, 那么终端 X就需要启用 wc(m。≤m )个普通视频编 码器。
本步骤中, 终端 X分别统计出自身已启用的焦点视频编码器数和普通视 频编码器数。
步骤 12: 终端 X根据自身的处理能力以及最新的统计结果, 分別确定出 焦点视频的编码参数以及普通视频的编码参数, 并通知各已启用的焦点视频 编码器和普通视频编码器分别按照最新确定出的编码参数进行视频编码。
由于焦点视频的关注度高于普通视频, 因此针对焦点视频和普通视频, 可采用不同的编码参数, 所述编码参数主要是指最高编码帧率, 当然, 在实 际应用中, 也可能包括其它参数, 具体包括哪些参数可根据实际需要而定。
即可适当提高焦点视频的最高编码帧率, 从而将有限的计算资源等集中 在关注度高的焦点视频上, 使得焦点视频具有更高的质量, 进而提高用户对 视频质量的主观体验。
但是, 焦点视频的分辨率高, 提高焦点视频的最高编码帧率后, 会导致 终端的计算资源消耗等急剧上升, 因此, 在提高焦点视频的最高编码帧率的 同时, 还需要控制终端的计算资源消耗等处于终端的处理能力范围内。
根据上述本发明的多人视频通信中的视频编码方法实施例, 可针对参与 到多人视频通信中的每个终端, 分别根据其处理能力以及所启用的焦点视频 编码器数和普通视频编码器数等, 有针对性地确定其焦点视频的编码参数以 及普通视频的编码参数, 从而使得终端中的计算资源等能够得到更为合理的 利用, 进而提高了整体视频通信质量。
基于上述介绍, 图 2为本发明终端 X确定焦点视频的最高编码帧率以及 普通视频的最高编码帧率的过程示意图。 如图 2所示, 包括:
步驟 21 : 根据自身的处理能力以及最新的统计结果, 分别初始化焦点视 频的最高编码帧率 fb和普通视频的最高编码帧率 :。
fb = fs = F I(m + n - r) ; ( 1 ) 其中, 表示最新统计出的普通视频编码器数; n表示最新统计出的焦点视频编码器数;
r表示分別对分辨率为 ¾>< 的焦点视频和分辨率为 wsxhs的普通视频进 行编码时计算资源消耗的比例因子, 可以简单地按分辨率比例计算:
r = wbx.hbl(wsxhs); ( 2 ) 或者,也可以通过实验测试得到分别对分辨率为 wbxhb的焦点视频和分辨 率为 Wix 的普通视频进行编码时的编码速度比,将得到的编码速度比作为 r, 所述测试是指在终端 X上进行的测试;
F表示终端 X的处理能力, 用编码一个分辨率为 ^ 的视频允许的最 高编码帧率进行表示, 可通过如下方式计算得到:
F = fpsenc-wusage; (3 ) 其中, fpsenc表示终端 X 的处理器满负荷编码一个分辨率为 的视频 时的编码速度(用平均每秒钟编码的帧数表示) , 可预先通过实验测试得到 各不同主频的处理器分别对应的 fpSenc , 并以表格的形式进行存储, 然后通过 查表得到终端 X的处理器的主频对应的 fPSmc; 表示所期望的编码处理对 终端 X的处理器的占用率, 具体取值可根据实际需要而定, 可由用户输入设 定, 并可根据需要进行调整, 如在终端 X 的电量充足时, 设定 ^=60%, 而在终端 X的电量有限或用户期望其省电运行时, 设定 =20%。
步驟 22: 增大 和降低 。
本步骤中, 提高焦点视频的最高编码帧率, 并降低普通视频的最高编码 帧率, 如, 可以令 + fs=fs- f , Δ/>0。 Δ/的具体取值可根据实际 需要而定, 通常为 1。
步碟 23: 确定最新得到的 /6和/:是否满足预定条件, 如果是, 则重复执 行步骤 22, 如果否, 则执行步骤 24。
如果最新得到的 Α和 fs, 即经过步樣 22处理后的 /A和 fs同时满足( a)和 (b) 两个条件, 则重复执行步骤 22, 如果否, 则执行步骤 24。
条件(a)
焦点视频的主观平均意见(MOS, Mean Opinion Score )分值与普通视频 的 MOS分值之差的绝对值小于预定阈值 A aSraa
^MOSTheshold的具体取值可根据实际需要而定, 比如为 0.5;
MOS分值是反映视频的主观质量的一种度量值,可以通过不同观测者对 于视频质量评价得出的主观分值进行平均得到, 一般采用满分为 5分的打分 方式;
可根据最新得到的 /6确定出焦点视频的 MOS分值,并根据最新得到的 fs 确定出普通视频的 MOS分值;
MOS分值:
Figure imgf000007_0001
或, MOS加 (/) = 5_ln(/ (5) 其中, 《和 ?均为计算系数;
/max表示视频通信中可能的最高编码帧率;
在实际应用中, 可预先按照公式(4)或 (5)分別计算出视频通信中可 能用到的每个编码帧率对应的 MOS分值,并以表格的形式进行存储, 然后当 需要时, 通过查表分别得到最新得到的 fb和 fs对应的 MOS分值;
通常, =2, ? =1.4, /max=30, / = 1,2,3,..., 30 , 那么, 按照公式( 4 ) 计算出的表格中将存储有以下内容: (1, 1.5469)、 (2, 2.4902)、 (3, 3)、 (4, 3.3409)、 (5,3.5923)、 (6,3.7886)、(7,3.9479)、 (8, 4.0805)、(9, 4.1932)、 (10, 4.2902)、 (11,4.3749)、 (12, 4.4495)、(13, 4.5156)、 (14, 4.5746)、(15, 4.6275)、 (16, 4.6752)、 (17, 4.7182)、 (18, 4.7571)、 (19, 4.7923)、 (20, 4.8242)、 (21, 4.8531)、 (22,4.8792)、 (23, 4.9027)、(24, 4.9238)、 (25,4.9426)、 (26,4.9591)、 (27,4.9734)、 (28,4.9853)、 (29, 4.9946)、 (30, 5), 其中, 括号中的前一项表 示编码帧率, 即 f, 后一项表示对应的 MOS分值;
殳最新得到的 /A为 10, 那么经查表可知其对应的焦点视频的 MOS分 值为 4.2902, 殳最新得到的 为 8, 那么经查表可知其对应的普通视频的 MOS分值为 4.0805;
或者, 也可以不通过查表的方式, 而是直接按照公式(4)或 (5)分别 计算出最新得到的 fb和 fs对应的 MOS分值。
条件(b)
所有视频的计算资源消耗不超过终端 X的处理能力 F, 即:
m-fs+n-fb-r≤F , \≤fb,fs≤L。 (6) 步驟 24: 根据最新得到的 y确定出焦点视频的最高编码帧率, 根据最新 得到的 /;确定出普通视频的最高编码帧率, 即: 用最新得到的 Λ-Δ/作为确 定出的焦点视频的最高编码帧率, 用最新得到的 y; + Δ/作为确定出的普通视 频的最高编码帧率。
上述步骤 21 24可举例说明如下:
假设初始化后的 fb = fs = 15;
之后, 令/ = 16, fs = 14;
确定 fb = 16和 fs = 14是否满足预定条件, 如果是, 则令 fb = 17, fs = 13 , 并确定 fb = 17和 = 13是否满足预定条件, 如果否, 则将 16作为确定出的 焦点视频的最高编码帧率, 将 14作为确定出的普通视频的最高编码帧率。
经过步骤 21 ~ 24的处理后, 在提高焦点视频的最高编码帧率, 使得焦点 视频具有更高的质量的同时, 又保证了焦点视频和普通视频的主观质量差距 在合理范围内, 而且将计算资源消耗控制在了终端的处理能力范围之内。
进一步地, 在上述多人视频通信中的视频编码方法实施例的基础上, 还 可包括以下步骤: 终端 X确定出已启用的焦点视频编码器数和普通视频编码 器数中的至少一个发生了变化 , 重复执行步骤 11和 12。
已启用的焦点视频编码器数和普通视频编码器数中的至少一个发生变 化, 可包括以下情况: 参与多人视频通信的终端总数增加或减少 (即有终端 加入或退出) , 或者总数未发生变化, 但发生了焦点视频到普通视频之间的 切换(如一个远程终端将终端 X产生的视频由焦点视频切换为了普通视频), 或者为上述各种情况的任意组合等。
本发明中, 能够根据焦点视频编码器数和普通视频编码器数的变化快速 有效地作出调节响应,重新确定焦点视频的编码参数和普通视频的编码参数, 从而进一步提高了整体视频通信质量。
当终端 X关闭后, 结束流程。
需要说明的是, 以上涉及到的各个公式也可以有其它选择或变形, 只要 能够达到相应的目的即可。
至此, 即完成了关于本发明方法实施例的介绍。
基于上述介绍, 图 3为本发明终端实施例的组成结构示意图。 如图 3所 示, 包括: 焦点视频编码器(如图 3所示 ESI ~ EBn )、 普通视频编码器(如 图 3所示 ~ Esm ) 以及自适应调节模块 31。
其中, 自适应调节模块 31 , 用于当终端参与到一多人视频通信中后, 分 别统计出终端中已启用的焦点视频编码器数和普通视频编码器数;
自适应调节模块 31, 还用于根据终端的处理能力以及最新的统计结果, 分别确定出焦点视频的编码参数以及普通视频的编码参数, 并通知各已启用 的焦点视频编码器和普通视频编码器分別按照最新确定出的编码参数进行视 频编码。
根据上述本发明的终端实施例, 根据其处理能力以及所启用的焦点视频 编码器数和普通视频编码器数等, 有针对性地确定其焦点视频的编码参数以 及普通视频的编码参数, 从而使得终端中的计算资源等能够得到更为合理的 利用, 进而提高了整体视频通信质量。
进一步地, 基于上述本发明的终端实施例, 自适应调节模块还可用于当 已启用的焦点视频编码器数和普通视频编码器数中的至少一个发生变化时, 重复执行自身功能。
因此, 当已启用的焦点视频编码器数和普通视频编码器数中的至少一个 发生变化, 如参与多人视频通信的终端总数增加或减少, 或总数未发生变化, 但发生了焦点视频到普通视频之间的切换, 导致焦点视频编码器数和普通视 频编码器数变化时,可重新确定焦点视频的编码参数和普通视频的编码参数, 从而进一步提高了整体视频通信质量。
所述编码参数包括: 最高编码帧率。
自适应调节模块 31中可具体包括(为简化附图, 未图示) :
第一处理单元, 用于分别统计终端中已启用的焦点视频编码器数和普通 视频编码器数, 并将统计结果发送给第二处理单元;
第二处理单元, 用于根据终端的处理能力以及最新的统计结果, 分别初 始化焦点视频的最高编码帧率 Λ和普通视频的最高编码帧率 fs, 并发送给第 三处理单元;
第三处理单元, 用于增大/ 4和降低/ s, 并确定最新得到的 Λ和 是否满 足预定条件, 如果是, 则重复执行自身功能,如果否, 则根据最新得到的 确 定出焦点视频的最高编码帧率, 根据最新得到的 /s确定出普通视频的最高编 码帧率, 并通知各已启用的焦点视频编码器和普通视频编码器分别按照最新 确定出的最高编码帧率进行视频编码;
第四处理单元, 用于当已启用的焦点视频编码器数和普通视频编码器数 中的至少一个发生变化时, 通知第一处理单元重复执行自身功能。
其中, 第二处理单元中又可进一步包括:
第一处理子单元, 用于计算 = = /(« + « ;
其中, m表示最新统计出的普通视频编码器数;
n表示最新统计出的焦点视频编码器数;
r表示分别对分辨率为 ¾>< ¾的焦点视频和分辨率为 wsxhs的普通视频进 行编码时计算资源消耗的比例因子, r = Wbxhbl(w» 或者, 通过实验测试 得到分别对分辨率为 ¾>< 的焦点视频和分辨率为 ^ xhs的普通视频进行编码 时的编码速度比, 将得到的编码速度比作为 Γ;
表示终端的处理能力, F = fpSe„,wusage, 其中, 表示终端的处理器 满负荷编码一个分辨率为 ^ X 的视频时的编码速度, wusage表示所期望的编码 处理对终端的处理器的占用率;
第二处理子单元, 用于将计算出的 Λ和 /;发送给第三处理单元。
第三处理单元中又可进一步包括:
第三处理子单元, 用于计算 /=/ + A/, =fs-Af , Δ >0, 并将计算结 果发送给第四处理子单元;
第四处理子单元, 用于根据最新得到的 /6确定出焦点视频的 MOS分值, 根据最新得到的 确定出普通视频的 MOS分值;
如果焦点视频的 MOS分值与普通视频的 MOS分值之差的绝对值小于预 定阈值 Δ ¾¾ , lm-fs + n-fb-r≤F , \≤fb,fs≤f , /max表示视频通信中可 能的最高编码帧率, 则确定最新得到的 /ft和 满足预定条件, 通知第三处理 子单元重复执行自身功能, 否则, 用最新得到的 作为确定出的焦点视 频的最高编码帧率, 用最新得到的 + Δ/作为确定出的普通视频的最高编码 帧率, 并通知各已启用的焦点视频编码器和普通视频编码器分别按照最新确 定出的最高编码帧率进行视频编码。
其中, 第四处理子单元预先按照预定公式分別计算出视频通信中可能 用到的每个编码帧率对应的 MOS分值,并以表格的形式进行存储,通过查表 分别得到最新得到的 fb和 fs对应的 MOS分值;
或者, 第四处理子单元直接按照预定公式分別计算出最新得到的 /έ和 对应的 MOS分值; 所 述预 定公 式 为 :
Figure imgf000011_0001
, 或 , MOSfps (f) = 5 - \n(f_ l f) - 其中, "和 均为计算系数。
在实际应用中, 除上面提到的外, 图 3所示终端中还会进一步包括一些 其它组成部分, 如捕获模块 32、 视频解码器(如图 3所示 D1 ~ DN )和显示 模块 33。
捕获模块 32中可进一步包括:焦点视频源捕获单元 321和普通视频源捕 获单元 322; 其中, 焦点视频源捕获单元 321将捕获到的视频数据发送给各 已启用的焦点视频编码器, 普通视频源捕获单元 322将捕获到的视频数据发 送给各已启用的普通视频编码器; 焦点视频源捕获单元 321和普通视频源捕 获单元 322捕获到的视频内容一样, 但分辨率不同。
通常, 针对每个远程终端, 都需要启用一个视频解码器, 但一个视频编 码器编码生成的码流可以发送给一个或多个远程终端。 4叚设图 3所示终端(终 端 0 )针对每个远程终端, 即终端 1 ~ N分别启用了一个视频编码器和一个视 频解码器, 那么, 终端 0中共需要启用 N个视频编码器和 N个视频解码器。
显示模块 33将各视频解码器解码得到的焦点视频和普通视频进行显示。 图 3所示终端实施例的具体工作流程请参照前述方法实施例中的相应说 明, 此处不再赘述。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通 过硬件来完成, 也可以通过程序来指令相关的硬件完成, 所述的程序可以存 储于一种计算机可读存储介质中, 上述提到的存储介质可以是只读存储器 ( Read Only Memory, ROM ) /易挥发性随机存取存储器( Ramdom Access Memory, RAM ) 、 磁盘或光盘等。
以上所述仅为本发明的较佳实施例而已, 并不用以限制本发明, 凡在本 发明的精神和原则之内, 所做的任何修改、 等同替换、 改进等, 均应包含在 本发明保护的范围之内。

Claims

权 利 要 求 书
1、 一种多人视频通信中的视频编码方法, 其特征在于, 当任一终端 X 参与到一多人视频通信中后, 进行以下处理:
A、 分别统计出自身已启用的焦点视频编码器数和普通视频编码器数; B、根据自身的处理能力以及最新的统计结果,分別确定出焦点视频的编 码参数以及普通视频的编码参数, 并通知各已启用的焦点视频编码器和普通 视频编码器分别按照最新确定出的编码参数进行视频编码。
2、 根据权利要求 1所述的方法, 其特征在于, 所述编码参数包括: 最高 编码帧率; 所述分别确定出焦点视频的编码参数以及普通视频的编码参数包 括:
bl、 根据自身的处理能力以及最新的统计结果, 分别初始化焦点视频的 最高编码帧率 /6和普通视频的最高编码帧率 fs
b2、 增大/ Α和降低 , 并确定最新得到的 Λ和/:是否满足预定条件, 如 果是, 则重复执行步骤 b2, 如果否, 则根据最新得到的/ 4确定出焦点视频的 最高编码帧率, 根据最新得到的 /s确定出普通视频的最高编码帧率。
3、 根据权利要求 2所述的方法, 其特征在于,
所述增大 /6和降低 ;包括: 令/ 4 =/6 +Δ/, fs=fs-Af , Af>0;
所述根据最新得到的 Λ确定出焦点视频的最高编码帧率, 根据最新得到 的 确定出普通视频的最高编码帧率包括:
用最新得到的 /4 - Δ/作为确定出的焦点视频的最高编码帧率, 用最新得 到的 + 作为确定出的普通视频的最高编码帧率。
4、 根据权利要求 2所述的方法, 其特征在于, 所述分別初始化焦点视频 的最高编码帧率 y;和普通视频的最高编码帧率 包括:
计算 y =/s=F/(w+".r);
其中, 表示最新统计出的普通视频编码器数;
n表示最新统计出的焦点视频编码器数;
r表示分别对分辨率为 wbxhb的焦点视频和分辨率为 wsxhs的普通视频进 行编码时计算资源消耗的比例因子, nbxhblW» 或者, 通过实验测试 得到分别对分辨率为 wbxhb的焦点视频和分辨率为 ^ xhs的普通视频进行编码 时的编码速度比, 将得到的编码速度比作为 r; 表示终端 X的处理能力, F=fps wus。 其中, 加 表示终端 X的处 理器满负荷编码一个分辨率为 ^ xhs的视频时的编码速度, v„,表示所期望的 编码处理对终端 X的处理器的占用率。
5、 根据权利要求 4所述的方法, 其特征在于, 该方法进一步包括: 预先通过实验测试得到各不同主频的处理器分别对应的 并以表格 的形式进行存储; 通过查表得到终端 X的处理器的主频对应的 fpSmc
6、 根据权利要求 4所迷的方法, 其特征在于, 所迷确定最新得到的 /6和 是否满足预定条件包括:
根据最新得到的 确定出焦点视频的主观平均意见 MOS分值,并根据最 新得到的 确定出普通视频的 MOS分值;
如果焦点视频的 MOS分值与普通视频的 MOS分值之差的绝对值小于预 定阈值 Δ α Μ, m.fs+n.fb.r≤F , \< fb,fs < , 则确定最新得到的 /6和 fs满足预定条件 表示视频通信中可能的最高编码帧率。
7、 根据权利要求 6所述的方法, 其特征在于, 所述根据最新得到的 /6确 定出焦点视频的 MOS分值, 并根据最新得到的 确定出普通视频的 MOS分 值包括:
预先按照预定公式分别计算出视频通信中可能用到的每个编码帧率对应 的 MOS分值, 并以表格的形式进行存储; 通过查表分别得到最新得到的 /6和 对应的 MOS分值;
或者, 直接按照预定公式分别计算出最新得到的 Λ和 Λ对应的 MOS 分 值。
8、根据权利要求 7所述的方法,其特征在于,所述预定公式为以下之一: MOS^ (f) = 5-a- [log ( _ ) - log( )]/? , OS i ( ) = 5-ln(/max / ); 其中, cr和9均为计算系数。
9、 根据权利要求 1-8任一所述的方法, 其特征在于, 当已启用的焦点视 频编码器数和普通视频编码器数中的至少一个发生变化时, 重复执行所述步 骤 A和所述步骤 B
10、 一种终端, 其特征在于, 包括: 焦点视频编码器、 普通视频编码器、 自适应调节模块;
所述自适应调节模块, 用于当终端参与到一多人视频通信中后, 分别统 计出终端中已启用的焦点视频编码器数和普通视频编码器数; 所述自适应调节模块,还用于根据终端的处理能力以及最新的统计结果, 分别确定出焦点视频的编码参数以及普通视频的编码参数, 并通知各已启用 的焦点视频编码器和普通视频编码器分别按照最新确定出的编码参数进行视 频编码。
1 1、 根据权利要求 10所述的终端, 其特征在于, 所述编码参数包括: 最 高编码帧率; 所迷自适应调节模块中包括:
笫一处理单元, 用于分别统计终端中已启用的焦点视频编码器数和普通 视频编码器数, 并将统计结果发送给第二处理单元;
第二处理单元, 用于根据终端的处理能力以及最新的统计结果, 分别初 始化焦点视频的最高编码帧率 和普通视频的最高编码帧率 fs, 并发送给第 三处理单元;
第三处理单元, 用于增大 /4和降低 fs, 并确定最新得到的 fb和 fs是否满 足预定条件, 如果是, 则重复执行自身功能,如果否, 则根据最新得到的/ 6确 定出焦点视频的最高编码帧率, 根据最新得到的 /s确定出普通视频的最高编 码帧率, 并通知各已启用的焦点视频编码器和普通视频编码器分别按照最新 确定出的最高编码帧率进行视频编码;
第四处理单元, 用于当已启用的焦点视频编码器数和普通视频编码器数 中的至少一个发生变化时, 通知第一处理单元重复执行自身功能。
12、 根据权利要求 1 1所述的终端, 其特征在于, 所述第二处理单元中包 括:
第一处理子单元, 用于计算/6 = = ^/ + « ;
其中, m表示最新统计出的普通视频编码器数;
n表示最新统计出的焦点视频编码器数;
r表示分别对分辨率为 wb x hb的焦点视频和分辨率为 ws x hs的普通视频进 行编码时计算资源消耗的比例因子, r = Wb x hb l(w» 或者, 通过实验测试 得到分别对分辨率为 wb x hb的焦点视频和分辨率为 ^ x hs的普通视频进行编码 时的编码速度比, 将得到的编码速度比作为 r;
F表示终端的处理能力, F = fpsenc - wusage, 其中, 加 表示终端的处理器 满负荷编码一个分辨率为 ws x hs的视频时的编码速度, wusage表示所期望的编码 处理对终端的处理器的占用率;
第二处理子单元, 用于将计算出的 y和/发送给第三处理单元。
13、 根据权利要求 12所述的终端, 其特征在于, 所述第三处理单元中包 括:
第三处理子单元, 用于计算 A=/i +A/, fs=H Af>0, 并将计算结 果发送给第四处理子单元;
笫四处理子单元, 用于根据最新得到的 /确定出焦点视频的主观平均意 见 MOS分值, 根据最新得到的 /s确定出普通视频的 MOS分值;
如杲焦点视频的 MOS分值与普通视频的 MOS分值之差的绝对值小于预 定阈值 Δ ¾ , lm.fs + n.fb.r≤F, \≤fb,fs≤f /max表示视频通信中可 能的最高编码帧率, 则确定最新得到的/ 4和 满足预定条件, 通知第三处理 子单元重复执行自身功能, 否则, 用最新得到的 Λ-Δ/作为确定出的焦点视 频的最高编码帧率, 用最新得到的 + Δ/作为确定出的普通视频的最高编码 帧率, 并通知各已启用的焦点视频编码器和普通视频编码器分别按照最新确 定出的最高编码帧率进行视频编码。
14、 根据权利要求 13所述的终端, 其特征在于,
第四处理子单元预先按照预定公式分別计算出视频通信中可能用到的每 个编码帧率对应的 MOS分值,并以表格的形式进行存储,通过查表分别得到 最新得到的 /4和/:对应的 MOS分值;
或者, 第四处理子单元直接按照预定公式分別计算出最新得到的/ 4和 对应的 MOS分值;
所 述预 定公 式 为 : ^¾^/ 5-«.[1(^/皿;)-1(^/;)] , 或 '
MOSfps(f) = 5-\n(f_lf); 其中, "和 均为计算系数。
15、 根据权利要求 10-14任一所述的终端, 其特征在于, 所述自适应调 节模块, 还用于当已启用的焦点视频编码器数和普通视频编码器数中的至少 一个发生变化时, 重复执行自身功能。
16、 一种计算机存储介质, 存储有计算机可执行程序, 其特征在于, 所 述计算机可执行程序用于执行权利要求 1-9任一所述的多人视频通信中的视 频编码方法。
PCT/CN2013/070833 2012-02-22 2013-01-22 一种多人视频通信中的视频编码方法以及终端 WO2013123839A1 (zh)

Priority Applications (10)

Application Number Priority Date Filing Date Title
RU2014138120/07A RU2570355C1 (ru) 2012-02-22 2013-01-22 Способ и терминал для кодирования видеоданных при обмене видеоданными между множеством участников
CA2865045A CA2865045C (en) 2012-02-22 2013-01-22 Method and terminal for video encoding in multi-party video communication
EP13752176.1A EP2819405B1 (en) 2012-02-22 2013-01-22 Video coding method in multi-person video communications and terminal
KR1020147025280A KR101629226B1 (ko) 2012-02-22 2013-01-22 다자간 비디오 통신에서의 비디오 코딩 방법 및 단말기
BR112014020225-7A BR112014020225B1 (pt) 2012-02-22 2013-01-22 Método de codificação de vídeo em comunicações de vídeo com múltiplas pessoas e terminal
AP2014007931A AP2014007931A0 (en) 2012-02-22 2013-01-22 Video coding method in multiperson video communications and terminal
NZ630215A NZ630215A (en) 2012-02-22 2013-01-22 Video coding method in multi-person video communications and terminal
US14/378,091 US9332267B2 (en) 2012-02-22 2013-01-22 Method and terminal for video encoding in multi-party video communication
SG11201404904VA SG11201404904VA (en) 2012-02-22 2013-01-22 Video coding method in multi-person video communications and terminal
PH12014501820A PH12014501820A1 (en) 2012-02-22 2014-08-12 Video coding method in multi-person video communications and terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201210040481XA CN102547214B (zh) 2012-02-22 2012-02-22 一种多人视频通信中的视频编码方法以及终端
CN201210040481.X 2012-02-22

Publications (1)

Publication Number Publication Date
WO2013123839A1 true WO2013123839A1 (zh) 2013-08-29

Family

ID=46353028

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/070833 WO2013123839A1 (zh) 2012-02-22 2013-01-22 一种多人视频通信中的视频编码方法以及终端

Country Status (14)

Country Link
US (1) US9332267B2 (zh)
EP (1) EP2819405B1 (zh)
KR (1) KR101629226B1 (zh)
CN (1) CN102547214B (zh)
AP (1) AP2014007931A0 (zh)
BR (1) BR112014020225B1 (zh)
CA (1) CA2865045C (zh)
CL (1) CL2014002194A1 (zh)
MY (1) MY168126A (zh)
NZ (1) NZ630215A (zh)
PH (1) PH12014501820A1 (zh)
RU (1) RU2570355C1 (zh)
SG (1) SG11201404904VA (zh)
WO (1) WO2013123839A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10575047B2 (en) * 2014-01-23 2020-02-25 Sony Corporation Decoding apparatus, decoding method, encoding apparatus, and encoding method

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104254018B (zh) * 2014-03-19 2018-04-03 深圳深讯和科技有限公司 多媒体播放中的视频帧处理方法和装置
CN105657321A (zh) * 2015-11-09 2016-06-08 乐视致新电子科技(天津)有限公司 视频清晰度自适应调节方法、装置及终端
CN108076301A (zh) * 2016-11-11 2018-05-25 联芯科技有限公司 VoLTE视频多方电话的视频处理方法和系统
KR20190033849A (ko) * 2017-09-22 2019-04-01 삼성에스디에스 주식회사 다자간 회의 제공 장치 및 상기 장치에서의 인코더 할당 방법
US11025964B2 (en) 2019-04-02 2021-06-01 Wangsu Science & Technology Co., Ltd. Method, apparatus, server, and storage medium for generating live broadcast video of highlight collection
CN109862388A (zh) * 2019-04-02 2019-06-07 网宿科技股份有限公司 直播视频集锦的生成方法、装置、服务器及存储介质
US11470300B2 (en) * 2020-05-08 2022-10-11 Tencent America LLC Event-based trigger interval for signaling of RTCP viewport for immersive teleconferencing and telepresence for remote terminals
CN112511788B (zh) * 2020-11-27 2022-04-01 厦门亿联网络技术股份有限公司 一种视频会议的视频传输控制方法及视频传输系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1380795A (zh) * 2001-04-06 2002-11-20 松下电器产业株式会社 多点播放会议装置及多点播放会议程序
CN1788494A (zh) * 2003-02-21 2006-06-14 红河资源公司 用于大量视频序列传输的系统和方法
US20080309751A1 (en) * 2007-06-12 2008-12-18 Quanta Computer Inc. High-definition video conference system and method
CN101588252A (zh) * 2008-05-23 2009-11-25 华为技术有限公司 一种多点会议的控制方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8184720B2 (en) * 2005-04-11 2012-05-22 Tubitak Biltien Optimal video adaptation for resource constrained mobile devices based on subjective utility models
RU2378790C1 (ru) * 2005-09-27 2010-01-10 Квэлкомм Инкорпорейтед Методики масштабируемости на основе информации содержимого
US7456760B2 (en) * 2006-09-11 2008-11-25 Apple Inc. Complexity-aware encoding
US8345083B2 (en) * 2007-07-31 2013-01-01 Cisco Technology, Inc. Dynamic management of picture quality in a video conference with diversified constraints
CN101365125B (zh) * 2008-09-27 2012-03-21 腾讯科技(深圳)有限公司 多路视频通信方法与系统
CN101742223B (zh) * 2010-01-19 2012-08-22 华平信息技术股份有限公司 视频流带宽智能调节方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1380795A (zh) * 2001-04-06 2002-11-20 松下电器产业株式会社 多点播放会议装置及多点播放会议程序
CN1788494A (zh) * 2003-02-21 2006-06-14 红河资源公司 用于大量视频序列传输的系统和方法
US20080309751A1 (en) * 2007-06-12 2008-12-18 Quanta Computer Inc. High-definition video conference system and method
CN101588252A (zh) * 2008-05-23 2009-11-25 华为技术有限公司 一种多点会议的控制方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2819405A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10575047B2 (en) * 2014-01-23 2020-02-25 Sony Corporation Decoding apparatus, decoding method, encoding apparatus, and encoding method

Also Published As

Publication number Publication date
KR101629226B1 (ko) 2016-06-10
PH12014501820B1 (en) 2014-11-24
CA2865045A1 (en) 2013-08-29
BR112014020225A8 (pt) 2017-07-11
PH12014501820A1 (en) 2014-11-24
CN102547214A (zh) 2012-07-04
CL2014002194A1 (es) 2015-02-27
EP2819405A4 (en) 2015-03-04
BR112014020225A2 (zh) 2017-06-20
KR20140128420A (ko) 2014-11-05
CN102547214B (zh) 2013-05-29
EP2819405B1 (en) 2021-08-11
SG11201404904VA (en) 2014-10-30
RU2570355C1 (ru) 2015-12-10
US9332267B2 (en) 2016-05-03
EP2819405A1 (en) 2014-12-31
AP2014007931A0 (en) 2014-09-30
US20150215627A1 (en) 2015-07-30
NZ630215A (en) 2015-05-29
BR112014020225B1 (pt) 2019-09-17
CA2865045C (en) 2016-10-18
MY168126A (en) 2018-10-11

Similar Documents

Publication Publication Date Title
WO2013123839A1 (zh) 一种多人视频通信中的视频编码方法以及终端
US11089305B2 (en) Video frame coding method during scene change, terminal and storage medium
CN113490055B (zh) 数据处理方法和装置
CN108347580B (zh) 一种处理视频帧数据的方法及电子设备
WO2014054325A1 (ja) 符号化制御装置および符号化制御方法
WO2013085585A1 (en) Syntax extension of adaptive loop filter in hevc
US9344680B2 (en) Server and non-transitory computer readable medium storing program for remote conference
CN105516638A (zh) 一种视频通话方法、装置和系统
US9420028B2 (en) Remote conference system and non-transitory computer readable medium storing program for remote conference
CN113301342A (zh) 视频编码方法、网络直播方法、装置和终端设备
CN105208388B (zh) 视频通讯中动态调整编码帧率的方法和系统
CN114245196B (zh) 一种录屏推流方法、装置、电子设备及存储介质
CN111617466A (zh) 编码格式的确定方法、装置及云游戏的实现方法
CN107493478B (zh) 编码帧率设置方法及设备
CN107733732B (zh) 一种判断网络状况的方法及装置
WO2018192518A1 (zh) 数据处理方法、装置及存储介质
CN105992055B (zh) 一种视频解码的方法及装置
CN114945097B (zh) 视频流处理方法及装置
CN106254873B (zh) 一种视频编码方法及视频编码装置
US20230247069A1 (en) Systems and Methods for Adaptive Video Conferencing
CN115988269A (zh) 一种视频播放方法、装置、系统、电子设备和存储介质
CN111510703B (zh) 一种视频播放方法、装置及系统
CN112312370B (zh) 切换门限值处理方法、装置
CN114222182A (zh) 一种视频播放方法及装置
CN113965714B (zh) 视频流的处理方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13752176

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 14378091

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 12014501820

Country of ref document: PH

Ref document number: 2013752176

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014002194

Country of ref document: CL

Ref document number: P879/2014

Country of ref document: AE

ENP Entry into the national phase

Ref document number: 2865045

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20147025280

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: IDP00201405612

Country of ref document: ID

WWE Wipo information: entry into national phase

Ref document number: 2014138120

Country of ref document: RU

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112014020225

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112014020225

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20140815