WO2021164670A1 - 视频编码方法、装置、设备及存储介质 - Google Patents

视频编码方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021164670A1
WO2021164670A1 PCT/CN2021/076351 CN2021076351W WO2021164670A1 WO 2021164670 A1 WO2021164670 A1 WO 2021164670A1 CN 2021076351 W CN2021076351 W CN 2021076351W WO 2021164670 A1 WO2021164670 A1 WO 2021164670A1
Authority
WO
WIPO (PCT)
Prior art keywords
encoding
strategy
preset
speed
coding
Prior art date
Application number
PCT/CN2021/076351
Other languages
English (en)
French (fr)
Inventor
张文忠
Original Assignee
百果园技术(新加坡)有限公司
张文忠
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百果园技术(新加坡)有限公司, 张文忠 filed Critical 百果园技术(新加坡)有限公司
Publication of WO2021164670A1 publication Critical patent/WO2021164670A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone

Definitions

  • This application relates to the field of video processing, such as video encoding methods, devices, equipment, and storage media.
  • Video data contains a large amount of image and sound information. In some application scenarios, it is usually necessary to encode the video data during video transmission. Take instant messaging products as an example. Video chat or live video is an indispensable basic function. The quality of video coding schemes directly affects the effect and experience of video chat or live video.
  • This application provides a video encoding method, device, equipment, and storage medium, which can optimize a video encoding scheme.
  • a video encoding method which includes:
  • a video encoding device which includes:
  • the first encoding module is configured to encode the first original video data by using a preset encoder and a first encoding strategy within a preset time period;
  • a duration statistics module configured to count the encoding duration corresponding to each original video frame in the first original video data, where the encoding strategy is related to the encoding complexity
  • a strategy adjustment module configured to adjust the first coding strategy according to the coding duration to obtain a second coding strategy
  • the second encoding module is configured to use the preset encoder and adopt the second encoding strategy to encode second original video data after the first original video data.
  • a computer device including a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor implements the video encoding method as provided in the embodiment of the present application when the processor executes the computer program .
  • a computer-readable storage medium which stores a computer program, and when the computer program is executed by a processor, the video encoding method as provided in the embodiment of the present application is implemented.
  • FIG. 1 is a schematic flowchart of a video encoding method provided by an embodiment of the application
  • FIG. 2 is a schematic flowchart of another video encoding method provided by an embodiment of this application.
  • FIG. 3 is a schematic diagram of switching between encoding quality and encoding speed provided by an embodiment of the application
  • FIG. 4 is a structural block diagram of a video encoding device provided by an embodiment of the application.
  • Fig. 5 is a structural block diagram of a computer device provided by an embodiment of the application.
  • FIG. 1 is a schematic flowchart of a video encoding method provided by an embodiment of the application.
  • the method may be executed by a video encoding device, where the device may be implemented by software and/or hardware, and may generally be integrated in a computer device. As shown in Figure 1, the method includes the following steps.
  • Step 101 Use a preset encoder to encode first original video data using a first encoding strategy within a preset duration, and count the encoding duration corresponding to each original video frame in the first original video data, where the encoding
  • the strategy is related to the coding complexity.
  • the computer devices in the embodiments of the present application may include mobile devices such as mobile phones, tablet computers, and notebook computers, and may also include other types of computer devices, which are not limited.
  • video data needs to be encoded before transmission.
  • the device of the call party collects video information through a device such as a camera and generates original video data. After encoding the original video data, the encoded video data is sent to the call party through the network.
  • the embodiment of the present application does not limit the type of the preset encoder, and can be selected according to actual needs.
  • h.26x series encoders or vpx series encoders can be used.
  • the h.26x series encoders are open source toll encoder series formulated by the International Telecommunication Union (ITU).
  • the mainstream ones are h.264, h.265, and the newer h.266;
  • vpx series encoders It is an open source free encoder series developed by Google.
  • the mainstream ones are vp8, vp9, and the newer vp10 (av1).
  • a set of fixed encoding parameters is generally set for the encoder, and then the encoder is called to perform encoding.
  • the performance difference is large, a set of fixed coding parameters can not be well adapted to the actual coding capabilities of different devices, in addition, for the same device
  • the encoding capabilities of the device will also vary.
  • the flexibility of the fixed encoding parameter scheme is poor, and the encoding performance and encoding quality of the device cannot be well balanced, making the video image The quality and smoothness of the video cannot be well balanced.
  • the length of the preset duration can be freely set according to actual conditions, for example, it can be 10 seconds.
  • the start timing of the preset duration can also be set freely according to the actual situation. For example, it can be the moment when the video encoding event is triggered, or it can be the end time of the last preset duration (the preset duration can be understood as the preset period at this time) .
  • the trigger condition of the video encoding event can be set according to the actual scene, for example, it can be that the video call successfully establishes a connection.
  • the original video data may include, for example, video image data collected by an image capture device such as a camera configured in a computer device, and may include multiple original video frames.
  • the first original video data may refer to original video data that needs to be encoded within a preset time period.
  • a preset encoding index of the preset encoder may be set, and the preset encoding index includes at least one of resolution, bit rate, and frame rate, and optional ,
  • the preset coding index can include resolution, bit rate and frame rate.
  • the coding strategy in the embodiments of the present application is related to coding complexity.
  • the coding complexity can also be understood as coding strength, which can be measured by coding parameters such as coding quality or coding speed, and can be determined according to the type or configuration of a preset encoder. Generally, the higher the coding quality, the higher the coding complexity; the slower the coding speed, the higher the coding complexity.
  • the encoding duration corresponding to each original video frame in the first original video data may be counted by a timer or the like.
  • the encoding duration can be understood as the encoding time, that is, the time consumed for encoding the original video frame.
  • the timer is started, when the preset encoder is detected to complete the encoding operation, the timing is ended, and the time obtained by the timer is recorded as corresponding to the original video frame Encoding duration.
  • Step 102 Adjust the first coding strategy according to the coding duration to obtain a second coding strategy.
  • the current encoding situation can be measured by referring to the encoding duration corresponding to each original video frame when the first encoding strategy is adopted. It can be understood as evaluating the current actual encoding capability of the computer device.
  • the first encoding strategy can be adjusted with reference to the average encoding duration of a single frame of the video frame, or the first encoding strategy can be adjusted with reference to the duration of the preset encoder in the encoding state per unit time.
  • Step 103 Use the preset encoder and adopt the second encoding strategy to encode second original video data after the first original video data.
  • the obtained second encoding strategy is more suitable for the current encoding capability of the device, balancing the encoding performance and encoding quality of the device, and also making the video quality and the smoothness of the video better balanced .
  • the second original video data may include all original video data after the first original video data until the end of the video encoding event; the second original video data may include the original video data within a preset time period after the first original video data.
  • the encoding strategy can be adjusted periodically, for example, continue to use the preset encoder to apply the second encoding strategy to the second original video after the first original video data within a preset period of time.
  • the data is encoded, and the encoding duration corresponding to each original video frame in the second original video data is counted.
  • the second encoding strategy is adjusted according to the encoding duration at this time to obtain the third encoding strategy, and the preset encoder adopts The third coding strategy continues coding.
  • a preset encoder is used to encode first original video data using a first encoding strategy within a preset time period, and statistics are calculated for each original video frame in the first original video data.
  • Encoding duration where the encoding strategy is related to the encoding complexity.
  • the first encoding strategy is adjusted according to the encoding duration to obtain the second encoding strategy, and the second encoding strategy after the first original video data is determined by the preset encoder.
  • the original video data is encoded.
  • the adjusting the first encoding strategy according to the encoding duration to obtain the second encoding strategy includes: determining, according to the encoding duration, that the preset encoder is in an encoding state per unit time When the first time length is less than the first preset threshold, the first coding strategy is adjusted so that the coding complexity corresponding to the second coding strategy is greater than that corresponding to the first coding strategy Encoding complexity.
  • the unit time may be, for example, 1 second.
  • the first preset threshold can be set according to actual conditions. When the first duration is less than the first preset threshold, it means that the encoding speed is faster at this time, the preset encoder idle time per unit time is more, and the encoding performance of the device is excessive.
  • the first encoding strategy can be adjusted to Increasing the encoding complexity is conducive to improving the video quality and enhancing the video quality.
  • the first duration there may be multiple calculation methods for the first duration, for example, calculating the quotient of the sum of the encoding durations and the preset duration, and determining that the preset encoder is in the encoding state per unit time according to the quotient
  • the product of the average value of the encoding duration and the current frame rate of the preset encoder is calculated, and the first time that the preset encoder is in the encoding state in the unit time is determined according to the product For a long time.
  • the preset duration is C
  • the sum of the encoding durations of all original video frames in the first original video data is t
  • the first duration can be expressed as t/C*1 second, for example, t is 7 seconds and C is 10 seconds.
  • the quotient is 0.7, and it can be known that the first duration is 0.7 seconds, that is, within the unit duration of 1 second, the preset encoder is in the encoding state for 0.7 seconds.
  • the total number of original video frames in the first original video data is n
  • the current frame rate is the number of frames per second (Frames Per Second, fps)
  • the first duration can be expressed as (t/n)*fps *1 second, for example, t is 7 seconds, n is 300 frames, the average is 7 seconds/300 frames, and the fps is 30 frames/second, then the product is 0.7. It can be seen that the first duration is 0.7 seconds, that is, Within the unit time of 1 second, the preset encoder is in the encoding state for 0.7 seconds.
  • the second preset threshold is less than the unit time, and the value can be set according to actual conditions.
  • the first duration is greater than the second preset threshold, it means that the encoding speed is slower, the idle time of the preset encoder is less in unit time, and the encoding performance of the device is insufficient.
  • the first encoding strategy can be adjusted to Reducing coding complexity is conducive to improving video fluency.
  • the encoding strategy includes an encoding quality gear and an encoding speed, and one encoding quality gear corresponds to at least one encoding speed. This setting can adjust the coding strategy in two dimensions, improve the diversity of the coding strategy, and make the coding strategy more reasonable.
  • the first coding strategy when the first duration is less than a first preset threshold, the first coding strategy is adjusted so that the coding complexity corresponding to the obtained second coding strategy is greater than that of the first coding strategy.
  • the encoding complexity corresponding to the encoding strategy includes: determining a first adjustment coefficient according to the first duration and the first preset threshold; adjusting the first encoding quality in the first encoding strategy according to the first adjustment coefficient The gear and the first encoding speed to obtain the second encoding quality gear and the second encoding speed; wherein the second encoding speed is less than the first encoding speed, and/or the second encoding quality gear is high In the first encoding quality gear.
  • the first adjustment coefficient may be determined according to the quotient of the first preset threshold and the first duration
  • the second encoding speed may be determined according to the difference between the first encoding speed and the first adjustment coefficient.
  • the encoding quality gear is increased, that is, the second encoding quality gear is higher than the first encoding quality gear; otherwise, the encoding quality gear may not be adjusted and the first encoding quality gear may be maintained.
  • One encoding quality gear is unchanged, that is, the second encoding quality gear is equal to the first encoding quality gear.
  • There may also be other ways of determining the first adjustment coefficient, the second encoding quality gear, and the second encoding speed which are not limited in the embodiment of the present application.
  • the adjusting the first encoding quality gear and the first encoding speed in the first encoding strategy according to the first adjustment coefficient to obtain the second encoding quality gear and the second encoding speed includes : Determine a second encoding speed according to the first adjustment coefficient and the first encoding speed in the first encoding strategy; when the second encoding speed is less than the minimum encoding speed in the first encoding quality gear, It is determined that the second encoding quality gear is an encoding quality gear higher than the first encoding quality gear, and the second encoding speed is updated to the first preset encoding speed under the second encoding quality gear .
  • the first preset encoding speed may be the highest encoding speed in the second encoding quality gear, or other encoding speeds, which can be set according to actual needs.
  • the first coding strategy when the first duration is greater than a second preset threshold, the first coding strategy is adjusted so that the coding complexity corresponding to the obtained second coding strategy is less than that of the first coding strategy.
  • the encoding complexity corresponding to the encoding strategy includes: determining a second adjustment coefficient according to the first duration and the second preset threshold; adjusting the first encoding quality in the first encoding strategy according to the second adjustment coefficient The gear and the first encoding speed to obtain the second encoding quality gear and the second encoding speed; wherein the second encoding speed is greater than the first encoding speed, and/or the second encoding quality gear is low In the first encoding quality gear.
  • the second adjustment coefficient may be determined according to the quotient of the first duration and the second preset threshold, and the second encoding speed may be determined according to the sum of the first encoding speed and the second adjustment coefficient.
  • the second adjustment coefficient exceeds the second threshold, the encoding quality gear is reduced, that is, the second encoding quality gear is lower than the first encoding quality gear; otherwise, the encoding quality gear may not be adjusted and the first encoding quality gear may be maintained.
  • One encoding quality gear is unchanged, that is, the second encoding quality gear is equal to the first encoding quality gear.
  • There may also be other ways of determining the second adjustment coefficient, the second encoding quality gear, and the second encoding speed which are not limited in the embodiment of the present application.
  • the adjusting the first encoding quality gear and the first encoding speed in the first encoding strategy according to the second adjustment coefficient includes: according to the second adjustment coefficient and the first The first encoding speed in the encoding strategy determines the second encoding speed; when the second encoding speed exceeds the maximum encoding speed in the first encoding quality gear, it is determined that the second encoding quality gear is lower than the first encoding speed.
  • the second preset encoding speed may be the lowest encoding speed in the second encoding quality gear, or other encoding speeds, which can be set according to actual requirements.
  • the process further includes: detecting whether the network condition meets the preset adjustment condition, and When the network condition meets the preset adjustment condition, adjust the preset encoding index in the preset encoder; re-use the adjusted preset encoder within the preset time period and adopt the first encoding strategy to the first original Video data is encoded.
  • the preset encoding index in the preset encoder can be adaptively adjusted according to the network status, and the video quality and fluency can be dynamically adjusted.
  • the network condition includes network speed
  • the preset encoding index includes at least one of resolution, code rate, and frame rate.
  • the video encoding gear When the network speed is slow (such as the current network speed is less than the first network speed threshold), reduce the video encoding gear, such as reducing at least one of the resolution, bit rate and frame rate; when the network speed is relatively fast (such as the current network speed) When the speed is greater than the second network speed threshold, and the second network speed threshold is greater than the first network speed threshold), increase the video encoding gear, such as increasing at least one of resolution, bit rate, and frame rate.
  • Fig. 2 is a schematic flow diagram of another video encoding method provided by an embodiment of the application.
  • the method is described by using a vpx series encoder for encoding in a video call scene as an example.
  • the encoding complexity is mainly achieved through two parameters, namely, the encoding quality gear (deadline) and the encoding speed (cpuUsed).
  • deadline is used to specify the quality level of encoding
  • vpx series generally have three grades, namely REAL_TIME (real-time), GOOD_QUALITY (better quality), BEST_QUALITY (best quality); cpuUsed is used to specify the encoding speed under a quality level .
  • the range of cpuUsed in the REAL_TIME file can be limited to the range of [-16, -1], the smaller the value, the lower the coding complexity, that is, the faster the speed; the range of cpuUsed in the GOOD_QUALITY file can be limited to [0 , 5], the smaller the value, the higher the coding complexity, that is, the slower the speed; cpuUsed in the BEST_QULAITY file is fixed at 0 and cannot be adjusted.
  • the method may include the following steps.
  • Step 201 It is detected that a video call event is triggered.
  • the coding strategy adjustment period C (that is, the preset duration) can be initialized, the lower threshold T1 (that is, the first preset threshold) and the upper threshold T2 (that is, the second Preset threshold).
  • T1 may be 0.5 seconds, for example, and T2 may be 0.9 seconds, for example.
  • Step 202 Use the vpx encoder to encode the original video data using the current encoding strategy within a preset time period, and count the encoding time length corresponding to each original video frame in the original video data.
  • the current encoding strategy may include the preset encoding quality gear and the preset encoding speed, that is, using the vpx encoder to use the preset encoding quality gear and the preset within the preset time period.
  • Encoding speed encodes the original video data.
  • the original video data such as yuv format data, can be collected through the device's camera, and the original video data is sent to the vpx encoder.
  • the encoding time is calculated and accumulated to t, and the number of frames n is accumulated by 1.
  • step 202 may also include detecting whether the network condition meets the preset adjustment condition, and if so, adjusting the preset coding index in the preset encoder, and re-executing step 202; otherwise, Proceed to step 202.
  • the preset encoding index may include resolution, code rate, and frame rate, and any one or more of them may be adjusted during adjustment.
  • Step 203 Determine, according to the encoding duration, a first duration during which the preset encoder is in the encoding state per unit time.
  • Step 204 Determine whether the first duration is less than a first preset threshold, and if so, perform step 205; otherwise, perform step 206.
  • Step 205 Adjust the current coding strategy to increase the coding complexity corresponding to the coding strategy, obtain a new current coding strategy, and perform step 208.
  • the first adjustment coefficient is determined according to the first time length and the first preset threshold, and the first encoding quality gear and the first encoding speed in the current encoding strategy are adjusted according to the first adjustment coefficient to obtain the second encoding quality gear And the second encoding speed, the second encoding speed is less than the first encoding speed, and the second encoding quality gear is higher than or equal to the first encoding quality gear.
  • the first encoding quality gear and the first encoding speed correspond to the current encoding strategy
  • the second encoding quality gear and the second encoding speed correspond to the adjusted new encoding strategy.
  • the second encoding speed can be determined according to the first adjustment coefficient and the first encoding speed.
  • the second encoding quality gear is determined to be higher than the first encoding quality And update the second encoding speed to the first preset encoding speed under the second encoding quality gear.
  • cpuUsed is switched to -1, if the coding complexity needs to be increased, the deadline is switched to GOOD_QUALITY, and cpuUsed is also switched to 2, where 2 can be set according to actual needs.
  • BEST_QULAITY If the first encoding quality gear is BEST_QULAITY, it can be considered that the encoding complexity has reached the maximum and cannot be increased.
  • Step 206 Determine whether the first duration is greater than the second preset threshold, and if so, perform step 207; otherwise, perform step 208.
  • Step 207 Adjust the current coding strategy to reduce the coding complexity corresponding to the coding strategy and obtain a new coding strategy.
  • the second adjustment coefficient is determined according to the first time length and the second preset threshold, and the first encoding quality gear and the first encoding speed in the first encoding strategy are adjusted according to the second adjustment coefficient to obtain the second encoding quality gear
  • the second encoding speed is greater than the first encoding speed
  • the second encoding quality gear is lower than or equal to the first encoding quality gear.
  • the second encoding speed can be determined according to the second adjustment coefficient and the first encoding speed.
  • cpuUsed is switched to 5
  • the deadline is switched to REAL_TIME
  • cpuUsed is also switched to -4, where -4 can be set according to actual needs.
  • the deadline can be switched to GOOD_QUALITY, and the cpuUsed can remain unchanged.
  • FIG. 3 is a schematic diagram of switching between encoding quality and encoding speed provided by an embodiment of the application, and the foregoing switching process can be understood in conjunction with FIG. 3.
  • RT is REAL_TIME
  • GOOD is GOOD_QUALITY
  • BEST is BEST_QULAITY.
  • Step 208 Judge whether the video call is over, if yes, end the process; otherwise, return to step 202.
  • the vpx series encoder is used to encode the call video, and the encoding quality gear and encoding speed are dynamically adjusted periodically according to the average encoding time per unit time to achieve Dynamically adjust the encoding strength.
  • the video quality is improved by increasing the encoding complexity.
  • the video fluency is improved by reducing the encoding complexity, and the flexibility of video encoding is improved.
  • Video call experience is used to encode the call video, and the encoding quality gear and encoding speed are dynamically adjusted periodically according to the average encoding time per unit time to achieve Dynamically adjust the encoding strength.
  • FIG 4 is a structural block diagram of a video encoding device provided by an embodiment of the application.
  • the device can be implemented by software and/or hardware, generally can be integrated in a computer device, and can perform video encoding by executing a video encoding method. As shown in Figure 4, the device includes the following modules.
  • the first encoding module 401 is configured to encode the first original video data by using a preset encoder and a first encoding strategy within a preset duration;
  • the duration statistics module 402 is configured to count the encoding duration corresponding to each original video frame in the first original video data, where the encoding strategy is related to the encoding complexity;
  • the strategy adjustment module 403 is configured to adjust the first encoding strategy according to the encoding duration to obtain a second encoding strategy
  • the second encoding module 404 is configured to use the preset encoder and adopt the second encoding strategy to encode second original video data after the first original video data.
  • the video encoding device uses a preset encoder to encode first original video data within a preset time period and adopts a first encoding strategy, and calculates the encoding corresponding to each original video frame in the first original video data.
  • the encoding strategy is related to the encoding complexity.
  • the first encoding strategy is adjusted according to the encoding duration to obtain the second encoding strategy.
  • the preset encoder adopts the second encoding strategy for the second original video data after the first original video data.
  • Video data is encoded.
  • the embodiment of the present application provides a computer device, and the video encoding device provided in the embodiment of the present application can be integrated in the computer device.
  • Fig. 5 is a structural block diagram of a computer device provided by an embodiment of the application.
  • the computer device 500 includes a memory 501, a processor 502, and a computer program that is stored on the memory 501 and can run on the processor 502.
  • the processor 502 implements the video encoding method provided in the embodiment of the present application when the computer program is executed.
  • An embodiment of the present application also provides a storage medium containing computer-executable instructions, which are used to execute the video encoding method provided in the embodiments of the present application when the computer-executable instructions are executed by a computer processor.
  • the storage medium includes various types of memory devices or storage devices.
  • storage medium is intended to include: installation media; computer system memory or random access memory; non-volatile memory, etc.
  • the video encoding apparatus, equipment, and storage medium provided in the foregoing embodiments can execute the video encoding method provided in any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method.
  • the video encoding method provided in any embodiment of the present application can execute the video encoding method provided in any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例公开了视频编码方法、装置、设备及存储介质。其中，该视频编码方法包括：在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；根据编码时长对第一编码策略进行调整，得到第二编码策略；利用预设编码器采用第二编码策略对第一原始视频数据之后的第二原始视频数据进行编码。

Description

视频编码方法、装置、设备及存储介质
本申请要求在2020年02月20日提交中国专利局、申请号为202010106043.3的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理领域，例如涉及视频编码方法、装置、设备及存储介质。
背景技术
随着多媒体信息技术的不断发展，视频信息大量涌现。视频数据作为一种表达信息的综合媒体，已成为现实生活中一个重要的信息载体。
视频数据中包含了大量的图像以及声音信息，在一些应用场景中，在进行视频传输时，通常需要对视频数据进行编码。以即时通信产品为例，视频聊天或视频直播等是不可或缺的基本功能，视频编码方案的优劣直接影响视频聊天或视频直播的效果及体验。
发明内容
本申请提供了视频编码方法、装置、设备及存储介质，可以优化视频编码方案。
提供了一种视频编码方法，该方法包括：
在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；
根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略；
利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
还提供了一种视频编码装置，该装置包括：
第一编码模块，设置为在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码；
时长统计模块，设置为统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；
策略调整模块，设置为根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略；
第二编码模块，设置为利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
还提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如本申请实施例提供的视频编码方法。
还提供了一种计算机可读存储介质，存储有计算机程序，该计算机程序被处理器执行时实现如本申请实施例提供的视频编码方法。
附图说明
图1为本申请实施例提供的一种视频编码方法的流程示意图；
图2为本申请实施例提供的又一种视频编码方法的流程示意图；
图3为本申请实施例提供的编码质量与编码速度的切换示意图；
图4为本申请实施例提供的一种视频编码装置的结构框图；
图5为本申请实施例提供的一种计算机设备的结构框图。
具体实施方式
下面结合附图和实施例对本申请进行说明。此处所描述的实施例仅仅用于解释本申请，而非对本申请的限定。为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。此外，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
图1为本申请实施例提供的一种视频编码方法的流程示意图，该方法可以由视频编码装置执行，其中该装置可由软件和/或硬件实现，一般可集成在计算机设备中。如图1所示，该方法包括以下步骤。
步骤101、在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关。
示例性的，本申请实施例中的计算机设备可包括手机、平板电脑以及笔记 本电脑等移动设备，也可包括其他类型的计算机设备，不做限定。在许多涉及视频传输的应用场景中，例如视频通话或视频直播等，需要对视频数据进行编码后再进行传输。以视频通话为例，通话一方的设备通过摄像头等装置采集视频信息并生成原始视频数据，并对原始视频数据进行编码后，将编码后的视频数据通过网络发送至通话对方。
本申请实施例对预设编码器的类型不做限定，可根据实际需求进行选择。例如，对于一些对实时性要求较高的应用场景来说，比如实时视频聊天，一般可采用h.26x系列编码器或vpx系列编码器。其中，h.26x系列编码器是国际电信联盟(International Telecommunication Union，ITU)制定的开源收费编码器系列，主流的有h.264、h.265、以及较新的h.266；vpx系列编码器是谷歌(google)公司研发的开源免费编码器系列，主流的有vp8、vp9、以及较新的vp10(av1)。相关技术中，在诸如分辨率、码率以及帧率等编码指标不变的情况下，一般会针对编码器设置一套固定的编码参数，然后调用编码器进行编码。然而，对于不同的计算机设备来说，例如对于不同的移动设备来说，性能差异较大，一套固定的编码参数不能很好的适应不同设备的实际编码能力，另外，对于同一个设备来说，在实际编码过程中，针对不同的视频数据以及设备运行情况，设备的编码能力也是会存在变化的，固定编码参数的方案灵活性差，不能很好地平衡设备编码性能和编码质量，使得视频画质以及视频流畅度也不能得到较好的平衡。
本申请实施例中，预设时长的长度可以根据实际情况自由设置，例如可以是10秒。预设时长的开始时机也可以根据实际情况自由设置，例如，可以是视频编码事件被触发的时刻，也可以是上一个预设时长的结束时刻(此时预设时长可理解为预设周期)。视频编码事件的触发条件可根据实际场景设定，例如，可以是视频通话成功建立连接。原始视频数据例如可以包括计算机设备配置的摄像头等图像采集装置所采集的视频图像数据，其中可包括多个原始视频帧。第一原始视频数据可以指在预设时长内需要进行编码的原始视频数据。
示例性的，在开始对第一原始视频数据进行编码前，可以设置预设编码器的预设编码指标，预设编码指标包括分辨率、码率和帧率中的至少一项，可选的，预设编码指标可包括分辨率、码率和帧率。
本申请实施例中的编码策略与编码复杂度相关，编码复杂度也可以理解为编码强度，可以采用编码质量或编码速度等编码参数来衡量，可根据预设编码器的类型或配置来决定。一般的，编码质量越高，可说明编码复杂度越高；编码速度越慢，可说明编码复杂度越高。
示例性的，在利用预设编码器采用第一编码策略对第一原始视频数据进行 编码的过程中，可以通过计时器计时等方式统计第一原始视频数据中每个原始视频帧对应的编码时长，此处编码时长可理解为编码耗时，也即对原始视频帧进行编码所消耗的时间。例如，在将摄像头采集的一帧原始视频数据输入至预设编码器时启动计时器，在检测到预设编码器完成编码操作时，结束计时，将计时器所得时间记为对应该原始视频帧的编码时长。在一实施例中，在两帧原始视频帧之间可能存在空闲时间，在空闲时间内，预设编码器未处于编码状态。
步骤102、根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略。
示例性的，可以参考在采用第一编码策略的情况下每个原始视频帧对应的编码时长来衡量目前的编码情况，可理解为评估计算机设备当前的实际编码能力，在设备编码性能过剩时，可通过增加编码复杂度来改善视频画质，而在设备编码性能不足时，可通过降低编码复杂度提升视频流畅度。可选的，可参考视频帧的单帧平均编码时长对第一编码策略进行调整，也可参考单位时间内预设编码器处于编码状态的时长对第一编码策略进行调整。
步骤103、利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
示例性的，经过对编码策略的调节后，所得到的第二编码策略更加适合设备的当前编码能力，平衡设备编码性能和编码质量，也使得视频画质以及视频流畅度能得到更好的平衡。示例性的，第二原始视频数据可以包括第一原始视频数据之后的直至视频编码事件结束之前的所有原始视频数据；第二原始视频数据可以包括第一原始视频数据之后的预设时长内的原始视频数据。可选的，可以周期性的对编码策略进行调整，例如，继续在预设时长内利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码，并统计所述第二原始视频数据中每个原始视频帧对应的编码时长，根据此时的编码时长对第二编码策略进行调整，得到第三编码策略，利用预设编码器采用第三编码策略继续进行编码。
本申请实施例中提供的视频编码方法，在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关，根据编码时长对第一编码策略进行调整，得到第二编码策略，利用预设编码器采用第二编码策略对第一原始视频数据之后的第二原始视频数据进行编码。通过采用上述技术方案，可以根据预设时长内的实际编码耗时情况对后续的编码策略进行动态调整，从而实现动态调节编码复杂度，提高视频编码的灵活性。
在一些实施例中，所述根据所述编码时长对所述第一编码策略进行调整， 得到第二编码策略，包括：根据所述编码时长确定所述预设编码器在单位时间内处于编码状态的第一时长；当所述第一时长小于第一预设阈值时，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度大于所述第一编码策略对应的编码复杂度。单位时间例如可以是1秒。第一预设阈值可以根据实际情况进行设置。当第一时长小于第一预设阈值时，说明此时编码速度较快，单位时间内预设编码器空闲时间较多，设备编码性能存在过剩的情况，可以对第一编码策略进行调整，以增大编码复杂度，有利于改善视频画质，提升视频质量。
可选的，第一时长可以存在多种计算方式，例如，计算所述编码时长的总和与所述预设时长的商，根据所述商确定所述预设编码器在单位时间内处于编码状态的第一时长；又如，计算所述编码时长的平均值与所述预设编码器的当前帧率的乘积，根据所述乘积确定所述预设编码器在单位时间内处于编码状态的第一时长。假设预设时长为C，第一原始视频数据中所有原始视频帧的编码时长总和为t，则第一时长可以表示为t/C*1秒，例如，t为7秒，C为10秒，商为0.7，可得知第一时长为0.7秒，也即在1秒的单位时长内，预设编码器有0.7秒处于编码状态。又如，第一原始视频数据中原始视频帧的总帧数为n，当前帧率为每秒传输帧数(Frames Per Second，fps)，则第一时长可以表示为(t/n)*fps*1秒，例如，t为7秒，n为300帧，平均值为7秒/300帧，fps为30帧/秒，则乘积为0.7，可得知第一时长为0.7秒，也即在1秒的单位时长内，预设编码器有0.7秒处于编码状态。
在一些实施例中，还可包括：当所述第一时长大于第二预设阈值时，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度小于所述第一编码策略对应的编码复杂度，其中，所述第二预设阈值大于所述第一预设阈值。第二预设阈值小于单位时间，数值可以根据实际情况进行设置。当第一时长大于第二预设阈值时，说明此时编码速度较慢，单位时间内预设编码器空闲时间较少，设备编码性能存在不足的情况，可以对第一编码策略进行调整，以降低编码复杂度，有利于提升视频流畅度。
在一些实施例中，所述编码策略中包含编码质量档位和编码速度，一个编码质量档位对应至少一个编码速度。这样设置，可以在两个维度对编码策略进行调节，提升编码策略的多样性，也使得编码策略更加合理。
在一些实施例中，所述当所述第一时长小于第一预设阈值时，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度大于所述第一编码策略对应的编码复杂度，包括：根据所述第一时长和所述第一预设阈值确定第一调整系数；根据所述第一调整系数调整所述第一编码策略中的第一编码 质量档位和第一编码速度，得到第二编码质量档位和第二编码速度；其中，所述第二编码速度小于所述第一编码速度，和/或，所述第二编码质量档位高于所述第一编码质量档位。示例性的，可以根据第一预设阈值和第一时长的商确定第一调整系数，根据第一编码速度和第一调整系数的差确定第二编码速度。当第一调整系数超过第一阈值时，提高编码质量档位，即第二编码质量档位高于所述第一编码质量档位，否则，可不对编码质量档位进行调整，保持所述第一编码质量档位不变，即第二编码质量档位等于所述第一编码质量档位。此外，也可不调整编码速度，直接提高编码质量档位；也可在降低编码速度的同时提高编码质量档位。第一调整系数、第二编码质量档位以及第二编码速度还可以存在其他确定方式，本申请实施例不做限定。
在一些实施例中，所述根据所述第一调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度，包括：根据所述第一调整系数和所述第一编码策略中的第一编码速度确定第二编码速度；当所述第二编码速度小于所述第一编码质量档位中的最小编码速度时，确定第二编码质量档位为高于所述第一编码质量档位的编码质量档位，并将所述第二编码速度更新为所述第二编码质量档位下的第一预设编码速度。其中，第一预设编码速度可以是第二编码质量档位下的最高编码速度，也可以是其他编码速度，可根据实际需求进行设置。
在一些实施例中，当所述第一时长大于第二预设阈值时，所述对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度小于所述第一编码策略对应的编码复杂度，包括：根据所述第一时长和所述第二预设阈值确定第二调整系数；根据所述第二调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度；其中，所述第二编码速度大于所述第一编码速度，和/或，所述第二编码质量档位低于所述第一编码质量档位。示例性的，可以根据第一时长和第二预设阈值的商确定第二调整系数，根据第一编码速度和第二调整系数的和确定第二编码速度。当第二调整系数超过第二阈值时，降低编码质量档位，即第二编码质量档位低于所述第一编码质量档位，否则，可不对编码质量档位进行调整，保持所述第一编码质量档位不变，即第二编码质量档位等于所述第一编码质量档位。此外，也可不调整编码速度，直接降低编码质量档位；也可在提高编码速度的同时降低编码质量档位。第二调整系数、第二编码质量档位以及第二编码速度还可以存在其他确定方式，本申请实施例不做限定。
在一些实施例中，所述根据所述第二调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，包括：根据所述第二调整系数和所述第一编码策略中的第一编码速度确定第二编码速度；当所述第二编码速度超过所述 第一编码质量档位中的最大编码速度时，确定第二编码质量档位为低于所述第一编码质量档位的编码质量档位，并将所述第二编码速度更新为所述第二编码质量档位下的第二预设编码速度。其中，第二预设编码速度可以是第二编码质量档位下的最低编码速度，也可以是其他编码速度，可根据实际需求进行设置。
在一些实施例中，在所述在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码的过程中，还包括：检测网络状况是否满足预设调整条件，在网络状况满足预设调整条件的情况下，对所述预设编码器中的预设编码指标进行调整；重新在预设时长内利用调整后的预设编码器采用第一编码策略对第一原始视频数据进行编码。这样设置，可以根据网络状态自适应调整预设编码器中的预设编码指标，动态调节视频画质和流畅度。可选的，所述网络状况包括网速，所述预设编码指标包括分辨率、码率和帧率中的至少一项。当网速较慢时(如当前网速小于第一网速阈值)，降低视频编码档位，如降低分辨率、码率和帧率中的至少一项；当网速较快(如当前网速大于第二网速阈值，第二网速阈值大于第一网速阈值)时，提高视频编码档位，如提高分辨率、码率和帧率中的至少一项。
图2为本申请实施例提供的又一种视频编码方法的流程示意图，该方法以视频通话场景中采用vpx系列编码器进行编码为例进行说明。vpx系列编码器中，编码复杂度主要通过两个参数实现，即编码质量档位(deadline)和编码速度(cpuUsed)。其中，deadline用于指定编码的质量档次，vpx系列一般有三个档次，分别是REAL_TIME(实时)，GOOD_QUALITY(较优质量)，BEST_QUALITY(最优质量)；cpuUsed用于指定一个质量档次下的编码速度。可选的，REAL_TIME档中cpuUsed的范围可限制在[-16，-1]范围内，数值越小，编码复杂度越低，也即速度越快；GOOD_QUALITY档中cpuUsed的范围可限制在[0，5]范围内，数值越小，编码复杂度越高，也即速度越慢；BEST_QULAITY档中cpuUsed固定为0，不可调。
示例性的，该方法可包括以下步骤。
步骤201、检测到视频通话事件被触发。
示例性的，检测到应用程序中视频通话的双方建立通信连接时，可认为检测到视频通话事件被触发。随后，可初始化编码策略调节周期C(也即预设时长)、初始化编码单位时间耗时(第一时长)的下限阈值T1(也即第一预设阈值)和上限阈值T2(也即第二预设阈值)。T1例如可以是0.5秒，T2例如可以是0.9秒。
步骤202、在预设时长内利用vpx编码器采用当前编码策略对原始视频数据进行编码，并统计原始视频数据中每个原始视频帧对应的编码时长。
示例性的，首次执行本步骤时，当前编码策略中可包含预设编码质量档位和预设编码速度，也即，在预设时长内利用vpx编码器采用预设编码质量档位和预设编码速度对原始视频数据进行编码。在预设时长起始时刻，编码耗时t＝0，编码帧数n＝0。可通过设备的摄像头采集视频的原始数据，例如yuv格式数据，将原始视频数据送进vpx编码器，编码一帧结束后计算编码耗时并累加至t，同时帧数n累加1。
可选的，在执行步骤202的过程中，还可包括检测网络状况是否满足预设调整条件，若是，则对预设编码器中的预设编码指标进行调整，并重新执行步骤202；否则，继续执行步骤202。在一实施例中，重新执行步骤202时，进入新的周期，需要重置编码耗时t＝0，编码帧数n＝0，从新的预设时长起始时刻开始对原始视频数据进行编码。示例性的，预设编码指标可包括分辨率、码率及帧率，进行调整时，可调整其中的任意一个或多个。
步骤203、根据编码时长确定预设编码器在单位时间内处于编码状态的第一时长。
示例性的，若预设时长中原始视频数据中每个原始视频帧对应的编码时长统计完毕，假设vpx编码器的当前帧率为fps，则n＝fps*C，第一时长cost＝(t/n)*fps*1秒。
步骤204、判断第一时长是否小于第一预设阈值，若是，则执行步骤205；否则，执行步骤206。
步骤205、对当前编码策略进行调整，以增大编码策略对应的编码复杂度，得到新的当前编码策略，执行步骤208。
示例性的，根据第一时长和第一预设阈值确定第一调整系数，根据第一调整系数调整当前编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度，第二编码速度小于第一编码速度，第二编码质量档位高于或等于第一编码质量档位。第一编码质量档位和第一编码速度对应于当前编码策略，第二编码质量档位和第二编码速度对应于调整后的新的编码策略。可根据第一调整系数和第一编码速度确定第二编码速度，当第二编码速度小于第一编码质量档位中的最小编码速度时，确定第二编码质量档位为高于第一编码质量档位的编码质量档位，并将第二编码速度更新为第二编码质量档位下的第一预设编码速度。
在一实施例中，当cost小于T1时，可令第一调整系数k1＝T1/cost，当所得数值不是整数时，可向上取整或向下取整。
若第一编码质量档位为REAL_TIME档，可令cpuUsed+＝k，也即，令第二 编码速度的取值(cpuUsed2)为第一编码速度的取值(cpuUsed1)加上k。当cpuUsed切换至-1后，若还需要增加编码复杂度，则将deadline切换至GOOD_QUALITY，cpuUsed也跟随切换至2，此处的2可根据实际需求进行设置。
若第一编码质量档位为GOOD_QUALITY档，可令cpuUsed-＝k，也即，令第二编码速度的取值(cpuUsed2)为第一编码速度的取值(cpuUsed1)减去k。当cpuUsed切换至0后，若还需要增加编码复杂度，则将deadline切换至BEST_QULAITY，cpuUsed也跟随切换至0。
若第一编码质量档位为BEST_QULAITY，可认为编码复杂度已经达到最大，无法继续增加。
步骤206、判断第一时长是否大于第二预设阈值，若是，则执行步骤207；否则，执行步骤208。
步骤207、对当前编码策略进行调整，以减小编码策略对应的编码复杂度，得到新的编码策略。
示例性的，根据第一时长和第二预设阈值确定第二调整系数，根据第二调整系数调整第一编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度，第二编码速度大于第一编码速度，第二编码质量档位低于或等于第一编码质量档位。可根据第二调整系数和第一编码速度确定第二编码速度，当第二编码速度达到第一编码质量档位中的最大编码速度时，确定第二编码质量档位为低于第一编码质量档位的编码质量档位，并将第二编码速度更新为第二编码质量档位下的第二预设编码速度。
在一实施例中，当cost大于T2时，可令第二调整系数k2＝cost/T2，当所得数值不是整数时，可向上取整或向下取整。
若第一编码质量档位为REAL_TIME档，可令cpuUsed-＝k，也即，令第二编码速度的取值(cpuUsed2)为第一编码速度的取值(cpuUsed1)减去k。当cpuUsed切换至-16后，可认为编码复杂度已经达到最小，无法继续减小。
若第一编码质量档位为GOOD_QUALITY档，可令cpuUsed+＝k，也即，令第二编码速度的取值(cpuUsed2)为第一编码速度的取值(cpuUsed1)加上k。当cpuUsed切换至5后，若还需要降低编码复杂度，则将deadline切换至REAL_TIME，cpuUsed也跟随切换至-4，此处的-4可根据实际需求进行设置。
若第一编码质量档位为BEST_QULAITY，可将deadline切换至GOOD_QUALITY，cpuUsed可不变。
图3为本申请实施例提供的编码质量与编码速度的切换示意图，可结合图3 对上述切换过程进行理解。图中RT即REAL_TIME，GOOD即GOOD_QUALITY，BEST即BEST_QULAITY。
步骤208、判断视频通话是否结束，若是，则结束流程；否则，返回执行步骤202。
示例性的，可在判断视频通话是否结束之前对编码耗时和编码帧数进行重置，即t＝0，n＝0。
本申请实施例提供的视频编码方法，在视频通话场景下，利用vpx系列编码器进行通话视频的编码，周期性地依据单位时间内平均编码耗时动态调节编码质量档位和编码速度，以实现动态调节编码强度，当移动设备编码性能过剩时，通过增加编码复杂度改善视频画质，当移动设备编码性能不足时，通过降低编码复杂度提高视频流畅性，提高视频编码的灵活性，并提升视频通话体验。
图4为本申请实施例提供的一种视频编码装置的结构框图，该装置可由软件和/或硬件实现，一般可集成在计算机设备中，可通过执行视频编码方法来进行视频编码。如图4所示，该装置包括以下模块。
第一编码模块401，设置为在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码；
时长统计模块402，设置为统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；
策略调整模块403，设置为根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略；
第二编码模块404，设置为利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
本申请实施例提供的视频编码装置，在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关，根据编码时长对第一编码策略进行调整，得到第二编码策略，利用预设编码器采用第二编码策略对第一原始视频数据之后的第二原始视频数据进行编码。通过采用上述技术方案，可以根据预设时长内的实际编码耗时情况对后续的编码策略进行动态调整，从而实现动态调节编码复杂度，提高视频编码的灵活性。
本申请实施例提供了一种计算机设备，该计算机设备中可集成本申请实施例提供的视频编码装置。图5为本申请实施例提供的一种计算机设备的结构框 图。计算机设备500包括存储器501、处理器502及存储在存储器501上并可在处理器502上运行的计算机程序，所述处理器502执行所述计算机程序时实现本申请实施例提供的视频编码方法。
本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行本申请实施例提供的视频编码方法。
存储介质包括多种类型的存储器设备或存储设备。术语“存储介质”旨在包括：安装介质；计算机系统存储器或随机存取存储器；非易失性存储器等。
上述实施例中提供的视频编码装置、设备以及存储介质可执行本申请任意实施例所提供的视频编码方法，具备执行该方法相应的功能模块和有益效果。未在上述实施例中描述的技术细节，可参见本申请任意实施例所提供的视频编码方法。

Claims (15)

  1. 一种视频编码方法，包括：
    在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码，并统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；
    根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略；
    利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
  2. 根据权利要求1所述的方法，其中，所述根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略，包括：
    根据所述编码时长确定所述预设编码器在单位时间内处于编码状态的第一时长；
    在所述第一时长小于第一预设阈值的情况下，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度大于所述第一编码策略对应的编码复杂度。
  3. 根据权利要求2所述的方法，还包括：
    在所述第一时长大于第二预设阈值的情况下，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度小于所述第一编码策略对应的编码复杂度，其中，所述第二预设阈值大于所述第一预设阈值。
  4. 根据权利要求2所述的方法，其中，所述根据所述编码时长确定所述预设编码器在单位时间内处于编码状态的第一时长，包括：
    计算多个所述编码时长的总和与所述预设时长的商，根据所述商确定所述预设编码器在单位时间内处于所述编码状态的第一时长；或，
    计算多个所述编码时长的平均值与所述预设编码器的当前帧率的乘积，根据所述乘积确定所述预设编码器在单位时间内处于所述编码状态的第一时长。
  5. 根据权利要求3或4所述的方法，所述编码策略中包含编码质量档位和编码速度，一个编码质量档位对应至少一个编码速度。
  6. 根据权利要求5所述的方法，其中，所述在所述第一时长小于第一预设阈值的情况下，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度大于所述第一编码策略对应的编码复杂度，包括：
    根据所述第一时长和所述第一预设阈值确定第一调整系数；
    根据所述第一调整系数调整所述第一编码策略中的第一编码质量档位和第 一编码速度，得到第二编码质量档位和第二编码速度；其中，所述第二编码速度小于所述第一编码速度，或，所述第二编码质量档位高于所述第一编码质量档位，或所述第二编码速度小于所述第一编码速度和所述第二编码质量档位高于所述第一编码质量档位。
  7. 根据权利要求6所述的方法，其中，所述根据所述第一调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度，包括：
    根据所述第一调整系数和所述第一编码策略中的第一编码速度确定所述第二编码速度；
    在所述第二编码速度小于所述第一编码质量档位中的最小编码速度的情况下，确定所述第二编码质量档位为高于所述第一编码质量档位的编码质量档位，并将所述第二编码速度更新为所述第二编码质量档位下的第一预设编码速度。
  8. 根据权利要求5所述的方法，其中，在所述第一时长大于第二预设阈值的情况下，对所述第一编码策略进行调整，以使得到的第二编码策略对应的编码复杂度小于所述第一编码策略对应的编码复杂度，包括：
    根据所述第一时长和所述第二预设阈值确定第二调整系数；
    根据所述第二调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，得到第二编码质量档位和第二编码速度；其中，所述第二编码速度大于所述第一编码速度，或，所述第二编码质量档位低于所述第一编码质量档位，或所述第二编码速度大于所述第一编码速度和所述第二编码质量档位低于所述第一编码质量档位。
  9. 根据权利要求8所述的方法，其中，所述根据所述第二调整系数调整所述第一编码策略中的第一编码质量档位和第一编码速度，包括：
    根据所述第二调整系数和所述第一编码策略中的第一编码速度确定所述第二编码速度；
    在所述第二编码速度超过所述第一编码质量档位中的最大编码速度的情况下，确定所述第二编码质量档位为低于所述第一编码质量档位的编码质量档位，并将所述第二编码速度更新为所述第二编码质量档位下的第二预设编码速度。
  10. 根据权利要求1所述的方法，其中，在所述在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码的过程中，还包括：
    检测网络状况是否满足预设调整条件，在所述网络状况满足所述预设调整条件的情况下，对所述预设编码器中的预设编码指标进行调整；
    重新在所述预设时长内利用调整后的预设编码器采用所述第一编码策略对所述第一原始视频数据进行编码。
  11. 根据权利要求10所述的方法，其中，所述网络状况包括网速，所述预设编码指标包括分辨率、码率和帧率中的至少一项。
  12. 根据权利要求1所述的方法，其中，所述预设编码器包括vpx系列编码器。
  13. 一种视频编码装置，包括：
    第一编码模块，设置为在预设时长内利用预设编码器采用第一编码策略对第一原始视频数据进行编码；
    时长统计模块，设置为统计所述第一原始视频数据中每个原始视频帧对应的编码时长，其中，编码策略与编码复杂度相关；
    策略调整模块，设置为根据所述编码时长对所述第一编码策略进行调整，得到第二编码策略；
    第二编码模块，设置为利用所述预设编码器采用所述第二编码策略对所述第一原始视频数据之后的第二原始视频数据进行编码。
  14. 一种计算机设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1-12中任一项所述的视频编码方法。
  15. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1-12中任一项所述的视频编码方法。
PCT/CN2021/076351 2020-02-20 2021-02-09 视频编码方法、装置、设备及存储介质 WO2021164670A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010106043.3A CN111263153B (zh) 2020-02-20 2020-02-20 视频编码方法、装置、设备及存储介质
CN202010106043.3 2020-02-20

Publications (1)

Publication Number Publication Date
WO2021164670A1 true WO2021164670A1 (zh) 2021-08-26

Family

ID=70951440

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/076351 WO2021164670A1 (zh) 2020-02-20 2021-02-09 视频编码方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN111263153B (zh)
WO (1) WO2021164670A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024007770A1 (zh) * 2022-07-06 2024-01-11 北京字跳网络技术有限公司 视频资源管理方法、装置、电子设备及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111263153B (zh) * 2020-02-20 2022-10-11 广州市百果园信息技术有限公司 视频编码方法、装置、设备及存储介质
CN112383777B (zh) * 2020-09-28 2023-09-05 北京达佳互联信息技术有限公司 视频编码方法、装置、电子设备及存储介质
CN113784210B (zh) * 2021-09-03 2023-09-22 上海哔哩哔哩科技有限公司 预监视频播放方法及云导播台服务系统
CN114827662B (zh) * 2022-03-18 2024-06-25 百果园技术(新加坡)有限公司 视频分辨率自适应调节方法、装置、设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106878808A (zh) * 2017-03-17 2017-06-20 福建中金在线信息科技有限公司 一种视频播放方法及装置
CN107105240A (zh) * 2017-03-22 2017-08-29 中南大学 一种hevc‑scc复杂度控制方法及其系统
US20180332278A1 (en) * 2017-05-15 2018-11-15 City University Of Hong Kong Hevc with complexity control based on dynamic ctu depth range adjustment
CN110166771A (zh) * 2018-08-01 2019-08-23 腾讯科技（深圳）有限公司 视频编码方法、装置、计算机设备和存储介质
CN111263153A (zh) * 2020-02-20 2020-06-09 广州市百果园信息技术有限公司 视频编码方法、装置、设备及存储介质
CN111510715A (zh) * 2019-01-31 2020-08-07 上海哔哩哔哩科技有限公司 视频处理方法、系统、计算机设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108495130B (zh) * 2017-03-21 2021-04-20 腾讯科技（深圳）有限公司 视频编码、解码方法和装置、终端、服务器和存储介质
CN109120933B (zh) * 2018-10-11 2021-05-14 广州酷狗计算机科技有限公司 动态调整码率的方法、装置、设备及存储介质
CN110365600B (zh) * 2019-07-30 2023-02-17 广州市百果园信息技术有限公司 一种基于bbr的拥塞控制方法、装置、设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106878808A (zh) * 2017-03-17 2017-06-20 福建中金在线信息科技有限公司 一种视频播放方法及装置
CN107105240A (zh) * 2017-03-22 2017-08-29 中南大学 一种hevc‑scc复杂度控制方法及其系统
US20180332278A1 (en) * 2017-05-15 2018-11-15 City University Of Hong Kong Hevc with complexity control based on dynamic ctu depth range adjustment
CN110166771A (zh) * 2018-08-01 2019-08-23 腾讯科技（深圳）有限公司 视频编码方法、装置、计算机设备和存储介质
CN111510715A (zh) * 2019-01-31 2020-08-07 上海哔哩哔哩科技有限公司 视频处理方法、系统、计算机设备及存储介质
CN111263153A (zh) * 2020-02-20 2020-06-09 广州市百果园信息技术有限公司 视频编码方法、装置、设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024007770A1 (zh) * 2022-07-06 2024-01-11 北京字跳网络技术有限公司 视频资源管理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN111263153A (zh) 2020-06-09
CN111263153B (zh) 2022-10-11

Similar Documents

Publication Publication Date Title
WO2021164670A1 (zh) 视频编码方法、装置、设备及存储介质
CN102325274B (zh) 一种自适应网络带宽的视频流传输控制方法
CN113301392B (zh) 码率确定方法、装置、设备及存储介质
CN108347580B (zh) 一种处理视频帧数据的方法及电子设备
WO2019033877A1 (zh) 一种视频帧编码方法、终端及存储介质
US20110299588A1 (en) Rate control in video communication via virtual transmission buffer
CN102665131A (zh) 一种网络视频服务系统接收端的视频缓冲方法
WO2018059175A1 (zh) 视频码率处理方法和装置、存储介质及电子设备
US10142644B2 (en) Decoding frames
WO2012119459A1 (zh) 数据传输的方法、装置和系统
CN111617466B (zh) 编码格式的确定方法、装置及云游戏的实现方法
EP4366296A1 (en) Video call processing method and apparatus, device and storage medium
CN106993190A (zh) 软硬件协同编码方法及系统
WO2021036650A1 (zh) 信息处理方法、装置、设备及计算机可读存储介质
WO2023045551A1 (zh) 摄像头控制方法、系统、电子设备和存储介质
CN113727185A (zh) 视频帧播放方法及系统
CN112866746A (zh) 一种多路串流云游戏控制方法、装置、设备及存储介质
CN114827662A (zh) 视频分辨率自适应调节方法、装置、设备和存储介质
CN115037416A (zh) 数据前向纠错处理方法、装置、电子设备和存储介质
CN115514960A (zh) 视频编码方法、装置、电子设备与存储介质
CN117599412A (zh) 一种基于云游戏业务质量检测的自适应渲染系统及方法
Qi et al. LBVC: towards low-bandwidth video chat on smartphones
CN107018379A (zh) 一种视频流的传输方法和装置
Qi et al. A context-aware framework for reducing bandwidth usage of mobile video chats
US20230247069A1 (en) Systems and Methods for Adaptive Video Conferencing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21757706

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21757706

Country of ref document: EP

Kind code of ref document: A1