Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/146—Data rate or code amount at the encoder output
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
  • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/142—Detection of scene cut or scene change
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

• an electronic device comprising: at least one processor; and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores a computer program, and the computer program implements the above-mentioned video encoding method when executed by the at least one processor.
• the second prediction of the rate factor is based on the rate factor of the video coding standard with the shortest encoding time in the first prediction result. Since the computational complexity of the video coding standard with the shortest encoding time is the lowest, the rate factor under this video coding standard can reflect the accuracy of the rate factors under other video coding standards with longer encoding time to a certain extent, thereby associating different video coding standards, which is conducive to further reducing the amount of calculation required to predict the rate factor.
• FIG1 shows a flowchart of a video encoding method according to some embodiments of the present disclosure
• FIG2 shows a flowchart of a method for training a first rate factor prediction model according to some embodiments of the present disclosure
• FIG3 shows a structural block diagram of a first rate factor prediction model according to some embodiments of the present disclosure
• FIG4 shows a flowchart of determining multiple target bit rate factors of a video segment to be encoded under multiple preset video coding standards according to some embodiments of the present disclosure
• FIG5 shows a structural block diagram of a video encoding device according to an embodiment of the present disclosure
• step S110, obtaining a video segment to be encoded may include: obtaining a video to be encoded; performing scene detection on the video to be encoded; dividing the video to be encoded into one or more sub-video segments based on the scene detection result; and identifying each video sub-segment in the one or more video segments as a video segment to be encoded.
• scene detection can be performed on the video to be encoded based on an open source encoder.
• the open source encoder x264 can be called to encode the video V, and the scene detection switch can be turned on to obtain a corresponding scene switching detection result.
• the video V can be divided into t independent single-lens video segments (V1, V2...Vt), and each single-lens video segment can be used as a video segment to be encoded.
• the content shown in a single-shot video clip is usually similar and more coherent. Therefore, when encoding the video, it is more reasonable to use a specific and identical bitrate factor for each single-shot video clip, thereby ensuring a stable picture quality experience in all scenarios.
• the inter-frame coding information of each video frame pair can also be extracted. For example, for a video frame pair (f 1 , f 2 ), the inter-frame coding information of f 1 and f 2 is calculated; for a video frame pair (f 2 , f 3 ), the inter-frame coding information of f 2 and f 3 is calculated, and so on.
• Examples of multiple preset video coding standards may include H.264 video coding standard, H.265 video coding standard, H.266 video coding standard, AV1 video coding standard. It will be understood that the preset video standard may also include any other existing video coding standard or video coding standard generated as technology develops. It will also be understood that the number of multiple preset video coding standards may be any integer value that meets actual needs. The scope of the subject matter claimed in the present disclosure is not limited in both aspects.
• the sample rate factor may be a real rate factor, that is, the quality of the encoded video obtained after encoding the sample video segment using the rate factor meets the target quality requirement.
• the quality of the encoded video may be evaluated, for example, by a video coding score (also known as a video multimethod assessment fusion, VMAF).
• VMAF video multimethod assessment fusion
• the smaller the value of VMAF the lower the quality of the encoded video.
• the value range of VMAF may be 0-100 under normal circumstances.
• the real bit rate factor can be obtained by the following steps: for the video coding standard, selecting an initial sample bit rate factor and a preset sample video coding score range; based on the initial sample bit rate factor, using a corresponding encoder in a real transcoding system (for example, H.264 encoder, H.265 encoder, H.266 encoder, AV1 encoder, etc.) to encode the sample video clip to obtain a sample pre-encoded video clip under the video coding standard; comparing the sample video coding score of the obtained sample pre-encoded video clip with the preset sample video coding score range; if the sample video coding score is within If the sample video coding score is within the preset sample video coding score range (i.e., the encoded video meets the target quality requirements), it indicates that the initial sample rate factor under the video coding standard is feasible, and the initial sample rate factor can be used as the real rate factor; if the sample video coding score is outside the preset sample video coding score range (i.e., the encode
• initial parameters of a first rate factor prediction model may be selected for a plurality of video coding standards, and based on the initial parameters, a plurality of sample prediction rate factors of sample video clips under the plurality of video coding standards may be determined using the acquired sample spatiotemporal domain feature information.
• a model loss value may be calculated based on the corresponding sample prediction rate factor and the sample prediction rate factor, for example, via a loss function.
• loss functions include, but are not limited to, a cross entropy loss function, a maximum loss function, an average loss function, a 0-1 loss function, etc. It will be understood that in the present disclosure, other suitable methods may also be used to calculate the model loss value, and the scope of the subject matter claimed in the present disclosure is not limited in this respect.
• step S240 after calculating the model loss value under each preset video coding standard, these model loss values can be back-propagated, and the parameters of the first rate factor prediction model can be adjusted, and then steps S220-S240 are repeated until the model training stop condition is reached.
• the model training stop condition can be that the model loss value is lower than a preset threshold, and/or the number of rounds of model training reaches a preset number of rounds. It will be understood that the present disclosure does not limit the model training stop condition, and it can be adjusted according to actual needs.
• the screened features are input to the residual module 330, which may include multiple convolutional layers, which change the screened features as input, and the unchanged screened features will skip these convolutional layers and be transmitted to the subsequent layers, and then be processed as a whole to obtain the output result of the residual module, thereby reducing information loss.
• the output of the residual module 330 can be further input to the second attention mechanism module 340 to achieve further screening of the features, thereby further improving the accuracy of the prediction results.
• the input features may be aggregated via the fully connected module 350 to generate and output a predicted bit rate factor.
• FIG4 shows a flowchart of determining multiple target rate factors of a video segment to be encoded under multiple preset video coding standards according to some embodiments of the present disclosure.
• step S140 based on a first predicted rate factor of a first preset video coding standard among multiple preset video coding standards and a preset video coding score interval, determines multiple target rate factors of a video segment to be encoded under multiple preset video coding standards.
• the first preset video coding standard refers to the video coding standard that takes the shortest time to perform video coding.
• the video coding standard H.264 takes the shortest time to perform video coding, so the video coding standard H.264 is the first preset video coding standard.
• the encoding parameters used when encoding the video to be encoded may also include but are not limited to preset, GOP (Group of Picture), size, etc. Since the bit rate factor plays an important role in the quality of the encoded video, when encoding the video using the method of the present disclosure, the values of these encoding parameters can be set to fixed constants (for example, default preset values).
• the rate factor under this video coding standard can, to a certain extent, reflect the accuracy of the rate factors under other video coding standards with longer encoding times.
• the video coding score of the video clip encoded based on the first predicted rate factor corresponding to the first preset video coding standard (for example, the video coding standard H.264 in the above example) is within the preset video coding score range (meeting the video quality requirement)
• the accuracy of other predicted rate factors corresponding to other preset video coding standards for example, the video coding standards H.265, H.266 and AV1 in the above example
• the video coding scores of the encoded videos under the video coding standards H.265, H.266 and AV1 also have a
• the video coding scores of the video clips encoded using these predicted bitrate factors are usually also within the preset video coding score range (meeting the video quality requirements), so that different video coding standards can be associated, which is conducive to reducing the amount of calculation required for the predicted bitrate factors.
• the probability that the predicted bitrate factors under the video coding standards H.265, H.266 and AV1 meet the video quality requirements is 95%, so these predicted bitrate factors can be directly used as target bitrate factors for encoding the video to be encoded.
• multiple preset video coding standards being H.264, H.265, H.266 and AV1
• the respective predicted bitrate factors under the video coding standards H.265, H.266 and AV1 and the video coding score of the pre-encoded video clip under the video coding standard H.264 can be used as feedback information, together with the spatiotemporal domain feature information of the video clip to be encoded, to perform a second prediction of the bitrate factors under these video coding standards to obtain multiple target bitrate factors.
• step S448, based on the spatiotemporal domain feature information of the video segment to be encoded, multiple predicted rate factors and the video coding score of the first pre-encoded video segment, updating multiple predicted rate factors to obtain multiple target rate factors may include: inputting the spatiotemporal domain feature information of the video segment to be encoded, multiple predicted rate factors and the video coding score of the first pre-encoded video segment into a second rate factor prediction model, so as to determine multiple target rate factors of the video segment to be encoded under multiple preset video coding standards via the second rate factor prediction model.
• the second rate factor prediction model can be trained by the following operations: inputting the sample spatiotemporal feature information, the sample prediction rate factor under each preset video coding standard, and the first sample video coding score corresponding to the sample prediction rate factor under the first preset video coding standard into the second rate factor prediction model, so as to determine the second rate factor prediction model through the second rate factor prediction model.
• the structural framework of the second rate factor prediction model is also the same as the structural framework of the first rate factor prediction model, so it will not be repeated here.
• the training of the second rate factor prediction model is dependent on the training results of the first rate factor prediction model. Therefore, the training of the second rate factor prediction model is performed after the training of the first rate factor prediction model is completed.
• the corresponding encoders in the transcoding system After determining multiple target bit rate factors, the corresponding encoders in the transcoding system (based on the corresponding preset video coding standards) are used to encode the video segments to be encoded according to the multiple bit rate factors. After multiple test data show that after encoding the video segments using the target bit rate factors obtained by the second prediction, there is a 99% probability that the video coding score of the encoded video segments is within the preset video coding score range. Therefore, the encoding result of the video segments based on the target bit rate factors obtained by the second prediction can be directly trusted.
• method 100 may also include: for each preset video coding standard among multiple preset video coding standards, obtaining all target video segments under the preset video coding standard; and combining all target video segments based on the order of one or more sub-video segments in the video to be encoded to obtain the target video corresponding to the video to be encoded under the preset video coding standard.
• multiple target video segments for each video segment to be encoded can be obtained.
• the corresponding target video segment among the multiple target video segments for each video segment to be encoded can be written into the final video bitstream file, and the written target video segments are spliced according to the order of each video segment to be encoded in the video to be encoded, so as to obtain the target video corresponding to the video to be encoded under the preset video encoding standard.
• each video segment to be encoded is a single-shot video segment showing similar and coherent content. Therefore, encoding each video segment to be encoded and obtaining the corresponding target video segment can ensure that each encoded video segment is The quality of the video clips to be encoded is improved, thereby ensuring a stable picture quality experience in various scenarios.
• the target video clips are spliced and combined based on the order of these video clips to be encoded in the video to be encoded, so that the complete video file can be restored, avoiding the discontinuity of the video file caused by the splicing errors of the target video clips.
• FIG5 shows a structural block diagram of a video encoding device 500 according to an embodiment of the present disclosure.
• the device 500 may include: an acquisition module 510, configured to acquire a video segment to be encoded, the video segment to be encoded including one or more video frames; a calculation module 520, configured to calculate the spatiotemporal feature information of the video segment to be encoded based on the one or more video frames; a prediction rate factor determination module 530, configured to determine a plurality of prediction rate factors of the video segment to be encoded under a plurality of preset video encoding standards based on the spatiotemporal feature information; a target rate factor determination module 540, configured to determine a plurality of target rate factors of the video segment to be encoded under a plurality of preset video encoding standards based on a first prediction rate factor corresponding to a first preset video encoding standard among the plurality of preset video encoding standards and a preset video encoding score interval among the plurality of prediction rate factors, wherein
• the spatiotemporal domain feature information may include coding feature information
• the computing module 530 may include: a preprocessing module, configured to preprocess one or more video frames to generate a new video frame sequence, the new video frame sequence including a set of video frame pairs; and an intra-frame and inter-frame coding module, configured to perform intra-frame coding and inter-frame coding, respectively, for each video frame pair in the set of video frame pairs to obtain coding feature information of the video segment to be encoded.
• the intra-frame and inter-frame coding module may include: a module configured to perform intra-frame coding on the first video frame in each video frame pair in the video frame pair set to obtain intra-frame coding information; A module configured to perform inter-frame coding on each video frame pair in the video frame pair set to obtain inter-frame coding information; and a module configured to obtain coding feature information based on the intra-frame coding information and the inter-frame coding information.
• the coding feature information may be based on one or more of the following items: the number of coding bits, the proportion of intra-frame prediction modes, and the distribution information of the amplitude of inter-frame motion vectors.
• the module 530 for determining the predicted bit rate factor may include: a module configured to input the spatiotemporal domain feature information of the video segment to be encoded into a first bit rate factor prediction model, so as to determine multiple predicted bit rate factors of the video segment to be encoded under multiple preset video coding standards via the first bit rate factor prediction model.
• the update module may include: a module configured to input the spatiotemporal domain feature information of the video segment to be encoded, multiple predicted bit rate factors, and the video coding score of the first pre-encoded video segment into a second bit rate factor prediction model, so as to determine multiple target bit rate factors of the video segment to be encoded under multiple preset video coding standards via the second bit rate factor prediction model.
• the second rate factor prediction model is trained by the following operations: combining the sample spatiotemporal feature information, the sample prediction rate factor under each preset video coding standard, and the first preset video coding standard with the sample prediction rate factor; A first sample video coding score corresponding to a sample predicted rate factor under a coding standard is input into a second rate factor prediction model to determine multiple sample target rate factors of sample video clips under multiple preset video coding standards through the second rate factor prediction model; for each of the multiple preset video coding standards, a second model loss value is calculated based on the sample rate factor and the sample target rate factor corresponding to the preset video coding standard; and parameters of the second rate factor prediction model are adjusted based on the second model loss value under each preset video coding standard until the second model training stop condition is reached.
• modules 510-550 of the apparatus 500 shown in FIG5 may correspond to the steps S110-S150 in the method 100 described with reference to FIG1.
• the operations, features and advantages described above for the method 100 are also applicable to the apparatus 500 and the modules included therein. For the sake of brevity, some operations, features and advantages are not described in detail herein.
• modules described above with respect to FIG. 5 may be implemented in hardware or in hardware in combination with software and/or firmware.
• these modules may be implemented as computer program codes/instructions configured to be executed in one or more processors and stored in a computer-readable storage medium.
• these modules may be implemented as hardware logic/circuits.
• one or more of the acquisition module 510, the calculation module 520, the determination prediction rate factor module 530, the determination target rate factor module 540, and the encoding module 550 may be implemented together in a system on chip (System on Chip, SoC).
• a non-transitory computer-readable storage medium storing a computer program
• the computer program implements the above-mentioned video encoding method when executed by a processor.
• the communication unit 680 allows the electronic device 600 to exchange information/data with other devices via a computer network such as the Internet and/or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and/or a chipset, such as a Bluetooth TM device, an 802.11 device, a Wi-Fi device, a WiMAX device, a cellular communication device and/or the like.
• a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
• a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
• a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=86117486

Family Applications (1)

Country Status (3)

Cited By (1)

Families Citing this family (2)

Citations (6)

Family Cites Families (6)

• 2022
  • 2022-12-27 CN CN202211690147.9A patent/CN116055723A/zh active Pending
• 2023
  • 2023-07-10 JP JP2025533535A patent/JP2026500919A/ja active Pending
  • 2023-07-10 WO PCT/CN2023/106615 patent/WO2024139166A1/zh not_active Ceased

Patent Citations (6)

Also Published As

Similar Documents

Legal Events