WO2023065891A1 - 多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品 - Google Patents

多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品 Download PDF

Info

Publication number
WO2023065891A1
WO2023065891A1 PCT/CN2022/118502 CN2022118502W WO2023065891A1 WO 2023065891 A1 WO2023065891 A1 WO 2023065891A1 CN 2022118502 W CN2022118502 W CN 2022118502W WO 2023065891 A1 WO2023065891 A1 WO 2023065891A1
Authority
WO
WIPO (PCT)
Prior art keywords
data block
reference data
encoded
block
frame
Prior art date
Application number
PCT/CN2022/118502
Other languages
English (en)
French (fr)
Inventor
王英彬
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2023560291A priority Critical patent/JP2024514294A/ja
Priority to EP22882502.2A priority patent/EP4395312A1/en
Priority to KR1020237037580A priority patent/KR20230162989A/ko
Publication of WO2023065891A1 publication Critical patent/WO2023065891A1/zh
Priority to US18/474,712 priority patent/US20240015310A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present application relates to the technical field of the Internet, and relates to but not limited to a multimedia data processing method, device, equipment, computer-readable storage medium and computer program product.
  • the intra-frame predictive coding method is usually used to predictively encode the multimedia data frame, which can improve the coding performance of the multimedia data frame. Therefore, the intra-frame predictive coding method has been widely used in the multimedia data processing process. However, it is found in practice that in the current intra-frame predictive coding method, the coding performance of multimedia data frames is poor due to the limited number of reference data blocks used for predictive coding.
  • Embodiments of the present application provide a multimedia data processing method, device, device, computer-readable storage medium, and computer program product, which can improve the encoding performance of multimedia data frames.
  • An embodiment of the present application provides a multimedia data processing method, including: acquiring attribute information of a data block to be encoded in a multimedia data frame; acquiring a first target reference data block that matches the attribute information of the data block to be encoded; wherein, The first target reference data block is selected from a first reference data block set, a second reference data block set, or a third reference data block set fused with the first reference data block set and the second reference data block set , the first set of reference data blocks includes encoded data blocks that have no adjacent relationship with the data block to be encoded, and the second set of reference data blocks includes Adjacent coded data blocks; performing predictive coding on the to-be-coded data block according to the first target reference data block.
  • An embodiment of the present application provides a multimedia data processing method, including: acquiring attribute information of a data block to be decoded in a multimedia data frame; acquiring a second target reference data block that matches the attribute information of the data block to be decoded; wherein, The second target reference data block belongs to a first reference data block set, a second reference data block set, or a third reference data block set fused with the first reference data block set and the second reference data block set,
  • the first set of reference data blocks includes decoded data blocks that have no adjacent relationship with the data block to be decoded, and the second set of reference data blocks includes A decoded data block of an adjacent relationship; decoding the data block to be decoded according to the second target reference data block.
  • An embodiment of the present application provides a multimedia data processing device, including: a first acquisition module configured to acquire attribute information of a data block to be encoded in a multimedia data frame; a first matching module configured to acquire information related to the data block to be encoded The first target reference data block whose attribute information matches; wherein, the first target reference data block is selected from the first set of reference data blocks, the second set of reference data blocks, or the fusion of the first set of reference data blocks and the set of all reference data blocks.
  • the third reference data block set of the second reference data block set, the first reference data block set includes encoded data blocks that do not have an adjacent relationship with the to-be-encoded data block, and the second reference
  • the data block set includes coded data blocks that have an adjacent relationship with the data block to be coded; the coding module is configured to perform predictive coding on the data block to be coded according to the first target reference data block.
  • An embodiment of the present application provides a multimedia data processing device, including: a second obtaining module configured to obtain attribute information of a data block to be decoded in a multimedia data frame; a second matching module configured to obtain attribute information related to the data block to be decoded A second target reference data block whose attribute information matches; wherein, the second target reference data block belongs to the first reference data block set, the second reference data block set, or merges the first reference data block set and the The third reference data block set of the second reference data block set, the first reference data block set includes a decoded data block that does not have an adjacent relationship with the data block to be decoded, the second reference data block The block set includes decoded data blocks that have an adjacent relationship with the data block to be decoded; the decoding module is configured to decode the data block to be decoded according to the second target reference data block.
  • An embodiment of the present application provides a computer device, including: a processor and a memory; wherein the memory is configured to store program instructions, and the processor is configured to invoke the program instructions to execute steps in the multimedia data processing method.
  • An embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the above-mentioned computer program includes program instructions, and when the above-mentioned program instructions are executed by a processor, the steps in the above-mentioned multimedia data processing method are executed .
  • An embodiment of the present application provides a computer program product, including a computer program or a computer instruction, and when the computer program or computer instruction is executed by a processor, the steps of the above multimedia data processing method are implemented.
  • the embodiments of the present application have at least the following beneficial effects: by obtaining the first target reference data block matching the attribute information of the data block to be encoded, the first target reference data block is selected from the first reference data block set, the second reference data block block set, or a third reference block set that fuses the first reference block set and the second reference block set; that is, the alternative reference block sets include the first reference block set, the second The reference data block set and the third reference data block set, in this way, the selection range of the reference data block is expanded, and the first target reference data block can be adaptively selected according to the attribute information of the data block to be encoded, so as to realize based on the first The target reference data block performs predictive coding on the data block to be coded.
  • the flexibility and accuracy of obtaining the first target reference data block can be improved, and the coding performance of the multimedia data frame can be improved.
  • performing predictive coding on the data block to be coded according to the first target reference data block helps to remove redundancy in the multimedia data frame, improve the transmission efficiency of the multimedia data frame, and save transmission resources.
  • FIG. 1 is a flow chart of video processing provided by an embodiment of the present application
  • FIG. 2 is a schematic flow diagram of a processing method for multimedia data provided in an embodiment of the present application
  • FIG. 3 is a schematic diagram of an encoded data block provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a scenario of non-intra-coding of a multimedia data frame provided by an embodiment of the present application
  • FIG. 5 is a schematic diagram of a scenario of intra-frame block copying of a multimedia data frame provided by an embodiment of the present application
  • FIG. 6 is a schematic diagram of a scene of memory reuse of a reference data block of a multimedia data frame provided by an embodiment of the present application
  • FIG. 7 is a schematic flowchart of a multimedia data processing method provided by an embodiment of the present application.
  • Fig. 8 is a schematic diagram of the positional relationship between a first reference data set, a second reference data block set, and a data block to be encoded provided by an embodiment of the present application;
  • FIG. 9 is a schematic diagram of the positional relationship between another first reference data set, a second reference data block set, and a data block to be encoded provided by the embodiment of the present application;
  • FIG. 10 is a schematic flowchart of another multimedia data processing method provided by the embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a multimedia data processing device provided in an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of another multimedia data processing device provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • multimedia data refers to composite data formed by interrelated media data such as text, graphics, images, sounds, animations, and moving images in content.
  • the multimedia data mentioned in the embodiment of the present application mainly includes image data composed of images, or video data composed of images and sounds, and in the embodiments of the present application, the multimedia data is mainly taken as video data as an example.
  • the data processing process of the data will be described in detail, and when the multimedia data is image data, reference may be made to the embodiments of the present application.
  • the multimedia data processing process involved in the embodiment of the present application mainly includes media data collection, media data encoding, media data file encapsulation, media data file transmission, media data decoding and final data presentation, and the multimedia data is video data , the complete processing process for video data can be shown in Figure 1, which can include: S1, video capture; S2, video encoding; S3, video file encapsulation; S4, video file transmission; S5, video file decapsulation; S6, Video decoding; S7, video rendering.
  • Video capture is used to convert analog video into digital video and save it in the format of digital video files. That is to say, video capture can convert video signals into binary digital information, and the binary information converted from video signals is a A binary data stream, the binary information can also be referred to as the code stream or bit stream (Bitstream) of the video signal.
  • Video encoding refers to the conversion of an original video format file into another video format file through compression technology.
  • the generation of video media content mentioned in the embodiment of the present application includes the real scene captured by the camera and the screen content scene generated by the computer. From the perspective of the acquisition method of the video signal, the video signal can be divided into those captured by the camera and those captured by the computer.
  • the mainstream video coding technology adopts a hybrid coding framework, and performs the following series of operations on the input original video signal and processing can be shown in Figure 2:
  • Block partition structure Input a multimedia data frame (such as a video frame in video data) according to the size of a video frame, divide the video frame into several non-overlapping processing units, and each processing unit will perform similar compression operation.
  • this processing unit is called a coding tree unit (CTU, Coding Tree Unit), or a largest coding unit (LCU, Largest Coding Unit,).
  • CTU can be more finely divided to obtain one or more basic coding units, which are called coding units (CU, Coding Unit).
  • CU Coding Unit
  • Each CU is the most basic element in a coding process.
  • a processing unit is also referred to as a coded slice (Tile), which is a rectangular area of a multimedia data frame that can be decoded and coded independently.
  • the coding slice can be further divided more finely to obtain one or more largest coding blocks (SB, Superblock), and then, the largest coding block is further divided to obtain one or more data blocks (B, block).
  • SB largest coding blocks
  • B data blocks
  • Each data block is the most basic element in an encoding process. Described below are various encoding methods that may be adopted for each data block, wherein the relationship between SB and B is shown in FIG. 3 .
  • Predictive Coding Including intra-frame prediction and inter-frame prediction, the original video signal is predicted by the selected encoded video signal to obtain the residual video signal.
  • the encoding end needs to select the most suitable one among many possible predictive encoding modes for the current encoded data block (ie, the data block to be decoded), and inform the decoding end.
  • intra picture prediction Intra picture Prediction
  • Inter picture prediction Inter picture prediction
  • the predicted signal comes from another picture (called a reference picture) that has been encoded and is different from the current picture.
  • Transform coding and quantization The residual video signal is transformed into the transform domain through discrete Fourier transform (DFT, Discrete Fourier Transform), discrete cosine transform and other transformation operations.
  • DFT discrete Fourier transform
  • Discrete Fourier Transform discrete Fourier Transform
  • the signal in the transform domain is subjected to a lossy quantization operation, so that the quantized signal is conducive to compressed expression.
  • the encoding end also needs to select one of the transformation methods for the current encoded data block, and inform the decoding end.
  • the fineness of quantization is usually determined by the quantization parameter (QP, Quantization Parameter).
  • QP quantization parameter
  • QP Quantization Parameter
  • Entropy Coding or Statistical Coding The quantized transform domain signal will be statistically compressed and encoded according to the frequency of occurrence of each value, and finally output a binary (0 or 1) compressed code stream. At the same time, encoding generates other information, such as selected modes, motion vectors, etc., which also require entropy encoding to reduce the bit rate.
  • Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal.
  • Common statistical coding methods include variable length coding (VLC, Variable Length Coding) or context-based binary arithmetic coding (CABAC, Content Adaptive Binary Arithmetic Coding).
  • Loop Filtering The coded image (ie multimedia data frame), after inverse quantization, inverse transformation and prediction compensation operations, can obtain a reconstructed decoded image. Compared with the original image, the reconstructed image has some information different from the original image due to the influence of quantization, resulting in distortion (Distortion). Perform filtering operations on the reconstructed image, such as deblocking, sample adaptive compensation (SAO, Sample Adaptive Offset) or adaptive loop filtering (ALF, Adaptive Loop Filter) and other filters, which can effectively reduce quantization The degree of distortion produced. Since these filtered reconstructed images will be used as references for subsequent encoded images for predicting future signals, the above filtering operation is also called loop filtering, and filtering operations in the encoding loop.
  • SAO sample adaptive compensation
  • ALF adaptive Loop Filter
  • Figure 2 shows the basic flow of the video encoder.
  • Figure 2 takes the kth CU (marked as S k [x, y]) as an example to illustrate, where k is the current A positive integer of the number of CUs in the image, S k [x, y] represents the pixel with the coordinates [x, y] in the kth CU, x represents the abscissa of the pixel, y represents the middle coordinate of the pixel, S k [x, y] obtains the prediction signal after one of the better processing in motion compensation or intra prediction, etc. Sk [x,y] and Subtract the residual signal U k [x, y], and then transform and quantize the residual signal U k [x, y].
  • the quantized output data has two different places: one is to send an entropy encoder Entropy coding is performed, and the coded code stream is output to a buffer (buffer) for storage and waiting to be transmitted; another application is to perform inverse quantization and inverse transformation to obtain the signal U' k [x, y]. Combine the signal U' k [x, y] with A new prediction signal S* k [x, y] is obtained by adding, and S* k [x, y] is sent to the buffer of the current image for storage.
  • S* k [x, y] undergoes intra-image prediction to obtain f(S* k [x, y]), S* k [x, y] obtains S' k [x, y] after loop filtering, And S' k [x, y] is sent to the decoded image buffer for saving, so as to generate the reconstructed video.
  • S' k [x, y] is obtained after motion-compensated prediction S' r [x+m x , y+m y ], S' r [x+m x , y+m y ] represents the reference block, m x and m y denote the horizontal and vertical components of the motion vector, respectively.
  • the decoder obtains the compressed code stream, it first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient is inversely quantized and inversely transformed to obtain a residual signal.
  • the prediction signal corresponding to the data block can be obtained, and after the two are added, the coded data block can be obtained.
  • the encoded data block is subjected to a loop filtering operation to produce the final encoded data block.
  • the current mainstream video coding standards such as HEVC, VVC, AVS3, AV1 and AV2, all adopt a block-based hybrid coding framework.
  • These video coding standards divide the original multimedia data frame into a series of coded data blocks, and combine video coding methods such as prediction, transformation and entropy coding to realize video data compression.
  • motion compensation is a kind of predictive coding method commonly used in video coding. Motion compensation is based on the redundancy characteristics of video content in time domain or space domain, and derives the prediction value of the current coded data block from the coded area (ie, the reference data block). .
  • Such predictive coding methods include: inter-frame prediction, intra-frame block copy prediction, intra-frame string copy prediction, etc.
  • these predictive coding methods may be used alone or in combination.
  • one or more two-dimensional displacement vectors need to be encoded explicitly or implicitly in the code stream, indicating that the current coded data block (or the same block of the current block) is relative to the coded data The displacement of one or more reference data blocks of the block.
  • the displacement vector may have different names, and the embodiments of the present application are described in the following manner: 1)
  • the displacement vector in inter-frame prediction is called a motion vector (MV, Motion Vector ); 2)
  • the displacement vector in intra-frame block copying i.e. intra-frame predictive coding
  • block vector BV, Block Vector
  • the displacement vector in intra-frame string copying is called string vector (SV, String Vector) .
  • the related technologies in inter-frame prediction and intra-frame block copy prediction are introduced below.
  • inter-frame prediction utilizes the time-domain correlation of multimedia data frames to predict the pixels of the current image using the pixels of adjacent encoded images, so as to effectively remove the time-domain redundancy of multimedia data frames, which can effectively Save bits for encoding residual data.
  • P is the current frame
  • Pr is the reference frame
  • B is the current data block to be decoded
  • Br is the reference data block of B.
  • the coordinates of B' and B in the image are the same, the coordinates of Br are (x r , y r ), and the coordinates of B' are (x, y).
  • the displacement between the current encoded data block and its reference data block is called the motion vector (MV), which can be expressed by the following formula (1):
  • MV prediction technology can be used to further reduce the bits required for encoding MV.
  • Intra block copy prediction Intra Block Copy
  • IBC Intra Block Copy
  • SCC Screen Content Coding
  • Block Copy significantly improves the coding efficiency of screen content.
  • IBC technology is also adopted to improve the performance of screen content encoding. IBC utilizes the spatial correlation of the screen content video and uses the pixels of the encoded data block in the multimedia data frame to predict the pixels of the data block to be decoded, which can effectively save the bits required for encoding pixels.
  • Fig. 5 is a schematic diagram of the IBC technology, wherein the displacement between the data block to be decoded and its reference data block is called a block vector (BV, Block Vector).
  • BV Block Vector
  • the IBC mode adopts an implementation similar to inter-frame prediction. If the multimedia data frame allows the use of IBC mode, the currently coded but unfiltered image is used as a special reference frame, and the coded data block adopts the method of motion compensation with inter-frame prediction, and the data block to be decoded is derived from this reference frame. Predictive value.
  • This implementation has the following characteristics: 1) BV only uses integer precision; 2) uses a global reference data block set (ie, global reference range), that is, all encoded parts of the current multimedia data frame except some areas are based on parallel processing and chroma interpolation are forbidden to be used as reference data blocks, other can be used as reference data blocks of data blocks to be decoded; 3) In the decoded image cache, the currently encoded and unfiltered image will occupy an image size Cache; 4) The currently coded and unfiltered image will be at the end of the reference image list.
  • a global reference data block set ie, global reference range
  • the computing core can read and write data from on-chip memory or off-chip memory.
  • On-chip memory is usually static memory with very fast read and write speeds, but due to the relatively high price of on-chip memory, usually only smaller on-chip memories are used.
  • the off-chip memory is usually much larger, but if you want to access it efficiently, you need to follow some rules. There is usually a writing-back delay when writing the most recently encoded data block back to the off-chip image memory. .
  • the currently encoded image will be written to the off-chip memory after the loop filter operation is completed for future reference.
  • the relevant data of the reference data block needs to be read from the off-chip memory. This process involves memory bandwidth consumption for writes and reads.
  • the reference image of IBC is a version that has not been processed by loop filtering , which is different from the final output decoded image. Outputting this image off-chip will result in an increase in memory bandwidth.
  • An adjacent data block that has been currently encoded or has just been reconstructed may be used as a reference for the next encoded data block. This requires that the decoding of the reconstructed data block be completed within a limited clock cycle, output to the off-chip memory, and then read into the on-chip memory and other processes. This poses a great challenge to hardware implementation.
  • IBC-related optimization schemes have been adopted in subsequent standards such as VVC, AVS3 and AV1.
  • the IBC scheme in the VVC and AVS3 standards uses a local reference data block set, that is, uses the coded data blocks adjacent to the current coded data block as the reference data blocks of the current coded data block.
  • reference data blocks in the local reference data block set stored in the on-chip memory can be used to solve the above problems.
  • IBC Based on the consideration of hardware implementation cost and encoding performance, IBC only uses the reference data blocks in the local reference data block set stored in the memory of 1 CTU size, and adopts the reference memory reuse strategy as shown in Figure 6.
  • IBC can only search for reference data blocks in these three CUs of size 64x64 and the reconstructed part of the current 64x64CU.
  • the IBC scheme in the AV1 standard uses a global reference data block set, that is, the current coded data block is allowed to refer to the coded data blocks in the current multimedia data frame, but this scheme has the following two defects: a. According to the requirements of writing restrictions, the coded data blocks adjacent to the current coded data block are not allowed to be used as reference data blocks; b. If the current multimedia data frame allows the use of IBC, the loop filter will be disabled. However, screen content sequences often have strong spatial correlation, and adjacent data blocks have high reference value. Allowing adjacent coded data blocks to be referenced will help improve coding performance. On some sequential content, disabling the loop filter will also cause severe multimedia data degradation and affect encoding performance.
  • the IBC scheme based on the local reference data block set does not have the above defects, but due to the limited number of reference data blocks available in the local reference data block set, only using the local reference data block set may degrade the encoding performance of the multimedia data frame .
  • the coding parameters such as the global reference data block set, the local reference data block set and the loop filtering process have different proportions on the coding performance.
  • selecting an appropriate predictive coding method according to the attribute information of the multimedia data frame will be conducive to the improvement of the coding performance That is to say, in this application, according to the attribute information of the current coded data block, from the first reference data block set (ie, the global reference data block set), the second reference data block set (ie, the local reference data block set) ), or the reference data block selected in the third data block set that fuses the second reference data block set and the first data block set, expands the range of reference data blocks that can be selected, and can solve the problem of limited number of reference data blocks; Performing predictive coding on the current coded data block according to the selected reference data block can improve the coding performance of the multimedia data frame.
  • the present application may be executed by a terminal, may also be executed by a server, and may also be executed jointly by the terminal and the server, and the devices used to execute the method in this application may be collectively referred to as computer devices.
  • the server can be an independent physical server, or a server cluster or a distributed system composed of at least two physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, Cloud servers for basic cloud computing services such as cloud communications, middleware services, domain name services, security services, content delivery network (CDN, Content Delivery Network), and big data and artificial intelligence platforms.
  • the terminal can be a smart vehicle terminal, a smart camera, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a speaker with a screen, a smart watch, a smart TV, etc., but is not limited thereto.
  • the terminal and the server may be connected directly or indirectly through wired or wireless communication, and at the same time, the number of the terminal and the server may be one or at least two, which is not limited in this application.
  • the multimedia data frame is a video frame or an image frame obtained by shooting the road with a camera device, and the camera device may refer to a vehicle-mounted terminal or a camera device on the road side; Predictive encoding of the frame is beneficial to improve the encoding performance of the multimedia data frame.
  • the road condition of the road can be observed in real time, and the driving safety of the vehicle can be improved.
  • Fig. 7 is a schematic flow chart of a multimedia data processing method provided in the embodiment of the present application. As shown in Fig. 7, the method may at least include the following S101-S103:
  • the computer device may obtain the multimedia data frame; the multimedia data frame may be captured by the computer device, or the multimedia data frame may be downloaded from the network by the computer device.
  • the multimedia data frame refers to a video frame to be decoded in the video data or a frame of image to be decoded in the image data
  • the video data may refer to live video data, film and television video data, daily life video data, road monitoring video data, etc.
  • the image data may refer to person image data, architectural image data, item image data, and the like.
  • the attribute information of the data block to be encoded in the multimedia data frame can be obtained, the data block to be encoded can refer to the current encoded data block above, and the data block to be encoded can refer to the unencoded data block in the multimedia data frame
  • the data block to be encoded that is, the data block to be encoded may refer to an unencoded data block in the multimedia data frame with the closest distance to the encoded data block.
  • the attribute information of the data block to be encoded includes one or more of the media type of the multimedia data frame to which the data block to be encoded belongs, the reference mark of the multimedia data frame, the definition of the multimedia data frame to which the data block to be encoded belongs, etc.;
  • the types include intra-frame and non-intra-frame.
  • the intra-frame type can refer to a representative multimedia data frame.
  • the intra-frame type includes full intra-frame coding frame, Intra frame, I frame and key frame. Frames of the intra-type only allow the use of intra-frame coding techniques and do not need to rely on other frames for coding.
  • the intra-frame type may refer to a multimedia data frame that can embody the main meaning of video data or image data.
  • the non-intra-frame type may refer to multimedia data frames other than the intra-frame type in video data or image data, and the non-intra-frame type may be called an inter-coded frame, a P frame, a B frame or a non-key frame.
  • Non-intra-frame frames allow the use of inter-frame coding technology and intra-frame coding technology. Usually, when the inter-frame coding technology is used, coding can only be performed after the coding of the reference frame is completed.
  • the reference mark of the multimedia data frame is used to reflect the selection range of the reference data block of the data block to be encoded.
  • the first target reference data block is selected from the first reference data block set, the second reference data block set, or the fusion of the first
  • the first reference data block set and the third reference data block set of the second reference data block set includes encoded data blocks that do not have an adjacent relationship with the data block to be encoded
  • the second reference data block The set includes coded data blocks that have an adjacent relationship with the data block to be coded.
  • the third data block set includes data blocks in the first data block set and the second data block set, that is, the third data block set includes encoded data blocks that do not have an adjacent relationship with the data block to be encoded, and the coded data block that has an adjacent relationship with the data block to be coded.
  • the encoded data blocks that do not have an adjacent relationship between the multimedia data frame and the data block to be encoded, and the encoded data blocks that have an adjacent relationship with the data block to be encoded can be used as reference data blocks.
  • the computer device adaptively selects the first target reference data block from the encoded data blocks with adjacent relationship and the encoded data blocks without adjacent relationship according to the attribute information of the data block to be encoded, which can improve the acquisition The flexibility and accuracy of reference data blocks.
  • the encoded data block in the embodiment of the present application may also be referred to as the reconstructed data block at the encoding end, and the reconstructed data block means that the encoded data block has undergone recovery processing (for example, inverse quantization, Inverse transformation and prediction compensation operations, etc.) after the data block.
  • recovery processing for example, inverse quantization, Inverse transformation and prediction compensation operations, etc.
  • the first target reference data block may refer to an encoded data block that has an adjacent relationship with the data block to be encoded, or the first target reference data block may refer to an The coded data block in the adjacent relationship, or, the first area of the first target reference data block belongs to the coded data block that has an adjacent relationship with the data block to be coded, and the first target reference data block of the first area
  • the second area may belong to an encoded data block that does not have an adjacent relationship with the data block to be encoded.
  • the computer device may determine the block vector between the first target reference data block and the data block to be encoded according to the position information of the first target reference data block and the data block to be encoded respectively in the multimedia data frame, and then, According to the block vector and the first target reference data block prediction, the prediction signal of the data block to be encoded is obtained, and the residual signal of the data block to be encoded is obtained according to the actual signal and the prediction signal of the data block to be encoded, that is, the residual signal of the data block to be encoded
  • the difference signal is the predictive encoding value of the data block to be encoded; by performing predictive encoding on the data block to be encoded, it is beneficial to remove redundancy in the multimedia data frame, improve the transmission efficiency of the multimedia data frame, and save transmission resources.
  • video data includes multiple multimedia data frames
  • image data includes multiple multimedia data frames
  • the data block to be encoded and the first target reference data block all belong to the same multimedia data frame.
  • the computer device obtains the first target reference data block matching the attribute information of the data block to be encoded, and the first target reference data block is selected from the first reference data block set and the second reference data block set , or a third reference data block set that fuses the first reference data block set and the second reference data block set; that is, the alternative reference data block sets include the first reference data block set, the second reference data block set
  • the block set and the third reference data block set expand the selection range of the reference data block; at the same time, according to the attribute information of the data block to be encoded, adaptively select the first target reference data block, and according to the first target reference data block to be encoded
  • the predictive coding of the data block can improve the flexibility and accuracy of obtaining the first target reference data block, and improve the coding performance of the multimedia data frame.
  • by performing predictive coding on the data block to be coded according to the first target reference data block it is beneficial to remove redundancy in the multimedia data frame, improve the transmission efficiency of the multimedia data frame, and
  • the computer device may obtain the first target reference data block that matches the attribute information of the data block to be encoded by one or more combinations of the following three matching methods:
  • Matching method 1 If the media type of the multimedia data frame is an intra-frame type, obtain the first target reference matching the attribute information of the data block to be encoded from the first reference data block set or the third reference data block set
  • Data block if the media type of the multimedia data frame is a non-intra-frame type, obtain a first target reference data block matching the attribute information of the data block to be encoded from the second set of reference data blocks. That is to say, the computer device can adaptively select a reference data block set according to the attribute information of the data block to be encoded, and then determine the first target reference data block from the selected reference data set, thereby improving the flexibility of obtaining the reference data block and accuracy.
  • the number of data blocks in the first set of reference data blocks is more than the number of data blocks in the second set of reference data blocks; the first set of reference data blocks and the second set of reference data blocks are respectively stored in the first storage space and the second storage space space, the first storage space and the second storage space may be located in the same computer device, or in different computer devices.
  • the first storage space and the second storage space may refer to disks or storage medium arrays.
  • the first storage space may refer to an off-chip memory of the computer device, and the second storage space may refer to an on-chip memory of the computer device; or, the first The storage space may refer to the on-chip memory of the computer device, and the second storage space may refer to the off-chip memory of the computer device.
  • the media type of the multimedia data frame is an intra-frame type, it indicates that the importance of the multimedia data frame is relatively high, and the first reference data block set and the second reference data block set (that is, the third reference data block set) Data block set) to search for the data block with the strongest correlation between the data block to be encoded (that is, to search for the optimal data block), as the first target reference data block matching the attribute information of the data block to be encoded, the The first target reference data block is determined by the first set of reference data blocks and the second set of reference data blocks.
  • any data block in the first reference data block set and the second reference data block set is allowed to be used as a reference data block, which expands the selection range of reference data blocks and is beneficial to improve the encoding performance of multimedia data frames.
  • the media type of the multimedia data frame is a non-intra-frame type, indicating that the importance of the multimedia data frame is relatively low
  • the block is used as the first target reference data block matching the attribute information of the data block to be encoded.
  • the first target reference data block is a data block determined according to the first reference data block set and the second reference data block set, and has one of the following meanings: first, the first target reference data block selection From the first reference data block set; second, the first target reference data block is selected from the second reference data block set; third, the first area of the first target reference data block belongs to the first reference data block set, and the The second area of the first target reference data block may belong to the second reference data block set, that is, the first target reference data block partly belongs to the second reference data block set and partly belongs to the first data block set.
  • the correlation between the data block to be coded and the coded data block may be determined according to the similarity between the pixels in the data block to be coded and the pixels in the coded data block.
  • the data block with the strongest correlation with the data block to be encoded can be searched from the first reference data block set as the data block with the strongest correlation with the data block to be encoded.
  • the attribute information of the data block matches the first target reference data block.
  • search for a data block with the strongest correlation with the data block to be encoded from the second reference data block set as an attribute of the data block to be encoded The information matches the first target reference data block.
  • the computer searches the first reference data block set and the second reference data block set for the data block that has the strongest correlation with the data block to be encoded, as the data block that matches the attribute information of the data block to be encoded.
  • the first target reference data block may be from the area formed by the data blocks in the first reference data block set and the second reference data block set, and search for the pixel with the strongest correlation with the pixel in the data block to be encoded Pixels, the region where the pixel with the strongest correlation with the pixels in the data block to be encoded is located is used as the first target reference data block matching the attribute information of the data block to be encoded.
  • Matching mode 2 when the attribute information of the data block to be encoded includes the reference mark of the multimedia data frame to which the data block to be encoded belongs; the computer device acquires the first target reference data that matches the attribute information of the data block to be encoded block, including: if the reference mark of the multimedia data frame is the first reference mark, searching for the data block having the strongest correlation with the data block to be encoded from the first set of reference data blocks or the third set of reference data blocks , as the first target reference data block matching the attribute information of the data block to be encoded; if the reference mark of the multimedia data frame is the second reference mark, search for the data block to be encoded from the second set of reference data blocks The data block with the strongest correlation is used as the first target reference data block matching the attribute information of the data block to be encoded.
  • the reference flag may be preset by the sender (or receiver) of the multimedia data frame. Therefore, adaptively selecting the first target reference data block according to the reference flag is equivalent to adaptively selecting the first target reference data block according to the user's encoding requirements.
  • a target reference data block is beneficial to improve the flexibility and accuracy of the selected reference data block.
  • the first reference mark is used to indicate the reference data block of the data block to be encoded.
  • the selection range is the first reference data block set or the third reference data block set
  • the second reference mark is used to indicate the reference data block of the data block to be encoded.
  • the selection range of is the second set of reference data blocks.
  • the first reference mark and the second reference mark can be preset by the sender (or receiver) of the multimedia data frame, and the first reference mark and the second reference mark can be composed of one or more of numbers, letters, and Chinese characters. constitute. For example, the first reference flag (ibc_ref_flag) is 1, and the second reference flag is 0.
  • the computer device can obtain from the first reference data block set that is related to the to-be-encoded The first target reference data block that the attribute information of the data block matches; when the reference flag of the multimedia data frame in the multimedia data frame is 0, the computer device can obtain the attribute of the data block to be encoded from the second set of reference data blocks The information matches the first target reference data block.
  • the header of the multimedia data frame when the media type of the multimedia data frame is an intra-frame type, the header of the multimedia data frame includes a reference mark; when the media type of the multimedia data frame is a non-intra-frame type, the header of the multimedia data frame may not include a reference mark; or, when the multimedia data frame allows the use of IBC encoding, the header of the multimedia data frame includes a reference mark; when the multimedia data frame does not allow the use of IBC encoding, the multimedia data frame The header may not include the reference flag; at this time, the computer device may determine the first target reference data block matching the attribute information of the data block to be encoded according to whether the header of the multimedia data frame includes the reference flag.
  • the first target reference data block refers to any first reference data block in the first reference data block set; if the reference mark of the multimedia data frame is the second reference mark, search for the data block with the strongest correlation with the data block to be encoded from the second reference data block set, as the first target reference data matching the attribute information of the data block to be encoded
  • the first target reference data block may refer to any second reference data block in the second reference data block set.
  • the reference mark of the multimedia data frame is the first reference mark
  • search for the data block with the strongest correlation with the data block to be encoded from the first reference data block set and the second reference data block set, as the The attribute information of the data block to be coded matches the first target reference data block.
  • the first target reference data block is determined according to the first reference data block set and the second reference data block set; if the multimedia data frame If the reference mark is the second reference mark, search for the data block with the strongest correlation with the data block to be encoded from the second set of reference data blocks as the first data block that matches the attribute information of the data block to be encoded.
  • the first target reference data block may refer to any second reference data block in the second reference data block set.
  • Matching mode 3 when the attribute information of the data block to be encoded includes the definition of the multimedia data frame to which the data block to be encoded belongs, and when the definition of the multimedia data frame to which the data block to be encoded belongs is greater than the definition threshold, it indicates that the user is not interested in the multimedia data frame.
  • the quality requirement of the data frame is relatively high, and the first target reference data block matching the attribute information of the data block to be encoded can be obtained from the first reference data block set or the third reference data block set.
  • the definition of the multimedia data frame to which the data block to be encoded belongs is less than or equal to the definition threshold, it indicates that the user has relatively low requirements on the quality of the multimedia data frame, and the attributes related to the data block to be encoded can be obtained from the second set of reference data blocks
  • the information matches the first target reference data block.
  • the definition of the multimedia data frame can be used to reflect the quality requirements of the user for the multimedia data frame. Therefore, adaptively selecting the first target reference data block according to the definition of the multimedia data frame is equivalent to according to the quality requirement of the user for the multimedia data frame.
  • the adaptive selection of the first target reference data block is conducive to improving the flexibility and accuracy of the selected reference data block.
  • the computer device can Acquire the first target reference data block that matches the attribute information of the data block to be encoded from the second reference data block set, at this time, the computer device can only store the second reference data block set, and does not need to store the first reference data block
  • the collection can save the storage space of the computer equipment, reduce the search range of the reference data block, and improve the efficiency of searching the reference data block.
  • the computer device may select one of the following two methods according to the encoding method of the multimedia data frame to obtain the first set of reference data blocks:
  • the first reference data block set includes one or more first reference data blocks; when the encoding method of the multimedia data frame is a serial encoding method, the first reference data block is the parameter in the multimedia data frame
  • the association relationship satisfies the write-back restriction condition and belongs to the encoded data block of the same encoding slice as the data block to be encoded;
  • the parameter association relationship is used to reflect the first position information, and the size information of the largest encoding block and the second
  • the association relationship between the two position information, the first position information and the second position information respectively refer to the encoded data block and the position information of the to-be-encoded data block in the multimedia data frame;
  • the write-back The restriction condition is used to reflect the time delay of writing the encoded data block into the first set of reference data blocks.
  • the serial encoding method means that only one data block in the multimedia data frame is predictively encoded at the same time. If the encoding method of the multimedia data frame is the serial encoding method, this encoding method will not affect the Identify process impacts.
  • the coded data block can be used as a reference data block after waiting at least a preset time length, and the preset time length is based on the time limit for writing the coded data block to the first reference data block set
  • the limited delay may refer to one of an average delay, a maximum delay, and a minimum delay of writing the encoded data block into the first set of reference data blocks.
  • the first reference data block in the first reference data block set needs to meet the following two conditions: a. Encoding slice; b. Parameter correlation meets write-back restriction conditions.
  • the computer device may perform the following steps to obtain the first set of reference data blocks: when the encoding method of the multimedia data frame is a serial encoding method, respectively obtain the first position information and the maximum encoding block Size information and second position information; the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame. Acquiring a write-back constraint condition; the write-back constraint condition is used to reflect a limited time delay for writing the encoded data block to the first set of reference data blocks.
  • the parameter relationship is used to reflect The first position information, the association relationship between the size information of the largest coding block and the second position information.
  • the first reference data block is added to the first set of reference data blocks.
  • the computer device may separately obtain the first position information, the size information of the largest encoding block, and the second position information.
  • the first position information and the second position information respectively refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, wherein the first position information and the second position information may refer to the encoded data block
  • the size information of the largest encoding block may refer to the width and height of the largest encoding block.
  • the computer device can obtain the write-back restriction condition; from the multimedia data frame, the encoded data block whose parameter association relationship satisfies the write-back restriction condition and belongs to the same encoding slice as the data block to be encoded is selected as the first
  • For reference data blocks add the first reference data block to the first set of reference data blocks.
  • the screening of the first reference data block may be implemented in the following manner: First, according to the first position information and the second position information, the encoded data blocks belonging to the same encoding slice are screened out from the multimedia data frame as The first candidate reference data block. Then, determine the limit number of interval data blocks between the coded data block and the data block to be coded in the multimedia data frame under the write-back restriction condition. Then, count the number of interval data blocks between the first candidate reference data block and the data block to be encoded according to the first position information, the size information of the largest coding block and the second position information. Finally, the first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks is determined as the first reference data block.
  • the first position information and the second position information can refer to the pixels of the encoded data block and the coordinates of the pixels of the data block to be encoded in the multimedia data frame
  • the position information of the encoded data block is (ref_x, ref_y)
  • the position information of the data block to be encoded is (cur_x, cur_y)
  • the computer device can filter out the encoded data block belonging to the same encoding slice as the data block to be encoded from the multimedia data frame, as the first candidate reference data block .
  • the interval data block limit quantity can refer to meeting the write-back restriction condition The minimum number of data blocks that should be separated between the coded data blocks in the next multimedia data frame and the data blocks to be coded.
  • the first position information the size information of the largest coding block and the second position information, count the number of interval data blocks between the first candidate reference data block and the data block to be encoded, and the interval data blocks Quantity refers to the number of data blocks actually spaced between the first candidate reference data block and the data block to be encoded, and the first candidate reference data block whose number of spaced data blocks is greater than the limited number of spaced data blocks is determined as the first reference data block.
  • the first reference data block set can be obtained to ensure that the first reference data block and the data block to be encoded have a strong relationship. Spatial correlation, and can ensure the availability of the first reference data block. At the same time, no additional hardware costs are required, which can save costs.
  • the computer device may determine the first candidate reference data block satisfying the following formula (2) as the first reference data block:
  • cur_num-src_num in the formula (2) represents the interval data block number between the first candidate reference data block and the data block to be encoded
  • cur_num and src_num refer to the numbering of the first candidate reference data block and the data to be encoded respectively
  • the number of the block, D refers to the limited number of data blocks in the interval that meet the write-back restriction conditions
  • cur_num and src_num can be calculated by the following formula (3) and formula (4).
  • the above-mentioned counting of the number of interval data blocks between the first candidate reference data block and the data block to be encoded can be achieved in the following manner: first, determine the first position according to the first position information and the size information of the largest encoding block Row ID and column ID of the candidate reference data block. Then, determine the row identifier and column identifier of the data block to be encoded according to the second position information and the size information of the largest encoding block. Then, the number of data blocks in each row in the multimedia data frame is determined based on the size information of the data blocks to be encoded.
  • the first candidate reference data block and the data block to be encoded are counted.
  • the row identifier and column identifier of the first candidate reference data block are respectively ref_y_num and ref_x_num
  • the row identifier and column identifier of the data block to be coded are respectively cur_y_num and cur_x_num
  • the computer device can respectively use the following formula (3) to obtain the first candidate
  • W and H can be determined in any of the following four ways: Method 1, W and H are determined according to the size information of the largest coding block In the second way, W and H are determined according to the size information of the basic unit (VPDU, Virtual Pipeline Data Units) of the on-chip memory; in the third way, H can be determined by referring to the size information of the largest coding block.
  • the largest coding block is 128x128, H is 128; the largest coding block is 64x64, H is 64, and W can be a default value (such as 64).
  • Mode 4 W and H are preset values, which can be 4, 64, 128, etc.
  • a maximum coding block may include one or more data blocks, and the size of the maximum coding block and the size of the data block may be determined according to the block division method of the multimedia data frame.
  • the computer device may determine the number of data blocks per row in the multimedia data frame based on the size information of the data block to be encoded; according to the number of data blocks per row, the row identifier and the column identifier of the first candidate reference data block, Determine the numbering of the first candidate reference data block, and determine the numbering of the data block to be encoded according to the number of data blocks in each row and the row identifier and column identifier of the data block to be encoded, and set the numbering of the first candidate reference data block
  • the difference with the number of the data block to be encoded is determined as the number of interval data blocks between the first candidate reference data block and the data block to be encoded.
  • the computer device can use the following formula (4) to determine the number of data blocks per row in the multimedia data frame, the number of the first candidate reference data block, and the number of the data block to be encoded:
  • stride in the formula (4) refers to the number of data blocks per row in the multimedia data frame
  • cur_num and src_num refer to the numbering of the first candidate reference data block and the numbering of the data block to be encoded respectively
  • Ceil () represents rounding up
  • tile_w represents the width of the coded slice
  • the coded slice includes one or more largest coded blocks, therefore, the width of the coded slice may be determined according to the size information of the largest coded block. For example, when the size of the largest coding block is 128x128, the coding slice may be 1024x720.
  • the computer device can also obtain the first set of reference data blocks in the following manner: according to the first position information, the second position information and the largest coded block Size information, determine the distance between the encoded data block and the data block to be encoded, and determine the interval limit distance between the encoded data block and the data block to be encoded under the write-back delay condition, the interval limit distance It refers to the minimum separation distance between the coded data block and the data block to be coded under the condition of satisfying the write-back delay, and the multimedia data frame and the data block to be coded belong to the same coding slice, and the distance is greater than the distance limit
  • the coded data block is used as the first reference data block, and the first reference data block is added to the first reference data block set.
  • the first set of reference data blocks includes one or more second reference data blocks; when the encoding mode of the multimedia data frame is a parallel encoding mode or a serial encoding mode, the second reference data blocks are An encoded data block that belongs to the same encoding slice as the data block to be encoded, and the parameter association relationship in the multimedia data frame satisfies the write-back restriction condition, and the parameter association relationship meets the encoding processing conditions; the parameter association relationship is used to reflect The relationship between the coded data block, the position information of the data block to be coded in the multimedia data frame, and the size information of the largest coded block; the write-back restriction is used to reflect the coded data block A limited delay for writing to the first set of reference data blocks.
  • the parallel encoding method refers to performing predictive encoding on multiple data blocks in the multimedia data frame at the same time. If the encoding method of the multimedia data frame is a parallel encoding method, this encoding method will have a negative Influence. At the same time, writing the coded data blocks into the first reference data block set will bring a relatively large write-back delay. It can be seen that when the encoding method of the multimedia data frame is a parallel encoding method, the first reference data block set The second reference data block in the block set needs to meet the following three conditions: a. It belongs to the same coding slice as the data block to be coded; b. The parameter association relationship satisfies the writing-back restriction condition; c.
  • the parameter association relationship meets the encoding processing condition.
  • the encoding manner of the multimedia data frame is a serial encoding manner or a parallel encoding manner
  • the foregoing manner of obtaining the first reference data block set may be used for implementation.
  • the encoding processing conditions may refer to parallel encoding processing conditions.
  • the computer device may perform the following steps to obtain the first set of reference data blocks: obtain the first position information, the size information of the largest coding block, and the second position information; the first position information and the second position information are respectively Refers to the location information of the encoded data block and the data block to be encoded in the multimedia data frame. Then, the write-back constraint condition and the encoding processing condition are acquired; the write-back constraint condition is used to reflect the limited time delay of writing the encoded data block to the first set of reference data blocks.
  • the first position information and the second position information may refer to the coordinates of the upper left pixel of the encoded data block and the upper left pixel of the data block to be encoded in the multimedia data frame
  • the size information of the largest encoded block may refer to the largest The width and height of the encoded block.
  • the write-back restriction condition is used to reflect the limited delay of writing the encoded data block to the first reference data block set
  • the encoding processing condition is used to reflect the difference between the reference data block and the to-be-encoded data block when encoding the multimedia data frame in parallel.
  • the limited distance between data blocks; or, the encoding processing condition is used to reflect the limited number of interval data blocks between the reference data block and the data block to be encoded when the multimedia data frame is encoded in parallel.
  • the Adding the second reference data block to the first reference data block set can ensure that the second reference data block has a strong spatial correlation with the data block to be encoded, and can ensure the availability of the second reference data block.
  • the screening of the second reference data block may be implemented in the following manner: according to the first position information and the second position information, the encoded data that belongs to the same encoding slice as the data block to be encoded is selected from the multimedia data frame block as the first candidate reference data block. Then, the first candidate reference data block whose parameter association relationship satisfies the write-back restriction condition is selected from the first candidate reference data block as the second candidate reference data block. Then, determine the first data block limit distance between the coded data block in the multimedia data frame and the data block to be coded under the condition of write-back restriction. Determine the second data block limit distance between the coded data block in the multimedia data frame and the data block to be coded under the encoding processing condition.
  • the size information of the largest coding block, the second position information, the first data block limit distance and the second data block limit distance determine the second candidate reference data block that satisfies the encoding processing conditions.
  • the two candidate reference data blocks are used as the second reference data block.
  • the first position information and the second position information can refer to the pixels of the encoded data block and the coordinates of the pixels of the data block to be encoded in the multimedia data frame
  • the position information of the encoded data block is (ref_x, ref_y)
  • the position information of the data block to be encoded is (cur_x, cur_y)
  • the computer device can filter out the encoded data block belonging to the same encoding slice as the data block to be encoded from the multimedia data frame, as the first candidate reference data block .
  • the computer device may select the first candidate reference data block whose coordinates satisfy the above formula (2) from the first candidate reference data block as the second candidate reference data block.
  • the first data block limit distance between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back limit condition can be defined as meeting the write-back delay limit condition
  • the encoded data block in the multimedia data frame, and the minimum distance that should be kept between the data block to be encoded determine the second distance between the encoded data block in the multimedia data frame and the data block to be encoded under the encoding process conditions
  • the data block limit distance, the second data block limit distance refers to the minimum distance that should be kept between the coded data block in the multimedia data frame and the data block to be coded under the conditions of encoding processing.
  • the second candidate reference data block it can be determined from the second candidate reference data block according to the first position information, the size information of the largest coding block, the second position information, the first data block limit distance and the second data block limit distance.
  • the second candidate reference data block that satisfies the encoding processing condition is used as the second reference data block.
  • the above-mentioned determination of the second candidate reference data block that satisfies the encoding processing conditions from the second candidate reference data blocks as the second reference data block may also be achieved in the following manner: according to the first position information and the maximum encoding
  • the size information of the block determines the row identification and column identification of the second candidate reference data block. Determine the row identifier and column identifier of the data block to be encoded according to the second position information and the size information of the largest encoding block. Then, the second candidate reference data block whose row identifier is less than or equal to the row identifier of the data block to be encoded is selected from the second candidate reference data blocks as the third candidate reference data block.
  • the row distance between the third candidate reference data block and the data block to be encoded is determined according to the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded.
  • the column distance between the third candidate reference data block and the data block to be encoded is determined according to the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded.
  • the third candidate reference data block whose column distance is smaller than the first distance threshold in the third candidate reference data block is determined as the second reference data block; the first distance threshold is based on the first data block limit distance, the second data block The limit distance and the row distance between the third candidate reference data block and the data block to be encoded are determined.
  • the computer device can respectively use the above formula (3) to obtain the row identifier and column identifier of the second candidate reference data block, and the row identifier and column identifier of the data block to be encoded, and then, can filter from the second candidate reference data block A second candidate reference data block whose row identifier is less than or equal to the row identifier of the data block to be encoded is used as a third candidate reference data block.
  • the difference between the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded may be determined as the row distance between the third candidate reference data block and the data block to be encoded , determining the difference between the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded as the column distance between the third candidate reference data block and the data block to be encoded; and the Among the third candidate reference data blocks, the third candidate reference data blocks whose column distance is smaller than the first distance threshold are determined as the second reference data blocks.
  • the computer device may determine the second candidate reference data block whose coordinates satisfy the following formula (5) as the first reference data block:
  • ref_y_num and ref_x_num refer to the row identifier and column identifier of the second candidate reference data block respectively
  • cur_y_num and cur_x_num refer to the row identifier and column identifier of the data block to be encoded respectively
  • P refers to the The first data block limit distance between the encoded data block in the multimedia data frame and the data block to be encoded under the writing restriction condition
  • E refers to the encoded data block in the multimedia data frame under the encoding processing condition
  • the second data block limits the distance between the data block and the data block to be encoded.
  • the first data block limit distance may be determined according to the interval between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back limit condition, for example, P and the above-mentioned D
  • the values may be the same;
  • the second data block limit distance is determined according to the interval limit data block quantity between the coded data block in the multimedia data frame and the uncoded data block under the encoding processing condition.
  • ref_x_num-cur_x_num represents the column distance between the third candidate reference data block and the data block to be coded
  • cur_y_num-ref_y_num represents the row distance between the third candidate reference data block and the data block to be coded.
  • the computer device can also obtain the first set of reference data blocks in the following manner: according to the first position information, the second position information and The size information of the largest coded block determines the number of interval data blocks between the coded data block and the data block to be coded, and determines the coded data block and the coded data block under the write-back delay condition and the code processing condition
  • the target interval limits the number of data blocks.
  • the target interval limits the number of data blocks.
  • the coded data blocks in the data frame that belong to the same coding slice as the data block to be coded and whose number of interval data blocks is greater than the number of target interval limit data blocks are added to the first set of reference data blocks.
  • the computer device may select one of the following two ways or a combination of at least two ways to obtain the second set of reference data blocks:
  • the second reference data block set includes one or more third reference data blocks, and the third reference data blocks are encoded data blocks in the multimedia data frame that are located in the same encoding processing unit as the data block to be encoded.
  • the encoding processing unit can also be called the basic unit of the on-chip memory (VPDU, Virtual Pipeline Data Units).
  • the size information of the VPDU can be determined according to the encoding processing capability of the encoder. For example, the size of the VPDU is VPDU_W*VPDU_H, VPDU_W and VPDU_H Values can all be 4, 8, 16, 32, 64, 128, etc.
  • the encoded data blocks in the VPDU 64x64 to which the data block to be encoded belongs can be used as the reference data block of the data block to be encoded.
  • each pixel of the reference data block of the data block to be encoded in the multimedia data frame may be located in the same VPDU.
  • the second set of reference data blocks includes one or more fourth reference data blocks, the fourth reference data blocks are adjacent to the data block to be encoded in the multimedia data frame, and belong to the same An encoded chunk of data for an encoded slice.
  • the adjacent relationship between the second reference data block and the data block to be encoded may include one or more combinations of the following three situations: a.
  • the fourth reference data block has an adjacent relationship with the data block to be encoded It means: the fourth reference data block and the data block to be encoded belong to the same largest encoding block. b.
  • the fourth reference data block has an adjacent relationship with the data block to be encoded means: the fourth reference data block is located in the N largest encoding blocks adjacent to the data block to be encoded, and the size of the largest encoding block is smaller than the size threshold; N is determined according to the size information of the largest coding block.
  • the encoding direction of multimedia data frames is from left to right, and the size threshold can be dynamically set according to the application scenario or user needs.
  • the size threshold can be 128x128, then the first reference data block and the data block to be encoded have the same
  • the neighbor relationship means that: the fourth reference data block is located in the N largest coding blocks on the left of the data block to be coded, and the size of the largest coding block is smaller than 128x128.
  • N can be determined according to the following formula (6):
  • log2_sb_size refers to the intermediate variable
  • sb_size refers to the size of the largest coding block.
  • the fourth reference data block has an adjacent relationship with the data block to be encoded means that the fourth reference data block is located in the N largest encoding blocks adjacent to the data block to be encoded, and the size of the largest encoding block is greater than or equal to the size Threshold; in the multimedia data frame, if the fourth reference data block moves (for example, moves to the right) after a maximum coding block to the first direction, the moved fourth reference data block is located in the uppermost corner in the second direction ( For example, the data block in the upper left corner) is an unrebuilt data block.
  • the first direction and the second direction are two opposite directions, wherein the first direction refers to the moving direction of the fourth reference data block, and the first direction is the same as the coding direction of the data block to be coded, that is, for For a row of data blocks to be encoded horizontally, from left to right is the encoding direction of the data blocks to be encoded, then from left to right is the above-mentioned first direction, and from right to left is the above-mentioned second direction.
  • the position information when the fourth reference data block is moved (the moving direction is the same as the encoding direction) by the distance of a maximum encoding block is different from the position information of the data block to be encoded; N is determined according to the size information of the largest encoding block.
  • the fourth reference data block needs to meet the following three conditions: 1. It belongs to the same coding slice as the data block to be coded 2. After the fourth reference data block is moved to the right by 128 pixels, the coordinates are (ref_x+128, ref_y), then the coordinates in the multimedia data frame are (ref_x+128, ref_y) and the size is 64x64 3.
  • the position information of the fourth reference data block when it moves to the right by the distance of the largest coding block is different from the position information of the data block to be coded.
  • the condition can be expressed by formula (7):
  • the distance between the first target reference data block and the data block to be encoded is smaller than the second distance threshold;
  • the second distance threshold can be dynamically set according to the application scenario or user demand, such as the second
  • the distance threshold may be 2 14 , and at the same time, the first target reference data block is located at an integer pixel position of the multimedia data frame.
  • the maximum encoding block size is 128*128, and the interval between encoded data blocks and unencoded data blocks limits the number of data blocks under the condition of write-back delay limitation is 4, the interval between the encoded data block and the data block to be encoded under the encoding processing condition limits the number of data blocks to 2, the acquisition method of the second reference data block set is the above method 1, and when the size of the VPDU is 64x64, The relationship between the first set of reference data blocks, the second set of reference data blocks and the data blocks to be encoded is shown in FIG. 8 . In FIG.
  • the size of a smallest square (that is, the small square drawn by the dotted line) is 64x64, the size of the small square drawn by the solid line is 128x128, and the size of the data block is 32x32. Therefore, in the multimedia data frame in Fig. 8, the coded data block in the 19th grid of the first row belongs to the second reference data block set, and the second reference data block set is used for the 19th grid of the first row
  • the data blocks to be encoded in the grid are encoded.
  • the coded data blocks separated by 8 data blocks from the data block to be coded in the first row and the second row belong to the first reference data block set, and the first reference data block set is used for the first row
  • the data block to be encoded in the 19th grid is encoded. It can be seen from Fig. 8 that the number of reference data blocks in the first set of reference data blocks is greater than the number of reference data blocks in the second set of reference data blocks, and the reference data blocks in the first set of reference data blocks have no similarity with the data blocks to be encoded. Adjacent relationship, the reference data block in the second set of reference data blocks has an adjacent relationship with the data block to be encoded.
  • the maximum encoding block size is 128*128, and the interval between encoded data blocks and unencoded data blocks limits the number of data blocks under the condition of write-back delay limitation is 4, the interval between the encoded data block and the data block to be encoded under the encoding processing condition limits the number of data blocks to 2, the acquisition method of the second reference data block set is the above-mentioned method 2, and when the size of the VPDU is 64x64, The relationship among the first set of reference data blocks, the second set of reference data blocks and the data blocks to be encoded is shown in FIG. 9 . In FIG.
  • the size of a smallest square (that is, the small square drawn by the dotted line) is 64x64, the size of the small square drawn by the solid line is 128x128, and the size of the data block is 32x32. Therefore, in the multimedia data frame in Fig. 9, the coded data block located in the 19th grid of the first row and the coded data block located in the largest coded block to the left of the data block to be coded belong to the second reference data A block set, the second reference data block set is used to encode the data block to be encoded in the 19th grid of the first row.
  • the coded data blocks separated by 8 data blocks from the data block to be coded in the first row and the second row belong to the first reference data block set, and the first reference data block set is used for the first row
  • the data block to be encoded in the 19th grid is encoded. It can be seen from Fig. 9 that the number of reference data blocks in the first set of reference data blocks is greater than the number of reference data blocks in the second set of reference data blocks, and the reference data blocks in the first set of reference data blocks have no similarity with the data blocks to be encoded. Adjacent relationship, the reference data block in the second set of reference data blocks has an adjacent relationship with the data block to be encoded.
  • the first Both the luminance component and the chrominance component of the target reference data block need to meet the conditions related to the first reference data block, and the conditions related to the first reference data block may refer to the above-mentioned conditions for determining the first reference data block set.
  • the chrominance component of the data block to be coded is obtained by predictive coding according to the luminance component of the first target reference data block, and the first target reference data block is the second reference data block
  • the first target reference data block Both the luminance component and the chrominance component need to meet the conditions related to the second reference data block, and the conditions related to the second reference data block may refer to the above-mentioned conditions for determining the second reference data block set.
  • having an adjacent relationship between the data block to be encoded and the encoded data block may mean that the encoding order of the data block to be encoded is adjacent to the encoding order of the encoded data block; There is no adjacent relationship between the data blocks may mean that the encoding order of the data blocks to be encoded is not adjacent to the encoding order of the encoded data blocks.
  • the number of data blocks to be encoded in the multimedia data frame is multiple, and the attribute information of each data block to be encoded in the multimedia data frame can be obtained; when according to the attribute information of each data block to be encoded in the multimedia data frame, determine When the first target reference data blocks corresponding to all the data blocks to be encoded in the multimedia data frame belong to the second set of reference data blocks, filter processing is performed on the data blocks to be encoded in the multimedia data frame; The attribute information of the data block, when it is determined that there is a first target reference data block corresponding to the data block to be encoded in the multimedia data frame is selected from the first reference data block set, suspend the filtering process on the data block to be encoded in the multimedia data frame.
  • the filtering process on the data blocks to be decoded in the multimedia data frame is suspended. That is to say, at the encoding end, when the data blocks in the multimedia data frame are all encoded by the reference data blocks in the second reference data block set, it is allowed to use a loop filter to filter the multimedia data frame, which is beneficial to improve The quality of the multimedia data frame.
  • the reference data block in the first reference data block set is used for encoding, or, when the data block in the multimedia data frame exists, the first target reference data block part belongs to the first reference data block set, And when some of them belong to the second reference data block set, the use of the loop filter to filter the multimedia data frame is prohibited. By prohibiting the use of the loop filter to process the multimedia data frame, additional image storage requirements are avoided, which is beneficial to Save storage space.
  • Loop filters can include but are not limited to: Deblocking filter (DBF, Deblocking filter), sample point adaptive compensation (SAO, Sample Adaptive Offset), adaptive loop filter (ALF, Adaptive Loop Filter), constraint direction Enhancement filter (CDEF, constrained directional enhancement), loop recovery filter (LR, loop restoration), cross-component sample point compensation (CCSO, Cross Component Sample Offset), etc.
  • DPF Deblocking filter
  • SAO Sample Adaptive Offset
  • ALF adaptive loop filter
  • CDEF constraint direction Enhancement filter
  • LR loop recovery filter
  • CCSO Cross Component Sample Offset
  • Fig. 10 is a schematic flow chart of a multimedia data processing method provided by an embodiment of the present application. As shown in Fig. 10, the method may at least include the following S201-S203:
  • the computer device may acquire the attribute information of the data block to be decoded in the multimedia data frame
  • the data block to be decoded may refer to the current coded data block above
  • the data block to be decoded may refer to the multimedia data frame
  • the data block to be decoded may refer to the uncoded data block in the multimedia data frame with the closest distance to the decoded data block.
  • the attribute information of the data block to be decoded includes one or more of the media type of the multimedia data frame to which the data block to be decoded belongs, the reference mark of the multimedia data frame, the definition of the multimedia data frame to which the data block to be decoded belongs, etc.;
  • the types include intra-frame and non-intra-frame.
  • the intra-frame type can refer to a representative multimedia data frame.
  • the intra-frame type includes full intra-frame coding frame, Intra frame, I frame and key frame. Frames of the intra-type only allow the use of intra-frame coding techniques and do not need to rely on other frames for coding.
  • the intra-frame type may refer to a multimedia data frame that can embody the main meaning of video data or image data.
  • the non-intra-frame type may refer to multimedia data frames other than the intra-frame type in video data or image data, and the non-intra-frame type may be called an inter-coded frame, a P frame, a B frame or a non-key frame.
  • Non-intra-frame frames allow the use of inter-frame coding technology and intra-frame coding technology. Usually, when the inter-frame coding technology is used, coding can only be performed after the coding of the reference frame is completed.
  • the reference flag of the multimedia data frame is used to reflect the selection range of the reference data block of the data block to be decoded.
  • the decoded data block that does not have an adjacent relationship with the data block to be decoded and the decoded data block that has an adjacent relationship with the data block to be decoded can be used as a reference data block.
  • the computer device adaptively selects the second target reference data block from the decoded data blocks with adjacent relationship and the decoded data blocks without adjacent relationship according to the attribute information of the data block to be decoded, so that the acquisition rate can be improved.
  • the flexibility and accuracy of reference data blocks can be used as a reference data block.
  • the decoded data block in the embodiment of the present application may also be referred to as the reconstructed data block at the decoding end, and the reconstructed data block refers to the data block after the decoded data block has been restored.
  • the second target reference data block may refer to a decoded data block that has an adjacent relationship with the data block to be decoded, or, the second target reference data block may refer to a decoded data block that does not have an adjacent relationship with the data block to be decoded block, or, the second area of the second target reference data block belongs to the decoded data block having an adjacent relationship with the data block to be decoded, and the second area of the second target reference data block may belong to the data block to be decoded A decoded data block in which the data block does not have an adjacency relationship.
  • the computer device may determine the block between the second target reference data block and the data block to be decoded according to the position information of the second target reference data block and the data block to be decoded respectively in the multimedia data frame. vector, and then, predict according to the block vector and the second target reference data block to obtain a prediction signal of the data block to be decoded, and add the prediction signal to the residual signal of the data block to be decoded to obtain the original signal of the data block to be decoded .
  • decoding the data block to be decoded it is beneficial to remove the multimedia data frame for recovery processing and improve the quality of the multimedia data frame.
  • video data includes multiple multimedia data frames
  • image data includes multiple multimedia data frames
  • the data block to be decoded and the second target reference data block all belong to the same multimedia data frame.
  • the computer device acquires a second target reference data block that matches the attribute information of the data block to be decoded, and the second target reference data block belongs to the first reference data block set and the second reference data block set, Or a third reference data block set that fuses the first reference data block set and the second reference data block set; that is, the alternative reference data block sets include the first reference data block set, the second reference data block set set and the third set of reference data blocks expand the selection range of reference data blocks; at the same time, according to the attribute information of the data block to be decoded, adaptively select the second target reference data block, and the data to be decoded according to the second target reference data block
  • the flexibility and accuracy of obtaining the second target reference data block can be improved, and the decoding performance of the multimedia data frame can be improved.
  • the computer device may obtain the second target reference data block that matches the attribute information of the data block to be decoded by using one or more combinations of the following three matching methods:
  • Matching method 1 if the media type of the multimedia data frame is an intra-frame type, then obtain the second object matching the attribute information of the data block to be decoded from the first reference data block set or the third reference data block set Reference data block; the second target reference data block is determined according to the first reference data block and the second reference data block; if the media type of the multimedia data frame is a non-intra-frame type, then from the second reference data block A second target reference data block matching the attribute information of the data block to be decoded is obtained from the set.
  • There is an adjacent relationship between the data block to be decoded and the decoded data block may mean that the decoding order of the data block to be decoded is adjacent to the decoding order of the decoded data block; Having an adjacent relationship may mean that the decoding sequence of the data block to be decoded is not adjacent to the decoding sequence of the decoded data block.
  • first set of reference data blocks and the second set of reference data blocks are respectively stored in the first storage space and the second storage space, and the first storage space and the second storage space may be located in the same computer device, or may be on different computer devices.
  • the data block with the strongest correlation with the data block to be decoded can be searched from the first set of reference data blocks as the data block with the strongest correlation with the data block to be decoded.
  • the attribute information of the data block matches the second target reference data block.
  • search for a data block with the strongest correlation with the data block to be decoded from the second reference data block set as an attribute of the data block to be decoded The information matches the second target reference data block.
  • Matching mode 2 when the attribute information of the data block to be decoded includes the reference sign of the multimedia data frame to which the data block to be decoded belongs; the computer device obtains the second target reference data that matches the attribute information of the data block to be decoded block, including: if the reference mark of the multimedia data frame is the first reference mark, obtaining a second target matching the attribute information of the data block to be decoded from the first set of reference data blocks or the third set of reference data blocks Reference data block; if the reference mark of the multimedia data frame is a second reference mark, obtain a second target reference data block matching the attribute information of the data block to be decoded from the second set of reference data blocks.
  • the reference flag may be preset by the sender (or receiver) of the multimedia data frame. Therefore, adaptively selecting the second target reference data block according to the reference flag is equivalent to adaptively selecting the second target reference data block according to the user's coding requirements.
  • the second target reference data block is beneficial to improve the flexibility and accuracy of the selected reference data block.
  • the first reference mark is used to indicate the reference data block of the data block to be decoded.
  • the selection range is the first reference data block set or the third reference data block set, and the second reference mark is used to indicate the reference data block of the data block to be decoded.
  • the selection range of is the second set of reference data blocks.
  • the first reference mark and the second reference mark can be preset by the sender (or receiver) of the multimedia data frame, and the first reference mark and the second reference mark can refer to one or more of numbers, letters, and Chinese characters kind of composition.
  • the reference mark of the multimedia data frame is the second reference mark
  • search for the data to be decoded from the first reference data block set and the second reference data block set that is, the third reference data block set
  • the data block with the strongest correlation between the blocks is used as the second target reference data block matching the attribute information of the data block to be decoded.
  • the second target reference data block is based on the second reference data block set and the first Two sets of reference data blocks are determined; if the reference mark of the multimedia data frame is the second reference mark, then search for the data block with the strongest correlation between the data block to be decoded from the second set of reference data blocks, as A second target reference data block matching the attribute information of the data block to be decoded.
  • matching method 2 if the reference mark of the multimedia data frame is the second reference mark, search for the data block with the strongest correlation with the data block to be decoded from the first set of reference data blocks as the data block with the strongest correlation with the data block to be decoded.
  • the second target reference data block that matches the attribute information of the decoded data block refers to any reference data block in the first reference data block set; if the reference mark of the multimedia data frame is the first Two reference marks, then search for the data block having the strongest correlation with the data block to be decoded from the second reference data block set as the second target reference data block matching the attribute information of the data block to be decoded, At this time, the second target reference data block may refer to any reference data block in the second reference data block set.
  • Matching mode 3 when the attribute information of the data block to be decoded includes the definition of the multimedia data frame to which the data block to be decoded belongs, and when the definition of the multimedia data frame to which the data block to be decoded belongs is greater than the definition threshold, it indicates that the user is not interested in the multimedia data frame.
  • the quality requirement of the data frame is relatively high, and the second target reference data block matching the attribute information of the data block to be decoded can be obtained from the first reference data block set or the third reference data block set.
  • the definition of the multimedia data frame to which the data block to be decoded belongs is less than or equal to the definition threshold, it indicates that the user has relatively low requirements on the quality of the multimedia data frame, and the attributes related to the data block to be decoded can be obtained from the second set of reference data blocks.
  • the information matches the second target reference data block.
  • the definition of the multimedia data frame can be used to reflect the quality requirements of the user for the multimedia data frame. Therefore, adaptively selecting the second target reference data block according to the definition of the multimedia data frame is equivalent to, according to the user's requirement for the quality of the multimedia data frame
  • the demand-adaptive selection of the second target reference data block is conducive to improving the flexibility and accuracy of the selected reference data block.
  • the computer device can select one of the following two methods according to the decoding method of the multimedia data frame to obtain the first set of reference data blocks:
  • Method 1 when the decoding method of the multimedia data frame is serial decoding mode, the first reference data block in the first reference data block set is a decoded data block whose parameter correlation in the multimedia data frame satisfies the write-back restriction condition and belongs to the same decoded slice as the data block to be decoded; the parameter The association relationship is used to reflect the association relationship between the first position information, the size information of the largest decoded block, and the second position information.
  • the first position information and the second position information refer to the decoded data block, the undecoded data block, The location information of the data block in the multimedia data frame; the write-back restriction condition is used to reflect the limited time delay of writing the decoded data block to the first set of reference data blocks. That is to say, the serial decoding method means that only one data block in the multimedia data frame is decoded at the same time. If the decoding method of the multimedia data frame is the serial decoding method, the decoding method will not determine the reference data block. process affects.
  • the decoded data block can be used as a reference data block after waiting at least a preset time length, and the preset time length is based on the time limit for writing the decoded data block to the first set of reference data blocks
  • the limited delay may refer to one of an average delay, a maximum delay, and a minimum delay for writing the decoded data block into the first set of reference data blocks.
  • the first reference data block in the first reference data block set needs to meet the following two conditions: a. Decoding slice; b. Parameter correlation meets write-back restriction conditions.
  • Mode 2 when the decoding mode of the multimedia data frame is a parallel decoding mode or a serial decoding mode, the first reference data block of the first reference data block set belongs to the same decoding slice as the data block to be decoded, and The parameter association relationship in the multimedia data frame satisfies the write-back restriction condition, and the decoded data block whose parameter association relationship meets the decoding processing condition; the parameter association relationship is used to reflect the decoded data block and the data block to be decoded respectively The relationship between the location information in the multimedia data frame and the size information of the largest decoded block; the write-back restriction condition is used to reflect the limitation of writing the decoded data block to the first set of reference data blocks delay.
  • the parallel decoding method refers to performing predictive decoding on multiple data blocks in the multimedia data frame at the same time. If the decoding method of the multimedia data frame is a parallel decoding method, the decoding method will cause problems to the determination process of the reference data block. Influence. At the same time, writing the decoded data blocks into the first set of reference data blocks will bring a large write-back delay. It can be seen that when the decoding method of the multimedia data frame is a parallel decoding method, the first reference data block set The first reference data block in the block set needs to meet the following three conditions: a. It belongs to the same decoding slice as the data block to be decoded; b. The parameter association relationship meets the write-back restriction condition; c.
  • the parameter association relationship meets the decoding processing condition.
  • the decoding processing conditions may refer to parallel decoding processing conditions.
  • the manner of obtaining the first reference data block set can refer to the manner of obtaining the first reference data block set in the coding segment, and the repetition will not be repeated.
  • the computer device may select one or at least two combinations of the following two methods to obtain the second reference data block set: Method 1: the second reference data block of the second reference data block set It is a decoded data block in the same decoding processing unit as the data block to be decoded in the multimedia data frame.
  • Mode 2 The second reference data block of the second reference data block set is a decoded data block in the multimedia data frame that is adjacent to the data block to be decoded and belongs to the same decoded slice as the data block to be decoded.
  • having an adjacent relationship between the second reference data block and the data block to be decoded may be one or more combinations of the following situations: a, having an adjacent relationship between the second reference data block and the data block to be decoded means: The second reference data block and the data block to be decoded belong to the same largest decoding block. b.
  • the second reference data block has an adjacent relationship with the data block to be decoded: the second reference data block is located in the N largest decoding blocks adjacent to the data block to be decoded, and the size of the largest decoding block is smaller than the size threshold; N is determined according to the size information of the largest decoded block.
  • the distances between the first reference data block, the second reference data block, and the data block to be decoded are all smaller than a second distance threshold;
  • the second distance threshold can be dynamically set according to application scenarios or user requirements , for example, the second distance threshold may be 2 14 , and at the same time, the first reference data block and the second reference data block are located at integer pixel positions of the multimedia data frame.
  • the number of data blocks to be decoded in the multimedia data frame is multiple, and the attribute information of each data block to be decoded according to the multimedia data frame is obtained; when according to the attribute information of each data block to be decoded in the multimedia data frame, determine all in the multimedia data frame When the second target reference data blocks corresponding to the data blocks to be decoded belong to the second reference data block set, filter the data blocks to be decoded in the multimedia data frame; when according to the attribute information of each data block to be decoded in the multimedia data frame, When it is determined that the second target reference data block corresponding to the data block to be decoded in the multimedia data frame belongs to the first set of reference data blocks, the filtering process on the data block to be decoded in the multimedia data frame is suspended.
  • the first area belongs to the first reference data block set, and the second target reference data
  • the filtering process on the data blocks to be decoded in the multimedia data frame is suspended.
  • FIG. 11 is a schematic structural diagram of a multimedia data processing device provided in an embodiment of the present application.
  • the multimedia data processing device may be a computer program (including program code) running on a computer device.
  • the multimedia data processing device is an application Software; the device can be used to execute the corresponding steps in the method provided by the embodiment of the present application.
  • the multimedia data processing apparatus may include: a first acquiring module 111 , a first matching module 112 and an encoding module 113 .
  • the first obtaining module is configured to obtain the attribute information of the data block to be encoded in the multimedia data frame; the first matching module is configured to obtain the first target reference data block that matches the attribute information of the data block to be encoded; wherein, The first target reference data block is selected from a first reference data block set, a second reference data block set, or a third reference data block set fused with the first reference data block set and the second reference data block set , the first set of reference data blocks includes encoded data blocks that have no adjacent relationship with the data block to be encoded, and the second set of reference data blocks includes Adjacent encoded data blocks; an encoding module configured to perform predictive encoding on the to-be-encoded data block according to the first target reference data block.
  • the attribute information of the data block to be encoded includes the media type of the multimedia data frame to which the data block to be encoded belongs; the first matching module is further configured to: if the media type of the multimedia data frame is an intra-frame type, then acquire a first target reference data block that matches the attribute information of the data block to be encoded from the first set of reference data blocks or the third set of reference data blocks; if the If the media type of the multimedia data frame is a non-intra-frame type, the first target reference data block matching the attribute information of the data block to be encoded is obtained from the second set of reference data blocks.
  • the attribute information of the data block to be encoded includes the reference mark of the multimedia data frame to which the data block to be encoded belongs; the first matching module is further configured to: if the reference mark of the multimedia data frame is the first reference flag, then obtain the first target reference data block that matches the attribute information of the data block to be encoded from the first set of reference data blocks or the third set of reference data blocks; if the If the reference mark of the multimedia data frame is the second reference mark, the first target reference data block matching the attribute information of the data block to be encoded is acquired from the second set of reference data blocks.
  • the first reference data block set includes one or more first reference data blocks; when the encoding method of the multimedia data frame is a serial encoding method, the first reference data block is the multimedia data frame
  • the parameter association relationship in the data frame satisfies the write-back restriction condition, and belongs to the encoded data block of the same encoding slice as the data block to be encoded; the parameter association relationship is used to reflect the first position information, and the largest encoding block
  • the association relationship between the size information and the second position information, the first position information and the second position information respectively refer to the encoded data block and the position information of the to-be-encoded data block in the multimedia data frame
  • the write-back constraint condition is used to reflect a limited delay of writing the encoded data block into the first set of reference data blocks.
  • the first obtaining module is further configured to separately obtain the first position information, the size information and the second position information of the largest coding block when the encoding mode of the multimedia data frame is a serial encoding mode ;
  • the first position information and the second position information respectively refer to the encoded data block and the position information of the data block to be encoded in the multimedia data frame; obtain the write-back restriction condition; the write-back restriction condition It is used to reflect the limited delay of writing the encoded data block to the first set of reference data blocks; filter out the parameter association relationship from the multimedia data frame to meet the write-back restriction condition, and the
  • the coded data block belongs to the coded data block of the same coded slice as the first reference data block; the parameter correlation is used to reflect the first position information, the size information of the largest coded block and the second An association relationship between the two position information; adding the first reference data block to the first reference data block set.
  • the first acquiring module is further configured to: according to the first position information and the second position information, select from the multimedia data frame the The encoded data block of the slice is used as the first candidate reference data block; and the interval data block between the encoded data block in the multimedia data frame and the to-be-encoded data block is determined under the write-back restriction condition Restricting the number; counting the number of interval data blocks between the first candidate reference data block and the data block to be encoded according to the first position information, the size information of the largest coding block and the second position information ; Determining the first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks as the first reference data block.
  • the first obtaining module is further configured to: determine the row identifier and column identifier of the first candidate reference data block according to the first position information and the size information of the largest coding block; Information and the size information of the largest coded block determine the row identifier and column identifier of the data block to be coded; determine the number of data blocks per row in the multimedia data frame based on the size information of the data block to be coded; according to the The number of data blocks in each row, the row identifier and column identifier of the first candidate reference data block, and the row identifier and column identifier of the data block to be coded, count the first candidate reference data block and the pending The number of interval data blocks between encoded data blocks.
  • the first set of reference data blocks includes one or more second reference data blocks; the second reference data blocks belong to the same encoding slice as the data block to be encoded, and the multimedia
  • the parameter association relationship in the data frame satisfies the write-back restriction condition, and the encoded data block whose parameter association relationship meets the encoding processing condition; the parameter association relationship is used to reflect the encoded data block and the to-be-encoded data block respectively The relationship between the position information in the multimedia data frame and the size information of the largest coded block; the write-back restriction condition is used to reflect that the coded data block is written into the first reference data Bound latency for block collection.
  • the first obtaining module is further configured to obtain the first position information, and the size information and the second position information of the largest coding block respectively; the first position information and the second position information respectively refer to the The coded data block, the location information of the data block to be coded in the multimedia data frame; obtain the write-back restriction condition and the encoding processing condition; the write-back restriction condition is used to reflect the writing of the encoded data block Enter the limited delay into the first set of reference data blocks; screen out the parameter association relationship from the multimedia data frame to meet the write-back restriction conditions and the encoding processing conditions, and belong to the same data block as the data block to be encoded
  • the coded data block of the coded slice is used as the second reference data block; the parameter correlation is used to reflect the relationship between the first position information, the size information of the data block to be coded and the second position information association relationship; adding the second reference data block to the first reference data block set.
  • the first acquiring module is further configured to: according to the first location information and the second location information, filter out from the multimedia data frame the information that belongs to the same encoding slice as the data block to be encoded.
  • the encoded data block is used as a first candidate reference data block; the first candidate reference data block whose parameter association relationship meets the write-back restriction condition is selected from the first candidate reference data block as a second candidate reference data block; Determine the first data block limit distance between the encoded data block in the multimedia data frame under the write-back restriction condition and the data block to be encoded; determine the multimedia data under the encoding processing condition A second data block limit distance between the coded data block in the frame and the data block to be coded; according to the first position information, the size information of the largest coded block and the second position information, the The first data block limit distance and the second data block limit distance are used to determine a second candidate reference data block that satisfies the encoding processing condition from the second candidate reference data blocks as the second reference data block.
  • the first obtaining module is further configured to: determine the row identifier and column identifier of the second candidate reference data block according to the first position information and the size information of the largest coding block; The second position information and the size information of the largest coding block determine the row identifier and column identifier of the data block to be encoded; filter out rows whose row identifier is less than or equal to the data block to be encoded from the second candidate reference data block
  • the identified second candidate reference data block is used as a third candidate reference data block; according to the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded, it is determined that the third candidate reference data block and the The row distance between the data blocks to be encoded; determine the distance between the third candidate reference data block and the data block to be encoded according to the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded The column distance between; the third candidate reference data block whose column distance is smaller than the first distance threshold among the third
  • the second set of reference data blocks includes one or more third reference data blocks, and the third reference data blocks are located in the same encoding processing unit as the data block to be encoded in the multimedia data frame An encoded data block; or, the second reference data block set includes one or more fourth reference data blocks, and the fourth reference data block is a data block in the multimedia data frame that has the same Adjacent relationship, and the coded data block that belongs to the same coded slice as the data block to be coded.
  • the adjacent relationship between the fourth reference data block and the data block to be encoded means that: the second reference data block and the data block to be encoded belong to the same largest encoding block; or, The second reference data block is located in the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is less than or equal to a size threshold; N is based on the size information of the largest coding block Determined; or, the second reference data block is located in the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is greater than or equal to a size threshold; in the multimedia data frame
  • the data block located in the upper left corner of the area where the second reference data block is moved to the right by a maximum coding block is an unreconstructed data block; the position information when the second reference data block moves a distance of a maximum coding block, It is different from the position information of the data block to be encoded; N is determined according to the size information of the largest encoding block.
  • the distance between the first target reference data block and the data block to be encoded is smaller than a second distance threshold; the first target reference data block is located at an integer pixel position of the multimedia data frame .
  • the number of data blocks to be encoded in the multimedia data frame is multiple, and the first obtaining module is further configured to obtain attribute information according to each data block to be encoded in the multimedia data frame; when according to the The attribute information of each data block to be encoded in the multimedia data frame, when it is determined that the first target reference data blocks corresponding to all the data blocks to be encoded in the multimedia data frame belong to the second reference data block set, the multimedia data The data blocks to be encoded in the frame are filtered; when the attribute information of each data block to be encoded in the multimedia data frame is determined, it is determined that the first target reference data block corresponding to the data block to be encoded in the multimedia data frame is selected from the When the first set of reference data blocks is selected, the filtering process on the data blocks to be encoded in the multimedia data frame is suspended.
  • the steps involved in the multimedia data processing method shown in FIG. 7 may be executed by various modules in the multimedia data processing device shown in FIG. 11 .
  • step S101 shown in FIG. 7 can be performed by the first acquisition module 111 in FIG. 11
  • step S102 shown in FIG. 7 can be performed by the first matching module 112 in FIG. 11
  • Step S103 can be performed by the encoding module 113 in FIG. 11 .
  • Fig. 12 is a schematic structural diagram of another multimedia data processing device provided by the embodiment of the present application.
  • the multimedia data processing device may be a computer program (including program code) running in a computer device.
  • the multimedia data processing device is a Application software; the device can be used to execute the corresponding steps in the method provided by the embodiment of the present application.
  • the multimedia data processing apparatus may include: a second obtaining module 121 , a second matching module 122 and a decoding module 123 .
  • the second obtaining module is configured to obtain attribute information of the data block to be decoded in the multimedia data frame; the second matching module is configured to obtain a second target reference data block that matches the attribute information of the data block to be decoded; wherein, The second target reference data block belongs to a first reference data block set, a second reference data block set, or a third reference data block set fused with the first reference data block set and the second reference data block set,
  • the first set of reference data blocks includes decoded data blocks that have no adjacent relationship with the data block to be decoded, and the second set of reference data blocks includes A decoded data block in a neighboring relationship; a decoding module configured to decode the data block to be decoded according to the second target reference data block.
  • the attribute information of the data block to be decoded includes the media type of the multimedia data frame to which the data block to be decoded belongs; the second matching module is further configured to: if the media type of the multimedia data frame is an intra-frame type, obtaining a second target reference data block matching the attribute information of the data block to be decoded from the first set of reference data blocks; if the media type of the multimedia data frame is a non-intra-frame type, Then acquire a second target reference data block that matches the attribute information of the data block to be decoded from the first set of reference data blocks or the third set of reference data blocks.
  • the attribute information of the data block to be decoded includes a reference mark of the multimedia data frame to which the data block to be decoded belongs; the second matching module is further configured to: if the reference mark of the multimedia data frame is the first reference flag, then obtain a second target reference data block that matches the attribute information of the data block to be decoded from the first set of reference data blocks or the third set of reference data blocks; if the multimedia data If the reference mark of the frame is the second reference mark, the second target reference data block matching the attribute information of the data block to be decoded is obtained from the second set of reference data blocks.
  • the first set of reference data blocks includes one or more first reference data blocks; when the decoding mode of the multimedia data frame is a serial decoding mode, the first reference data blocks are all The parameter association relationship in the multimedia data frame satisfies the write-back restriction condition, and belongs to the decoded data block of the same decoded slice as the data block to be decoded; the parameter association relationship is used to reflect the first position information, and the maximum decoding The association relationship between the size information of the block and the second position information, the first position information and the second position information respectively refer to the position of the decoded data block and the data block to be decoded in the multimedia data frame Information; the write-back restriction condition is used to reflect a time delay limit for writing the decoded data block to the first set of reference data blocks.
  • the first set of reference data blocks includes one or more second reference data blocks; the second reference data blocks belong to the same decoding slice as the data block to be decoded, and the multimedia
  • the parameter association relationship in the data frame meets the write-back restriction condition, and the decoded data block whose parameter association relationship meets the decoding processing condition; the parameter association relationship is used to reflect the decoded data block and the data block to be decoded respectively The relationship between the position information in the multimedia data frame and the size information of the largest decoded block; the write-back restriction condition is used to reflect that the decoded data block is written into the first reference data Bound latency for block collection.
  • the second set of reference data blocks includes one or more third reference data blocks, and the third reference data blocks are located in the same decoding process as the data block to be decoded in the multimedia data frame The decoded data block of the unit; or, the second reference data block set includes one or more fourth reference data blocks, and the fourth reference data block is the data block to be decoded in the multimedia data frame A decoded data block that has an adjacent relationship and belongs to the same decoding slice as the data block to be decoded.
  • the number of data blocks to be decoded in the multimedia data frame is multiple, and the second obtaining module is further configured to obtain attribute information according to each data block to be decoded in the multimedia data frame; when according to the attribute information of each data block to be encoded in the multimedia data frame, when it is determined that the second target reference data blocks corresponding to all the data blocks to be decoded in the multimedia data frame belong to the second reference data block set, the multimedia data Perform filtering processing on the data blocks to be decoded in the frame; when it is determined according to the attribute information of each data block to be decoded in the multimedia data frame that the second target reference data block corresponding to the data block to be decoded in the multimedia data frame belongs to the When the first set of reference data blocks is used, the filtering process on the data blocks to be decoded in the multimedia data frame is suspended.
  • the steps involved in the multimedia data processing method shown in FIG. 10 may be executed by various modules in the multimedia data processing device shown in FIG. 12 .
  • step S201 shown in FIG. 10 can be performed by the second acquiring module 121 in FIG. 12
  • step S202 shown in FIG. 10 can be performed by the second matching module 122 in FIG. 12
  • Step S203 can be performed by the decoding module 123 in FIG. 12 .
  • each module in the multimedia data processing device shown in Fig. 11 and Fig. 12 can be respectively or all combined into one or several units to form, or one (some) of the units can also be further Splitting into at least two functionally smaller subunits can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned modules are divided based on logical functions. In practical applications, the functions of one module can also be realized by at least two units, or the functions of at least two modules can be realized by one unit. In other embodiments of the present application, the data processing device may also include other units, and in practical applications, these functions may also be implemented with the assistance of other units, and at least two units may be cooperatively implemented.
  • a general-purpose computer device such as a computer including processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements.
  • CPU central processing unit
  • RAM random access storage medium
  • ROM read-only storage medium
  • a computer program capable of executing the steps involved in the corresponding methods as shown in Figure 7 and Figure 10, to construct a multimedia data processing device as shown in Figure 11 and Figure 12, and to realize the present application
  • the multimedia data processing method of the embodiment can be recorded in, for example, a computer-readable storage medium, loaded into the above-mentioned computer device via the computer-readable storage medium, and run there.
  • FIG. 13 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the above-mentioned computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005.
  • the above-mentioned computer device 1000 may also include : a media content interface 1003, and at least one communication bus 1002.
  • the communication bus 1002 is used to realize connection and communication between these components.
  • the media content interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the optional media content interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface and a wireless interface (such as a WIFI interface).
  • the memory 1005 can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001 .
  • the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a media content interface module, and a device control application program.
  • the network interface 1004 can provide a network communication function; and the media content interface 1003 is mainly used to provide an input interface for media content.
  • the computer device 1000 described in the embodiment of the present application can execute the description of the above-mentioned multimedia data processing method in the embodiment corresponding to FIG. 7 and FIG. 10 above, and can also execute the embodiment corresponding to FIG. 11 and FIG. 12 above.
  • the description of the above-mentioned multimedia data processing device in will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • the embodiment of the present application also provides a computer-readable storage medium, and the above-mentioned computer-readable storage medium stores the computer program executed by the aforementioned multimedia data processing device, and the above-mentioned computer
  • the program includes program instructions.
  • the above-mentioned processor executes the above-mentioned program instructions, it can execute the description of the above-mentioned multimedia data processing method in the embodiment corresponding to FIG. 7 and FIG. 10 above, so details will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • the above-mentioned program instructions may be deployed on one computer device for execution, or deployed on at least two computer devices located at one location, or at least two computer devices distributed at at least two locations and interconnected by a communication network. At least two computer devices distributed in at least two locations and interconnected by a communication network can form a blockchain network.
  • the above-mentioned computer-readable storage medium may be the multimedia data processing apparatus provided in any of the foregoing embodiments or an internal storage unit of the above-mentioned computer equipment, such as a hard disk or memory of the computer equipment.
  • the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (SMC, Smart Media Card), a secure digital (SD, Secure Digital) card, Flash card (flash card), etc.
  • the computer-readable storage medium may also include both an internal storage unit of the computer device and an external storage device.
  • the computer-readable storage medium is used to store the computer program and other programs and data required by the computer device.
  • the computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
  • the embodiment of the present application also provides a computer program product, including a computer program or a computer instruction.
  • a computer program product including a computer program or a computer instruction.
  • the above-mentioned multimedia data processing method in the above-mentioned embodiments corresponding to FIG. 7 and FIG. 10 is implemented. description, and therefore will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • each flow and/or of the method flow charts and/or structural diagrams can be implemented by computer program instructions or blocks, and combinations of processes and/or blocks in flowcharts and/or block diagrams.
  • These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a A device for realizing the functions specified in one or more steps of the flowchart and/or one or more blocks of the structural diagram.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device implements the functions specified in one or more blocks of the flowchart and/or one or more blocks of the structural schematic diagram.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby
  • the instructions provide steps for implementing the functions specified in one or more steps of the flowchart and/or one or more blocks in the structural illustration.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例公开了一种多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品，其中，该方法包括：获取多媒体数据帧中待编码数据块的属性信息；获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合；根据所述第一目标参考数据块对所述待编码数据块进行预测编码。通过本申请，能够有效提高媒体数据的编码性能。

Description

多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品
相关申请的交叉引用
本申请基于申请号为202111221890.5、申请日为2021年10月20日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及互联网技术领域，涉及但不限于一种多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品。
背景技术
在多媒体数据处理过程中，通常采用帧内预测编码方式对多媒体数据帧进行预测编码，这样能够提升多媒体数据帧的编码性能，因此，帧内预测编码方式已经广泛应用到多媒体数据处理过程中。但实践中发现，目前的帧内预测编码方式中，由于用于进行预测编码的参考数据块有限，导致多媒体数据帧的编码性能较差。
发明内容
本申请实施例提供一种多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品，能够提高多媒体数据帧的编码性能。
本申请实施例提供一种多媒体数据处理方法，包括：获取多媒体数据帧中待编码数据块的属性信息；获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；其中，所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，所述第二参考数据块集合包括与所述待编码数据块之间具有相邻关系的已编码的数据块；根据所述第一目标参考数据块对所述待编码数据块进行预测编码。
本申请实施例提供一种多媒体数据处理方法，包括：获取多媒体数据帧中待解码数据块的属性信息；获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；其中，所述第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待解码数据块之间不具有相邻关系的已解码的数据块，所述第二参考数据块集合包括与所述待解码数据块之间具有相邻关系的已解码的数据块；根据所述第二目标参考数据块对所述待解码数据块进行解码。
本申请实施例提供一种多媒体数据处理装置，包括：第一获取模块，配置为获取多媒体数据帧中待编码数据块的属性信息；第一匹配模块，配置为获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；其中，所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所 述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，所述第二参考数据块集合包括与所述待编码数据块之间具有相邻关系的已编码的数据块；编码模块，配置为根据所述第一目标参考数据块对所述待编码数据块进行预测编码。
本申请实施例提供一种多媒体数据处理装置，包括：第二获取模块，配置为获取多媒体数据帧中待解码数据块的属性信息；第二匹配模块，配置为获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；其中，所述第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待解码数据块之间不具有相邻关系的已解码的数据块，所述第二参考数据块集合包括与所述待解码数据块之间具有相邻关系的已解码的数据块；解码模块，配置为根据所述第二目标参考数据块对所述待解码数据块进行解码。
本申请实施例提供一种计算机设备，包括：处理器及存储器；其中，存储器配置为存储程序指令，处理器配置为调用程序指令，以执行多媒体数据处理方法中的步骤。
本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，上述计算机程序包括程序指令，上述程序指令当被处理器执行时，以执行上述多媒体数据处理方法中的步骤。
本申请实施例提供了一种计算机程序产品，包括计算机程序或计算机指令，所述计算机程序或计算机指令被处理器执行时实现上述多媒体数据处理方法的步骤。
本申请实施例至少具有以下有益效果：通过获取与待编码数据块的属性信息相匹配的第一目标参考数据块，该第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合该第一参考数据块集合和该第二参考数据块集合的第三参考数据块集合；也就是说，可供选择的参考数据块集合包括第一参考数据块集合、第二参考数据块集合以及第三参考数据块集合，如此，扩展了参考数据块的选择范围，并能够根据待编码数据块的属性信息，自适应地选择第一目标参考数据块，以实现基于第一目标参考数据块对待编码数据块进行预测编码，因此，能够提高获取第一目标参考数据块的灵活性和准确度，提高对多媒体数据帧的编码性能。另外，根据第一目标参考数据块对待编码数据块进行预测编码，有利于去除多媒体数据帧中的冗余，提高多媒体数据帧的传输效率，并节省传输资源。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种视频处理的流程图；
图2是本申请实施例提供的针对多媒体数据的处理方法的流程示意图；
图3是本申请实施例提供的一种编码数据块的示意图；
图4是本申请实施例提供的一种多媒体数据帧的非帧内编码的场景示意图；
图5是本申请实施例提供的一种多媒体数据帧的帧内块复制的场景示意图；
图6是本申请实施例提供的多媒体数据帧的参考数据块的内存重用的场景示意图；
图7是本申请实施例提供的一种多媒体数据处理方法的流程示意图；
图8是本申请实施例提供的一种第一参考数据集合、第二参考数据块集合、待编码 数据块之间的位置关系示意图；
图9是本申请实施例提供的另一种第一参考数据集合、第二参考数据块集合、待编码数据块之间的位置关系示意图；
图10是本申请实施例提供的另一种多媒体数据处理方法的流程示意图；
图11是本申请实施例提供的一种多媒体数据处理装置的结构示意图；
图12是本申请实施例提供的另一种多媒体数据处理装置的结构示意图；
图13是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例涉及多媒体数据处理技术。其中，多媒体数据(或称之为媒体数据)是指由内容上相互关联的文本、图形、图像、声音、动画、活动图像等媒体数据所形成的复合数据。本申请实施例所提及的多媒体数据，主要包括由图像构成的图像数据、或者由图像和声音等构成的视频数据，且本申请实施例中，主要以多媒体数据为视频数据为例，对多媒体数据的数据处理过程进行详细说明，而在多媒体数据为图像数据时，可参见本申请实施例。本申请实施例涉及的对多媒体数据的处理过程，主要包括媒体数据采集、媒体数据编码、媒体数据文件封装、媒体数据文件传输、媒体数据解码和最终的数据呈现，而在该多媒体数据为视频数据时，针对视频数据的完整处理过程可如图1所示，可包括：S1，视频采集；S2，视频编码；S3，视频文件封装；S4，视频文件传输；S5，视频文件解封装；S6，视频解码；S7，视频呈现。
视频采集用于将模拟视频转换为数字视频，并按数字视频文件的格式进行保存，也就是说，视频采集可将视频信号转换为二进制数字信息，其中，由视频信号转换为的二进制信息是一种二进制数据流，该二进制信息也可称为该视频信号的码流或者位流(Bitstream)。视频编码是指通过压缩技术，将原始视频格式的文件转换为另一种视频格式文件。本申请实施例中提及的视频媒体内容的生成，包括摄像机采集生成的真实场景，以及计算机生成的屏幕内容场景，而从视频信号的获取方式看，视频信号可以分为摄像机拍摄到的以及计算机生成的两种信号，由于统计特性的不同，不同视频信号对应的压缩编码方式也可能有所区别，目前主流视频编码技术采用了混合编码框架，对输入的原始视频信号进行了如下一系列的操作和处理，可如图2所示：
块划分结构(block partition structure)：输入多媒体数据帧(如视频数据中的一个视频帧)根据一个视频帧的大小，将视频帧划分成若干个不重叠的处理单元，每个处理单元将进行类似的压缩操作。在一个实施例中，这个处理单元被称作编码树单元(CTU，Coding Tree Unit)，或者最大编码单元(LCU，Largest Coding Unit,)。其中，CTU可以进行更加精细的划分，得到一个或多个基本编码的单元，称之为编码单元(CU，Coding Unit)，每个CU是一个编码环节中最基本的元素。在另一个实施例中，处理单元也被称作编码片(Tile)，是一个可以独立解码和编码的多媒体数据帧的矩形区域。其中，编码片可以继续进行更加精细的划分，得到一个或多个最大编码块(SB，Superblock)，然后，最大编码块再往下划分，得到一个或多个数据块(B，block)。每个数据块是一个编码环节中最基本的元素。以下描述的是对每一个数据块可能采用的各种编码方式，其中，SB和B之间的关系如图3所示。
预测编码(Predictive Coding)：包括了帧内预测和帧间预测等方式，原始视频信号经过选定的已编码视频信号的预测后，得到残差视频信号。编码端需要为当前编码数据块(即待解码数据块)在众多可能的预测编码模式中选择最适合的一种，并告知解码端。其中，帧内预测(Intra picture Prediction)是指预测的信号来自于同一图像内已经编码重建过的区域。帧间预测(Inter picture Prediction)是指预测的信号来自已经编码过的，不同于当前图像的其他图像(称之为参考图像)。
变换编码及量化(Transform&Quantization)：残差视频信号经过离散傅里叶变换(DFT，Discrete Fourier Transform)、离散余弦变换等变换操作，将信号转换到变换域中。在变换域中的信号进行有损的量化操作，使得量化后的信号有利于压缩表达。
在一些视频编码标准中，可能有多于一种变换方式可以选择，因此，编码端也需要为当前编码数据块选择其中的一种变换，并告知解码端。量化的精细程度通常由量化参数(QP，Quantization Parameter)来决定，QP取值大于QP阈值时，表示大于预设取值范围的系数将被量化为同一个输出，因此通常会带来大于失真率阈值的失真及小于码率阈值的码率；相反，QP取值小于或等于QP阈值时，表示小于预设取值范围的系数将被量化为同一个输出，因此通常会带来小于失真率阈值的失真，同时带来大于码率阈值的码率。
熵编码(Entropy Coding)或统计编码：量化后的变换域信号将根据各个值出现的频率，进行统计压缩编码，最后输出二值化(0或者1)的压缩码流。同时，编码产生其他信息，例如选择的模式，运动矢量等，也需要进行熵编码以降低码率。
统计编码是一种无损编码方式，可以有效的降低表达同样的信号所需要的码率。常见的统计编码方式有变长编码(VLC，Variable Length Coding)或者基于上下文的二值化算术编码(CABAC，Content Adaptive Binary Arithmetic Coding)。
环路滤波(Loop Filtering)：已经编码过的图像(即多媒体数据帧)，经过反量化，反变换及预测补偿的操作，可获得重建的解码图像。重建图像与原始图像相比，由于存在量化的影响，部分信息与原始图像有所不同，产生失真(Distortion)。对重建图像进行滤波操作，例如去块效应滤波(deblocking)，样点自适应补偿(SAO，Sample Adaptive Offset)或者自适应环路滤波(ALF，Adaptive Loop Filter)等滤波器，可以有效的降低量化所产生的失真程度。由于这些经过滤波后的重建图像，将作为后续编码图像的参考，用于对将来的信号进行预测，所以上述的滤波操作也被称为环路滤波，及在编码环路内的滤波操作。
图2中展示了视频编码器的基本流程，图2中以第k个CU(标记为S k[x，y])为例进行举例说明，其中，k为大于等于1且小于等于输入的当前图像中的CU的数量的正整数，S k[x，y]表示第k个CU中坐标为[x，y]的像素点，x表示像素点的横坐标，y表示像素点的中坐标，S k[x，y]经过运动补偿或者帧内预测等中的一种较优处理后获得预测信号
Figure PCTCN2022118502-appb-000001
S k[x，y]与
Figure PCTCN2022118502-appb-000002
相减得到残差信号U k[x，y]，然后对该残差信号U k[x，y]进行变换和量化，量化输出的数据有两个不同的去处：一是送个熵编码器进行熵编码，编码后的码流输出到一个缓冲器(buffer)中保存，等待传出去；另一个应用是进行反量化和反变换后，得到信号U’ k[x，y]。将信号U’ k[x，y]与
Figure PCTCN2022118502-appb-000003
相加得到新的预测信号S* k[x，y]，并将S* k[x，y]送到当前图像的缓冲器中保存。S* k[x，y]经过帧内—图像预测获得f(S* k[x，y])，S* k[x，y]经过环路滤波后获得S’ k[x，y]，并将S’ k[x，y]送至解码图像缓冲器中保存，以用于生成重建视频。S’ k[x，y]经过运动—补偿预测后获得S’ r[x+m x，y+m y]，S’ r[x+m x，y+m y]表示参考块，m x和m y分别表示运动矢量的水平和竖直分量。
根据上述编码过程可以看出，在解码端，对于每一个数据块，解码器获得压缩码流 后，先进行熵解码，获得各种模式信息及量化后的变换系数。各个系数经过反量化及反变换，得到残差信号。另外，根据已知的编码模式信息，可获得该数据块对应的预测信号，两者相加之后，即可得到已编码的数据块。最后，已编码的数据块需要经过环路滤波的操作，产生最终的已编码的数据块。
目前主流的视频编码标准，如HEVC、VVC、AVS3、AV1和AV2均采用基于块的混合编码框架。这些视频编码标准将原始的多媒体数据帧分成一系列的编码数据块，结合预测、变换和熵编码等视频编码方法，实现视频数据的压缩。其中，运动补偿是视频编码常用的一类预测编码方法，运动补偿是基于视频内容在时域或空域的冗余特性，从已编码的区域(即参考数据块)导出当前编码数据块的预测值。这类预测编码方法包括：帧间预测、帧内块复制预测、帧内串复制预测等，在预测编码实现中，可能单独或组合使用这些预测编码方法。对于使用了这些预测编码方法的编码数据块，通常需要在码流显式或隐式的编码一个或多个二维的位移矢量，指示当前编码数据块(或当前块的同位块)相对编码数据块的一个或多个参考数据块的位移。
需要说明的是，在不同的预测编码模式下，位移矢量可能有不同的名称，本申请实施例统一按照以下方式进行描述：1)帧间预测中的位移矢量称为运动矢量(MV，Motion Vector)；2)帧内块复制中(即帧内预测编码)的位移矢量称为块矢量(BV，Block Vector)；3)帧内串复制中的位移矢量称为串矢量(SV，String Vector)。以下对帧间预测和帧内块复制预测中相关的技术进行介绍。
1.1、帧间预测。如图4所示，帧间预测利用多媒体数据帧的时间域的相关性，使用邻近已编码图像的像素预测当前图像的像素，以达到有效去除多媒体数据帧的时域冗余的目的，能够有效节省编码残差数据的比特。其中，P为当前帧，Pr为参考帧，B为当前待解码数据块，Br是B的参考数据块。B’与B在图像中的坐标位置相同，Br坐标为(x r，y r)，B’坐标为(x，y)。当前编码数据块与其参考数据块之间的位移，称为运动向量(MV)，即可采用如下公式(1)表示：
MV＝(x r-x，y r-y)      (1)
考虑到时域或空域邻近块具有较强的相关性，可以采用MV预测技术进一步减少编码MV所需要的比特。
2.1、帧内块复制预测。帧内块复制(IBC，Intra Block Copy)是HEVC屏幕内容编码(SCC，Screen Cotent Coding)扩展中采纳的一种帧内编码工具，帧内块复制显著的提升了屏幕内容的编码效率。在AVS3、VVC和AV1中，也采纳了IBC技术以提升屏幕内容编码的性能。IBC利用屏幕内容视频在空间的相关性，使用多媒体数据帧中已编码的数据块的像素预测待解码数据块的像素，能够有效节省编码像素所需的比特。图5为IBC技术的示意图，其中，待解码数据块与其参考数据块之间的位移，称为块矢量(BV，Block Vector)。
在HEVC SCC扩展中，IBC模式采用了类似于帧间预测的实现方式。如果多媒体数据帧允许使用IBC模式，则将当前已编码但未滤波的图像作为一个特殊的参考帧，编码数据块采用与帧间预测运动补偿的方式，从这个参考帧上导出待解码数据块的预测值。这种实现方式具有以下特点：1)BV只采用整数精度；2)采用了全局参考数据块集合(即全局参考范围)，即当前多媒体数据帧所有已编码的部分中除了某些区域基于并行处理和色度插值的考虑被禁止用作参考数据块以外，其他均可用作待解码数据块的参考数据块；3)在解码图像缓存中，当前已编码未滤波的图像会占用一个图像大小的缓存；4)当前已编码未滤波的图像将位于参考图像列表的最后。
在硬件实现中，计算核心可以从片上存储器或片外存储器中读写数据。片上存储器通常是静态内存，读写速度非常快，但由于片上存储器的价格相对较高，通常仅使用较 小的片上存储器。而片外存储器通常要大得多，但如果要高效的访问，需要遵循一些规则，在将最近已编码的数据块写回片外的图片存储器时通常存在回写时延(writing-back delay)。在运动补偿的硬件实现中，当前已编码的图像将在完成环路滤波操作后写入到片外存储器中，以供将来参考。当后续编码数据块的运动补偿需要参考位于片外存储器的已编码的数据块时，需要从片外存储器读取该参考数据块的相关数据。这个过程涉及写入和读取的内存带宽消耗。虽然在软件实现上HEVC-SCC中IBC方案对于相关技术中的软件设计具有较小的影响，然而在硬件实现中具有以下两个问题：a、IBC的参考图像是未经过环路滤波处理的版本，与最终输出的解码图像不同。将该图像输出到片外将导致内存带宽的增加。b、当前已编码或一个刚刚重建完成的相邻数据块可能用作下一个编码数据块的参考。这要求在有限的时钟周期完成重建数据块的解码，输出到片外存储器，再读入片内存储器等流程。这对硬件实现提出了较大的挑战。
针对IBC硬件实现的问题，后续的标准如VVC、AVS3和AV1中采纳了一些IBC相关的优化方案。VVC、AVS3标准中的IBC方案使用了局部参考数据块集合，即使用与当前编码数据块相邻的已编码的数据块作为当前编码数据块的参考数据块。在硬件实现中，可以使用片上存储器中存储的局部参考数据块集合中的参考数据块以解决上述问题。基于硬件实现成本和编码性能的考虑，IBC仅使用了1个CTU大小的内存存储的局部参考数据块集合中的参考数据块，并采用了如图6所示的参考内存重用策略。除了存储当前待重构的64x64的CU外，还有3个64x64大小的CU可用于存储已重构的数据块。因此，IBC仅能在这三个64x64大小的CU以及当前64x64CU的已重构部分中搜索参考数据块。
综上，AV1标准中的IBC方案使用了全局参考数据块集合，即当前编码数据块允许参考当前多媒体数据帧中已编码的数据块，但是，该方案存在如下两个缺陷：a、为了满足回写限制条件的要求，与当前编码数据块相邻近的已编码的数据块不允许用作参考数据块；b、如果当前多媒体数据帧允许使用IBC，则环路滤波器将被禁用。然而，屏幕内容序列往往具有较强的空间相关性，相邻数据块具有较高的参考价值，允许相邻已编码的数据块被参考将有利于提升编码性能。在一些序列内容上，禁用环路滤波器也将带来严重的多媒体数据质量下降并影响编码性能。虽然，基于局部参考数据块集合的IBC方案不存在上述缺陷，但由于局部参考数据块集合中可用的参考数据块的数量有限，仅使用局部参考数据块集合可能会使多媒体数据帧的编码性能下降。根据序列内容的特性，全局参考数据块集合、局部参考数据块集合以及环路滤波处理等编码参数对编码性能影响具有不同的占比。因此，基于全局参考数据块集合的IBC方案和局部参考数据块集合的IBC方案中存在的缺陷，本申请实施例中，根据多媒体数据帧的属性信息选择合适预测编码方式，将有利于编码性能的提升；也就是说，本申请中根据当前编码数据块的属性信息自适应地，从第一参考数据块集合(即全局参考数据块集合)、第二参考数据块集合(即局部参考数据块集合)，或融合第二参考数据块集合和第一数据块集合的第三数据块集合中选择的参考数据块，扩展了参考数据块可选择的范围，可解决参考数据块的数量有限的问题；根据选择的参考数据块对当前编码数据块进行预测编码，可提高对多媒体数据帧的编码性能。
需要说明的是，本申请可以由终端来执行，也可以由服务器来执行，还可以由终端和服务器共同执行，本申请中用于执行该方法的设备可以统称为计算机设备。其中，服务器可以是独立的一个物理服务器，也可以是至少两个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(CDN，Content Delivery Network)、以及大数据和人工智能平台等基础云计算服务的云服务器。终端可以是智能 车载终端、智能摄像机、智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、有屏音箱、智能手表、智能电视等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，同时，终端以及服务器的数量可以为一个或至少两个，本申请在此不做限制。
需要说明的是，本申请实施例可应用于各种场景，包括但不限于云技术、人工智能、智慧交通、辅助驾驶等。例如，在智慧交通场景中，多媒体数据帧为摄像装置对道路进行拍摄得到的视频帧或图像帧，该摄像装置可以是指车载终端，或者道路侧的摄像装置；通过本申请实施例对多媒体数据帧进行预测编码，有利于提高对多媒体数据帧的编码性能，通过编码后的多媒体数据帧能够实时观测道路的路况状态等，提高车辆的行驶安全。
图7是本申请实施例提供的一种多媒体数据处理方法的流程示意图，如图7所示，该方法至少可以包括以下S101-S103：
S101、获取多媒体数据帧中待编码数据块的属性信息。
本申请实施例中，计算机设备可以获取多媒体数据帧；该多媒体数据帧可以是由计算机设备所拍摄得到的，或者，该多媒体数据帧可以是由计算机设备从网络中下载得到的。该多媒体数据帧是指视频数据中待解码的视频帧或图像数据中待解码的一帧图像，视频数据可以是指直播视频数据、影视视频数据、日常生活视频数据、道路监控视频数据等等；图像数据可以是指人物图像数据、建筑图像数据、物品图像数据等等。可以获取该多媒体数据帧中的待编码数据块的属性信息，该待编码数据块可以是指上文中的当前编码数据块，待编码数据块可以是指多媒体数据帧中的未编码的数据块中即将编码的数据块，即待编码数据块可以是指多媒体数据帧中与已编码的数据块的距离最近的未编码数据块。
待编码数据块的属性信息包括待编码数据块所属的多媒体数据帧的媒体类型、多媒体数据帧的参考标志、待编码数据块所属的多媒体数据帧的清晰度等中的一种或多种；媒体类型包括帧内和非帧内，帧内类型可以是指具有代表意义的多媒体数据帧，帧内类型包括全帧内编码帧、Intra帧、I帧和关键帧。帧内类型的帧只允许使用帧内编码技术，不需要依赖其他帧进行编码。例如，帧内类型可以是指能够体现视频数据或图像数据中主要含义的多媒体数据帧。非帧内类型可以是指视频数据或图像数据中除帧内类型以外的多媒体数据帧，非帧内类型可以称为帧间编码帧、P帧、B帧或非关键帧。非帧内类型的帧允许使用帧间编码技术和帧内编码技术，通常当采用帧间编码技术时，需要在参考帧编码完成后才可以进行编码。多媒体数据帧的参考标志用于反映待编码数据块的参考数据块的选择范围。
S102、获取与该待编码数据块的属性信息相匹配的第一目标参考数据块；其中，第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合第一参考数据块集合和第二参考数据块集合的第三参考数据块集合，第一参考数据块集合包括与待编码数据块之间不具有相邻关系的已编码的数据块，第二参考数据块集合包括与待编码数据块之间具有相邻关系的已编码的数据块。第三数据块集合包括第一数据块集合和第二数据块集合中的数据块，即第三数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，和与待编码数据块之间具有相邻关系的已编码的数据块。
本申请实施例中，多媒体数据帧与待编码数据块不具有相邻关系的已编码的数据块、与待编码数据块具有相邻关系的已编码的数据块均能够用于作为参考数据块，计算机设备通过根据待编码数据块的属性信息，自适应地从具有相邻关系的已编码的数据块和不具有相邻关系的已编码的数据块中选择第一目标参考数据块，可提高获取参考数据块的灵活性以及准确度。
需要说明的是，本申请实施例中的已编码的数据块，也可以称为编码端已重建的数 据块，已重建的数据块为指已编码的数据块经过恢复处理(例如，反量化、反变换及预测补偿的操作等)后的数据块。
需要说明的是，该第一目标参考数据块可以是指与待编码数据块具有相邻关系的已编码的数据块，或，该第一目标参考数据块可以是指与待编码数据块不具有相邻关系的已编码的数据块，或者，该第一目标参考数据块的第一区域属于与待编码数据块具有相邻关系的已编码的数据块，且该第一目标参考数据块的第二区域可以属于与待编码数据块不具有相邻关系的已编码的数据块。
S103、根据第一目标参考数据块对待编码数据块进行预测编码。
本申请实施例中，计算机设备可以根据第一目标参考数据块、待编码数据块分别在多媒体数据帧的位置信息，确定第一目标参考数据块与待编码数据块之间的块矢量，然后，根据该块矢量以及第一目标参考数据块预测得到待编码数据块的预测信号，根据待编码数据块的实际信号和预测信号得到该待编码数据块的残差信号，即待编码数据块的残差信号为待编码数据块的预测编码值；通过对待编码数据块进行预测编码，有利于去除多媒体数据帧中的冗余，提高多媒体数据帧的传输效率，并节省传输资源。
需要说明的是，通常视频数据中包括多个多媒体数据帧，或者，图像数据中包括多个多媒体数据帧，但上述待编码数据块、第一目标参考数据块均属于同一多媒体数据帧。
本申请实施例中，计算机设备通过获取与待编码数据块的属性信息相匹配的第一目标参考数据块，该第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合该第一参考数据块集合和该第二参考数据块集合的第三参考数据块集合；也就是说，可供选择的参考数据块集合包括第一参考数据块集合、第二参考数据块集合以及第三参考数据块集合，扩展了参考数据块的选择范围；同时，根据待编码数据块的属性信息，自适应地选择第一目标参考数据块，根据第一目标参考数据块对待编码数据块进行预测编码，能够提高获取第一目标参考数据块的灵活性和准确度，提高对多媒体数据帧的编码性能。同时，通过根据第一目标参考数据块对待编码数据块进行预测编码，有利于去除多媒体数据帧中的冗余，提高多媒体数据帧的传输效率，并节省传输资源。
在一些实施例中，计算机设备可以通过如下三种匹配方式中的一种或多种组合方式，获取与待编码数据块的属性信息匹配的第一目标参考数据块：
匹配方式一：若该多媒体数据帧的媒体类型为帧内类型，则从第一参考数据块集合或第三参考数据块集合中获取与该待编码数据块的属性信息相匹配的第一目标参考数据块；若该多媒体数据帧的媒体类型为非帧内类型，则从第二参考数据块集合中获取与该待编码数据块的属性信息相匹配的第一目标参考数据块。也就是说，计算机设备可以根据待编码数据块的属性信息，自适应地选择参考数据块集合，进而，从选择的参考数据集合中确定第一目标参考数据块，提高获取参考数据块的灵活性和准确度。
通常第一参考数据块集合中的数据块数量多于第二参考数据块集合中的数据块数量；第一参考数据块集合和第二参考数据块集合分别存储在第一存储空间和第二存储空间，第一存储空间和第二存储空间可以位于同一个计算机设备中，也可以位于不同计算机设备中。第一存储空间和第二存储空间可以是指磁盘或者存储介质阵列等。当该第一存储空间和第二存储空间位于同一个计算机设备中时，第一存储空间可以是指计算机设备的片外存储器，第二存储空间可以是指计算机设备的片上存储器；或者，第一存储空间可以是指计算机设备的片上存储器，第二存储空间可以是指计算机设备的片外存储器。
在匹配方式一中，如果该多媒体数据帧的媒体类型为帧内类型，表明该多媒体数据帧的重要性比较高，可以从第一参考数据块集合和第二参考数据块集合(即第三参考数据块集合)中搜索与待编码数据块之间具有最强相关性的数据块(即搜索最优数据块)，作为与该待编码数据块的属性信息匹配的第一目标参考数据块，该第一目标参考数据块 是该第一参考数据块集合和该第二参考数据块集合确定的。即第一参考数据块集合和第二参考数据块集合中的任一数据块均允许作为参考数据块，扩展了参考数据块的选择范围，有利于提高对多媒体数据帧的编码性能。如果该多媒体数据帧的媒体类型为非帧内类型，表明该多媒体数据帧的重要性比较低，则从该第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块。通过根据多媒体数据帧的媒体类型，自适应地选择与待编码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
此处第一目标参考数据块是根据该第一参考数据块集合和该第二参考数据块集合确定得到的数据块，且具有如下含义中的一种：第一、第一目标参考数据块选自第一参考数据块集合；第二、第一目标参考数据块选自第二参考数据块集合；第三、该第一目标参考数据块的第一区域属于第一参考数据块集合，且该第一目标参考数据块的第二区域可以属于第二参考数据块集合，即第一目标参考数据块部分属于第二参考数据块集合，且部分属于第一数据块集合。其中，待编码数据块与已编码的数据块之间的相关性可以是根据待编码数据块中的像素与已编码的数据块中的像素之间的相似确定的。
在匹配方式一中，如果该多媒体数据帧的媒体类型为帧内类型，可以从第一参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息匹配的第一目标参考数据块。如果该多媒体数据帧的媒体类型为非帧内类型，则从该第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块。通过根据多媒体数据帧的媒体类型，自适应地选择与待编码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
需要说明的是，计算机从第一参考数据块集合和第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息匹配的第一目标参考数据块，可以是从第一参考数据块集合和第二参考数据块集合中的数据块所构成的区域中，搜索与待编码数据块中的像素之间具有最强相关性的像素，将与待编码数据块中的像素之间具有最强相关性的像素所在的区域作为与该待编码数据块的属性信息匹配的第一目标参考数据块。
匹配方式二：当该待编码数据块的属性信息包括所述待编码数据块所属的多媒体数据帧的参考标志时；计算机设备获取与该待编码数据块的属性信息相匹配的第一目标参考数据块，包括：若该多媒体数据帧的参考标志为第一参考标志，则从第一参考数据块集合或第三参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块。该参考标志可以是指由多媒体数据帧的发送者(或接收者)预先设置的，因此，根据参考标志自适应的选择第一目标参考数据块，相当于根据用户的编码需求自适应的选择第一目标参考数据块，有利于提高选择的参考数据块的灵活性和准确性。
第一参考标志为用于指示待编码数据块的参考数据块的选择范围为第一参考数据块集合或第三参考数据块集合，第二参考标志为用于指示待编码数据块的参考数据块的选择范围为第二参考数据块集合。第一参考标志、第二参考标志可以是由多媒体数据帧的发送者(或接收者)预先设置的，第一参考标志、第二参考标志可以由数字、字母、汉字中的一种或多种构成。例如，第一参考标志(ibc_ref_flag)为1，第二参考标志为0，当多媒体数据帧中多媒体数据帧的参考标志为1时，计算机设备可以从第一参考数据块集合中获取与该待编码数据块的属性信息相匹配的第一目标参考数据块；当多媒体 数据帧中多媒体数据帧的参考标志为0时，计算机设备可以从第二参考数据块集合中获取与该待编码数据块的属性信息相匹配的第一目标参考数据块。
需要说明的是，当多媒体数据帧的媒体类型为帧内类型时，该多媒体数据帧的头部中包括参考标志；当多媒体数据帧的媒体类型为非帧内类型时，该多媒体数据帧的头部中可以不包括参考标志；或者，当多媒体数据帧允许使用IBC编码方式时，该多媒体数据帧的头部中包括参考标志；当多媒体数据帧不允许采用IBC编码方式时，该多媒体数据帧的头部中可以不包括参考标志；这时计算机设备可以根据多媒体数据帧的头部中是否包括参考标志，来确定与待编码数据块的属性信息匹配的第一目标参考数据块。
在匹配方式二中，若该多媒体数据帧的参考标志为第一参考标志，则从第一参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块，这时第一目标参考数据块是指第一参考数据块集合中的任一第一参考数据块；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块，这时第一目标参考数据块可以是指第二参考数据块集合中任一第二参考数据块。若该多媒体数据帧的参考标志为第一参考标志，则从第一参考数据块集合中和第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块，这时第一目标参考数据块是根据第一参考数据块集合和第二参考数据块集合确定得到的；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中搜索与待编码数据块之间具有最强相关性的数据块，作为与该待编码数据块的属性信息相匹配的第一目标参考数据块，这时第一目标参考数据块可以是指第二参考数据块集合中任一第二参考数据块。通过根据多媒体数据帧的参考标志，自适应地选择与待编码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
匹配方式三，当该待编码数据块的属性信息包括待编码数据块所属的多媒体数据帧的清晰度时，当待编码数据块所属的多媒体数据帧的清晰度大于清晰度阈值，表明用户对多媒体数据帧的质量要求比较高，可以从第一参考数据块集合或第三参考数据块集合中获取与该待编码数据块的属性信息匹配的第一目标参考数据块。当待编码数据块所属的多媒体数据帧的清晰度小于或等于清晰度阈值，表明用户对多媒体数据帧的质量要求比较低，可以从第二参考数据块集合中获取与该待编码数据块的属性信息匹配的第一目标参考数据块。多媒体数据帧的清晰度可以用于反映用户对多媒体数据帧的质量要求，因此，根据多媒体数据帧的清晰度自适应的选择第一目标参考数据块，相当于根据用户对多媒体数据帧的质量需求自适应的选择第一目标参考数据块，有利于提高选择的参考数据块的灵活性和准确性。
需要说明的是，当该多媒体数据帧的媒体类型为非帧内类型，多媒体数据帧的参考标志为第二参考标志，或者，多媒体数据帧的清晰度小于或等于清晰度阈值时，计算机设备可以从第二参考数据块集合中获取与该待编码数据块的属性信息匹配的第一目标参考数据块，这时，计算机设备可以仅存储第二参考数据块集合，不需要存储第一参考数据块集合，可节省计算机设备的存储空间，并降低参考数据块的搜索范围，提高搜索参考数据块的效率。
在一些实施例中，计算机设备可以根据多媒体数据帧的编码方式从如下两种方式中选择一种方式，获取第一参考数据块集合：
方式一：该第一参考数据块集合包括一个或多个第一参考数据块；当该多媒体数据帧的编码方式为串行编码方式时，该第一参考数据块为所述多媒体数据帧中参数关联关系满足回写限制条件，且与所述待编码数据块属于同一个编码片的已编码的数据块；所 述参数关联关系用于反映第一位置信息，与最大编码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息、第二位置信息分别是指已编码的数据块、所述待编码数据块在所述多媒体数据帧中的位置信息；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延。也就是说，串行编码方式是指同一时间仅对多媒体数据帧中的一个数据块进行预测编码，如果多媒体数据帧的编码方式为串行编码方式时，该编码方式不会对参考数据块的确定过程造成影响。但是，已编码的数据块写入至第一参考数据块集合中会带来较大回写延时，即已编码的数据块写入至第一参考数据块集合中需要消耗较长时间，因此，已编码的数据块至少等待预设时长后，才能够被用于作为参考数据块，预设时长是根据将所述已编码的数据块写入至所述第一参考数据块集合的限制时延确定的，该限制时延可以是指将已编码的数据块写入至第一参考数据块集合中的平均时延、最大时延以及最小时延等中的一种。由此可知，当该多媒体数据帧的编码方式为串行编码方式时，该第一参考数据块集合中的第一参考数据块需要满足以下两个条件：a、与待编码数据块属于同一个编码片；b、参数关联关系满足回写限制条件。
在实现的过程中，计算机设备可以执行如下步骤，以获取第一参考数据块集合：当该多媒体数据帧的编码方式为串行编码方式时，分别获取第一位置信息，以及该最大编码块的尺寸信息和第二位置信息；该第一位置信息、第二位置信息分别是指已编码的数据块、该待编码数据块在该多媒体数据帧中的位置信息。获取回写限制条件；该回写限制条件用于反映将该已编码的数据块写入至该第一参考数据块集合的限制时延。从该多媒体数据帧中筛选出参数关联关系满足回写限制条件，且与该待编码数据块属于同一个编码片的已编码的数据块，作为第一参考数据块；该参数关联关系用于反映该第一位置信息，与该最大编码块的尺寸信息和该第二位置信息之间的关联关系。将该第一参考数据块添加至该第一参考数据块集合。
本申请实施例中，当该多媒体数据帧的编码方式为串行编码方式，计算机设备可以分别获取第一位置信息，以及最大编码块的尺寸信息以及第二位置信息。该第一位置信息、第二位置信息分别是指已编码的数据块、待编码数据块在多媒体数据帧的位置信息，其中，第一位置信息、第二位置信息可以是指已编码的数据块的左上角像素、待编码数据块的左上角像素在多媒体数据帧的坐标，该最大编码块的尺寸信息可以是指最大编码块的宽度和高度。并且，计算机设备可以获取回写限制条件；从该多媒体数据帧中筛选出参数关联关系满足回写限制条件，且与该待编码数据块属于同一个编码片的已编码的数据块，作为第一参考数据块，将该第一参考数据块添加至该第一参考数据块集合。通过根据回写限制条件、第一位置信息、第二位置信息和最大编码块的尺寸信息筛选得到第一参考数据块，可确保第一参考数据块与待编码数据块具有较强的空间相关性，并可确保第一参考数据块的可用性。
在一些实施例中，筛选第一参考数据块可以通过以下方式实现：首先，根据第一位置信息和第二位置信息，从多媒体数据帧中筛选出属于同一编码片的已编码的数据块，作为第一候选参考数据块。然后，确定在回写限制条件下多媒体数据帧中已编码的数据块，与待编码数据块之间的间隔数据块限制数量。再然后，根据第一位置信息、最大编码块的尺寸信息和第二位置信息，统计第一候选参考数据块与待编码数据块之间的间隔数据块数量。最后，将间隔数据块数量大于间隔数据块限制数量的第一候选参考数据块，确定为第一参考数据块。
这里，假设第一位置信息、第二位置信息可以是指已编码的数据块的像素、待编码数据块的像素在多媒体数据帧的坐标，已编码的数据块的位置信待息为(ref_x，ref_y)，待编码数据块的位置信息为(cur_x，cur_y)，计算机设备可以从多媒体数据帧中筛选出 与待编码数据块属于同一编码片的已编码的数据块，作为第一候选参考数据块。并确定在该回写限制条件下该多媒体数据帧中已编码的数据块，与该待编码数据块之间的间隔数据块限制数量，该间隔数据块限制数量可以是指在满足回写限制条件下多媒体数据帧中已编码的数据块，与待编码数据块之间应间隔的最少数据块数量。然后，根据所述第一位置信息、该最大编码块的尺寸信息和该第二位置信息，统计该第一候选参考数据块与该待编码数据块之间的间隔数据块数量，该间隔数据块数量是指该第一候选参考数据块与该待编码数据块之间实际间隔的数据块数量，将间隔数据块数量大于该间隔数据块限制数量的第一候选参考数据块，确定为第一参考数据块。通过已编码的数据块的位置信息、待编码数据块的位置信息以及回写延时限制条件等，获取第一参考数据块集合，可确保第一参考数据块与待编码数据块具有较强的空间相关性，并可确保第一参考数据块的可用性。同时，不需要额外的硬件成本，可节省成本。
例如，计算机设备可以将满足如下公式(2)的第一候选参考数据块，确定为第一参考数据块：
D<cur_num-src_num      (2)
其中，公式(2)中cur_num-src_num表示该第一候选参考数据块与该待编码数据块之间的间隔数据块数量，cur_num、src_num分别是指第一候选参考数据块的编号以及待编码数据块的编号，D是指满足回写限制条件下的该间隔数据块限制数量，cur_num、src_num可以通过下文公式(3)和公式(4)计算得到。
在一些实施例中，上述统计第一候选参考数据块与待编码数据块之间的间隔数据块数量，可通过以下方式实现：首先，根据第一位置信息以及最大编码块的尺寸信息确定第一候选参考数据块的行标识和列标识。然后，根据第二位置信息以及最大编码块的尺寸信息确定待编码数据块的行标识和列标识。再然后，基于待编码数据块的尺寸信息确定多媒体数据帧中每行的数据块数量。最后，根据每行的数据块数量、第一候选参考数据块的行标识和列标识，以及待编码数据块的行标识和列标识，统计第一候选参考数据块与待编码数据块之间的间隔数据块数量。
这里，第一候选参考数据块的行标识和列标识分别为ref_y_num、ref_x_num，待编码数据块的行标识和列标识分别为cur_y_num、cur_x_num，计算机设备可以分别采用如下公式(3)获取第一候选参考数据块的行标识和列标识、待编码数据块的行标识和列标识：
Figure PCTCN2022118502-appb-000004
其中，公式(3)中Floor()表示向下取整，W、H可以是采用如下4种方式中的任一种方式确定的：方式一，W、H是根据最大编码块的尺寸信息确定的；方式二，W、H是根据片上内存的基本单元(VPDU，Virtual Pipeline Data Units)的尺寸信息确定的；方式三，H可以是指最大编码块的尺寸信息确定的。如最大编码块为128x128，H为128；最大编码块为64x64，H为64，W可以为默认值(如64)。方式四，W、H均为预设值，可以为4、64、128等。其中，一个最大编码块可以包括一个或多个数据块，最大编码块的尺寸以及数据块的尺寸可以是根据多媒体数据帧的分块方式确定的。
计算机设备可以基于该待编码数据块的尺寸信息确定该多媒体数据帧中每行的数据块数量；根据所述每行的数据块数量、所述第一候选参考数据块的行标识和列标识，确定第一候选参考数据块的编号，并根据所述每行的数据块数量以及所述待编码数据块的行标识和列标识确定待编码数据块的编号，将第一候选参考数据块的编号与待编码数据块的编号之间的差值，确定为第一候选参考数据块与所述待编码数据块之间的间隔数 据块数量。例如，计算机设备可以采用如下公式(4)确定该多媒体数据帧中每行数据块数量、第一候选参考数据块的编号以及待编码数据块的编号：
Figure PCTCN2022118502-appb-000005
其中，公式(4)中stride是指该多媒体数据帧中每行数据块数量，cur_num、src_num分别是指第一候选参考数据块的编号以及待编码数据块的编号，Ceil()表示向上取整，tile_w表示编码片的宽度，编码片包括一个或多个最大编码块，因此，编码片的宽度可以是根据最大编码块的尺寸信息确定。例如，最大编码块的尺寸为128x128时，编码片可以为1024x720。
需要说明的是，当该多媒体数据帧的编码方式为串行编码方式时，计算机设备还可以通过如下方式获取第一参考数据块集合：根据第一位置信息、第二位置信息以及最大编码块的尺寸信息，确定已编码的数据块与待编码数据块之间的间隔距离，并确定在回写延时条件下已编码的数据块与待编码数据块之间的间隔限制距离，该间隔限制距离是指满足回写延时条件下已编码的数据块与待编码数据块之间的最小间隔距离，将多媒体数据帧中与待编码数据块属于同一编码片，且间隔距离大于该间隔限制距离的已编码的数据块，作为第一参考数据块，将第一参考数据块添加至第一参考数据块集合。
在一些实施例中，该第一参考数据块集合包括一个或多个第二参考数据块；当该多媒体数据帧的编码方式为并行编码方式或者为串行编码方式时，第二参考数据块为与该待编码数据块属于同一个编码片，且该多媒体数据帧中参数关联关系满足回写限制条件，以及该参数关联关系满足编码处理条件的已编码的数据块；该参数关联关系用于反映已编码的数据块、该待编码数据块分别在该多媒体数据帧中的位置信息，以及最大编码块的尺寸信息之间的关联关系；该回写限制条件用于反映将该已编码的数据块写入至该第一参考数据块集合的限制时延。也就是说，并行编码方式是指同一时间对多媒体数据帧中的多个数据块进行预测编码，如果多媒体数据帧的编码方式为并行编码方式时，该编码方式会对参考数据块的确定过程造成影响。同时，已编码的数据块写入至第一参考数据块集合中会带来较大回写延时，由此可知，当该多媒体数据帧的编码方式为并行编码方式时，该第一参考数据块集合中的第二参考数据块需要满足以下三个条件：a、与待编码数据块属于同一个编码片；b、参数关联关系满足回写限制条件；c、参数关联关系满足编码处理条件。在一些实施例中，当该多媒体数据帧的编码方式为串行编码方式或者并行编码方式时，均可以采用上述获取第一参考数据块集合的方式实现。此处编码处理条件可以是指并行编码处理条件。
本申请实施例中，计算机设备可以执行如下步骤以获取第一参考数据块集合：获取第一位置信息、最大编码块的尺寸信息和第二位置信息；第一位置信息、第二位置信息分别是指已编码的数据块、待编码数据块在多媒体数据帧中的位置信息。然后，获取回写限制条件以及编码处理条件；回写限制条件用于反映将已编码的数据块写入至第一参考数据块集合的限制时延。再然后，从多媒体数据帧中筛选出参数关联关系满足回写限制条件和编码处理条件，且与待编码数据块属于同一个编码片的已编码的数据块，作为第二参考数据块；参数关联关系用于反映第一位置信息，与待编码数据块的尺寸信息和第二位置信息之间的关联关系。最后，将第二参考数据块添加至第一参考数据块集合。
这里，第一位置信息、第二位置信息可以是指已编码的数据块的左上角像素、待编码数据块的左上角像素在多媒体数据帧的坐标，该最大编码块的尺寸信息可以是指最大编码块的宽度和高度。该回写限制条件用于反映将该已编码的数据块写入至该第一参考数据块集合的限制时延，编码处理条件用于反映并行对多媒体数据帧进行编码时参考数据块与待编码数据块之间的限制距离；或者，编码处理条件用于反映并行对多媒体数据 帧进行编码时参考数据块与待编码数据块之间的间隔数据块限制数量。通过从该多媒体数据帧中筛选出参数关联关系满足回写限制条件和该编码处理条件，且与该待编码数据块属于同一个编码片的已编码的数据块，作为第二参考数据块，将该第二参考数据块添加至该第一参考数据块集合，可确保第二参考数据块与待编码数据块具有较强的空间相关性，并可确保第二参考数据块的可用性。
在一些实施例中，筛选第二参考数据块可以通过以下方式实现：根据第一位置信息和第二位置信息，从多媒体数据帧中筛选出与待编码数据块属于同一编码片的已编码的数据块，作为第一候选参考数据块。然后，从第一候选参考数据块中筛选出参数关联关系满足回写限制条件的第一候选参考数据块，作为第二候选参考数据块。再然后，确定在回写限制条件下多媒体数据帧中已编码的数据块，与待编码数据块之间的第一数据块限制距离。确定在编码处理条件下多媒体数据帧中已编码的数据块，与待编码数据块之间的第二数据块限制距离。最后，根据第一位置信息、最大编码块的尺寸信息、第二位置信息、第一数据块限制距离和第二数据块限制距离，从第二候选参考数据块中确定出满足编码处理条件的第二候选参考数据块，作为第二参考数据块。
这里，假设第一位置信息、第二位置信息可以是指已编码的数据块的像素、待编码数据块的像素在多媒体数据帧的坐标，已编码的数据块的位置信待息为(ref_x，ref_y)，待编码数据块的位置信息为(cur_x，cur_y)，计算机设备可以从多媒体数据帧中筛选出与待编码数据块属于同一编码片的已编码的数据块，作为第一候选参考数据块。本申请实施例中，计算机设备可以从该第一候选参考数据块中筛选出坐标满足上述公式(2)的第一候选参考数据块，作为第二候选参考数据块。可以确定在回写限制条件下多媒体数据帧中已编码的数据块，与待编码数据块之间的第一数据块限制距离，第一数据块限制距离可以是指满足回写延时限制条件下该多媒体数据帧中已编码的数据块，与待编码数据块之间应保持的最小距离；确定在编码处理条件下多媒体数据帧中已编码的数据块，与待编码数据块之间的第二数据块限制距离，该第二数据块限制距离是指满足编码处理条件下该多媒体数据帧中已编码的数据块，与待编码数据块之间应保持的最小距离。然后，可以根据该第一位置信息、该最大编码块的尺寸信息、该第二位置信息、该第一数据块限制距离和第二数据块限制距离，从该第二候选参考数据块中确定出满足该编码处理条件的第二候选参考数据块，作为第二参考数据块。
在一些实施例中，上述从第二候选参考数据块中确定出满足编码处理条件的第二候选参考数据块作为第二参考数据块，还可以通过以下方式实现：根据第一位置信息以及最大编码块的尺寸信息确定第二候选参考数据块的行标识和列标识。根据第二位置信息以及最大编码块的尺寸信息确定待编码数据块的行标识和列标识。然后，从第二候选参考数据块中筛选出行标识小于或等于待编码数据块的行标识的第二候选参考数据块，作为第三候选参考数据块。再然后，根据第三候选参考数据块的行标识和待编码数据块的行标识确定第三候选参考数据块与待编码数据块之间的行距离。根据第三候选参考数据块的列标识和待编码数据块的列标识确定第三候选参考数据块与待编码数据块之间的列距离。最后，将第三候选参考数据块中列距离小于第一距离阈值的第三候选参考数据块，确定为第二参考数据块；第一距离阈值是根据第一数据块限制距离、第二数据块限制距离、第三候选参考数据块与待编码数据块之间的行距离确定的。
这里，计算机设备可以分别采用上述公式(3)获取第二候选参考数据块的行标识和列标识、待编码数据块的行标识和列标识，然后，可以从该第二候选参考数据块中筛选出行标识小于或等于该待编码数据块的行标识的第二候选参考数据块，作为第三候选参考数据块。本申请实施例中，可以将第三候选参考数据块的行标识和待编码数据块的行标识之间的差值，确定为该第三候选参考数据块与待编码数据块之间的行距离，将该 第三候选参考数据块的列标识和该待编码数据块的列标识之间的差值，确定为第三候选参考数据块与该待编码数据块之间的列距离；并将该第三候选参考数据块中列距离小于第一距离阈值的第三候选参考数据块，确定为第二参考数据块。例如，计算机设备可以将坐标满足如下公式(5)的第二候选参考数据块，确定为第一参考数据块：
Figure PCTCN2022118502-appb-000006
其中，公式(5)中，ref_y_num、ref_x_num分别是指第二候选参考数据块的行标识和列标识，cur_y_num、cur_x_num分别是指待编码数据块的行标识和列标识，P是指在该回写限制条件下该多媒体数据帧中已编码的数据块，与所述待编码数据块之间的第一数据块限制距离，E是指在所述编码处理条件下所述多媒体数据帧中已编码的数据块，与所述待编码数据块之间的第二数据块限制距离。第一数据块限制距离可以是根据该回写限制条件下该多媒体数据帧中已编码的数据块，与所述待编码数据块之间的间隔限制数据块数量确定的，如，P与上述D在数值上可以相同；第二数据块限制距离是根据该编码处理条件下该多媒体数据帧中已编码的数据块，与所述待编码数据块之间的间隔限制数据块数量确定的。ref_x_num-cur_x_num表示第三候选参考数据块与该待编码数据块之间的列距离，cur_y_num-ref_y_num表示第三候选参考数据块与所述待编码数据块之间的行距离。
需要说明的是，当该多媒体数据帧的编码方式为并行编码方式或串行编码方式时，计算机设备还可以通过如下方式获取第一参考数据块集合：根据第一位置信息、第二位置信息以及最大编码块的尺寸信息，确定已编码的数据块与待编码数据块之间的间隔数据块数量，并确定在回写延时条件下以及编码处理条件下已编码的数据块与待编码数据块之间的目标间隔限制数据块数量，目标间隔限制数据块数量是指满足回写延时条件以及编码处理条件下已编码的数据块与待编码数据块之间的最少间隔数据块数量，将多媒体数据帧中与待编码数据块属于同一编码片，且间隔数据块数量大于该目标间隔限制数据块数量的已编码的数据块，添加至第一参考数据块集合。
在一些实施例中，计算机设备可以从如下两种方式中选择一种方式或至少两种组合方式，获取第二参考数据块集合：
方式一：该第二参考数据块集合包括一个或多个第三参考数据块，第三参考数据块为该多媒体数据帧中与该待编码数据块位于同一编码处理单元的已编码的数据块。该编码处理单元也可以称为片上内存的基本单元(VPDU，Virtual Pipeline Data Units)，VPDU的尺寸信息可以根据编码器的编码处理能力确定，例如，VPDU的尺寸为VPDU_W*VPDU_H，VPDU_W和VPDU_H的值均可以为4、8、16、32、64、128等。也就是说，将多媒体数据帧中坐标满足ref_x/VPDU_W＝cur_x/VPDU_W，且ref_y/VPDU_H＝cur_y/VPDU_H的已编码的数据块，作为第二参考数据块。再例如，如果VPDU的大小为64x64，则待编码数据块所属的VPDU 64x64中已编码的数据块均可以作为待编码数据块的参考数据块。或者，如果VPDU的大小为64x64，将多媒体数据帧中坐标满足Floor(ref_x/64)＝Floor(cur_x/64)，且Floor(ref_y/64)＝Floor(cur_y/64)的已编码的数据块，作为第三参考数据块。
在一些实施例中，多媒体数据帧的待编码数据块的参考数据块的各个像素可以位于同一个VPDU中。
方式二：该第二参考数据块集合包括一个或多个第四参考数据块，该第四参考数据块为多媒体数据帧中与待编码数据块具有相邻关系，且与待编码数据块属于同一个编码片的已编码的数据块。其中，第二参考数据块与待编码数据块之间具有相邻关系可以包括如下三种情况中的一种或多种组合情况：a、第四参考数据块与待编码数据块具有相 邻关系是指：第四参考数据块与待编码数据块属于同一个最大编码块。b、第四参考数据块与待编码数据块具有相邻关系是指：第四参考数据块位于与待编码数据块相邻的N个最大编码块内，且最大编码块的尺寸小于尺寸阈值；N是根据所述最大编码块的尺寸信息确定的。例如，多媒体数据帧的编码方向为从左到右，尺寸阈值可以是根据应用场景动态或用户需求设置的，如尺寸阈值可以为128x128，则第一参考数据块与所述待编码数据块具有相邻关系是指：该第四参考数据块位于该待编码数据块的左侧的N个最大编码块内，且该最大编码块的尺寸小于128x128。N可以是根据如下公式(6)确定的：
Figure PCTCN2022118502-appb-000007
其中，log2_sb_size是指中间变量，sb_size是指最大编码块的尺寸。
c、第四参考数据块与待编码数据块具有相邻关系是指：第四参考数据块位于与待编码数据块相邻的N个最大编码块内，且最大编码块的尺寸大于或等于尺寸阈值；在多媒体数据帧中，若第四参考数据块向第一方向移动(例如向右移动)一个最大编码块后，移动后的第四参考数据块所在区域中位于第二方向上最上角(例如最左上角)的数据块为未重建的数据块。这里，第一方向和第二方向为相对的两个方向，其中，第一方向是指第四参考数据块的移动方向，第一方向与待编码数据块的编码方向相同，也就是说，对于横向排列的一排待编码数据块，从左往右是待编码数据块的编码方向，则从左往右为上述第一方向，从右往左为上述第二方向。第四参考数据块移动(移动方向与编码方向相同)一个最大编码块的距离时的位置信息，与待编码数据块的位置信息不相同；N是根据最大编码块的尺寸信息确定的。也就是说，当最大编码块的尺寸大于或等于尺寸阈值，如最大编码块的尺寸为128x128时，第四参考数据块需要满足如下三个条件：1、与待编码数据块属于同一个编码片的已编码的数据块；2、第四参考数据块向右移动128个像素后坐标为(ref_x+128，ref_y)，则多媒体数据帧中坐标为(ref_x+128，ref_y)尺寸为64x64的区域中位于左上角的数据块为未重建的数据块；3、该第四参考数据块向右侧移动一个最大编码块的距离时的位置信息，与该待编码数据块的位置信息不相同，该条件可以采用公式(7)表示：
Figure PCTCN2022118502-appb-000008
其中，公式(7)中！＝表示不等于。
本申请实施例中，该第一目标参考数据块与待编码数据块之间的距离均小于第二距离阈值；该第二距离阈值可以是根据应用场景或用户需求动态设置的，如该第二距离阈值可以为2 14，同时，该第一目标参考数据块位于多媒体数据帧的整像素位置。
例如，当多媒体数据帧的编码方式为并行编码方式，最大编码块的尺寸为128*128，在回写延时限制条件下已编码的数据块与待编码数据块之间的间隔限制数据块数量为4，在编码处理条件下已编码的数据块与待编码数据块之间的间隔限制数据块数量为2，第二参考数据块集合的获取方式为上述方式一，VPDU的尺寸为64x64时，第一参考数据块集合、第二参考数据块集合与待编码数据块之间的关系如图8所示。图8中一个最小方格(即虚线画出的小方格)的尺寸为64x64，实线画出的小方格的尺寸为128x128，数据块的尺寸为32x32。因此，图8中多媒体数据帧中位于第一行的第19个格子中的已编码的数据块属于第二参考数据块集合，该第二参考数据块集合用于对第一行的第19个格子中的待编码数据块进行编码。多媒体数据帧中第一行和第二行中与待编码数据块相隔8个数据块的已编码的数据块属于第一参考数据块集合，该第一参考数据块集合用于对第一行的第19个格子中的待编码数据块进行编码。从图8可知，第一参考数据块集合中的参考数据块数量比第二参考数据块集合中的参考数据块数量，第一参考数据块 集合中的参考数据块与待编码数据块不具有相邻关系，第二参考数据块集合中的参考数据块与待编码数据块之间具有相邻关系。
例如，当多媒体数据帧的编码方式为并行编码方式，最大编码块的尺寸为128*128，在回写延时限制条件下已编码的数据块与待编码数据块之间的间隔限制数据块数量为4，在编码处理条件下已编码的数据块与待编码数据块之间的间隔限制数据块数量为2，第二参考数据块集合的获取方式为上述方式二，VPDU的尺寸为64x64时，第一参考数据块集合、第二参考数据块集合与待编码数据块之间的关系如图9所示。图9中一个最小方格(即虚线画出的小方格)的尺寸为64x64，实线画出的小方格的尺寸为128x128，数据块的尺寸为32x32。因此，图9中多媒体数据帧中位于第一行的第19个格子中的已编码的数据块，以及位于待编码数据块的左边的最大编码块中的已编码的数据块属于第二参考数据块集合，该第二参考数据块集合用于对第一行的第19个格子中的待编码数据块进行编码。多媒体数据帧中第一行和第二行中与待编码数据块相隔8个数据块的已编码的数据块属于第一参考数据块集合，该第一参考数据块集合用于对第一行的第19个格子中的待编码数据块进行编码。从图9可知，第一参考数据块集合中的参考数据块数量比第二参考数据块集合中的参考数据块数量，第一参考数据块集合中的参考数据块与待编码数据块不具有相邻关系，第二参考数据块集合中的参考数据块与待编码数据块之间具有相邻关系。
需要说明的是，如果待编码数据块的色度分量是根据第一目标参考数据块的亮度分量进行预测编码得到的，且该第一目标参考数据块为第一参考数据块时，该第一目标参考数据块的亮度分量和色度分量均需要满足与第一参考数据块相关的条件，与第一参考数据块相关的条件可以是指上述确定第一参考数据块集合的条件。如果待编码数据块的色度分量是根据第一目标参考数据块的亮度分量进行预测编码得到的，且该第一目标参考数据块为第二参考数据块时，该第一目标参考数据块的亮度分量和色度分量均需要满足与第二参考数据块相关的条件，与第二参考数据块相关的条件可以是指上述确定第二参考数据块集合的条件。
在一些实施例中，待编码数据块与已编码数据块之间具有相邻关系可以是指待编码数据块的编码顺序与已编码的数据块的编码顺序相邻；待编码数据块与已编码的数据块之间不具有相邻关系可以是指待编码数据块的编码顺序与已编码的数据块的编码顺序不相邻。
在一些实施例中，多媒体数据帧的待编码数据块的数量为多个，可以获取多媒体数据帧的各个待编码数据块的属性信息；当根据多媒体数据帧各个待编码数据块的属性信息，确定多媒体数据帧中所有的待编码数据块对应的第一目标参考数据块均属于第二参考数据块集合时，对多媒体数据帧中的待编码数据块进行滤波处理；当根据多媒体数据帧各个待编码数据块的属性信息，确定多媒体数据帧中存在待编码数据块对应的第一目标参考数据块选自第一参考数据块集合时，暂停对多媒体数据帧中的待编码数据块进行滤波处理。当根据多媒体数据帧各个待解码数据块的属性信息，确定多媒体数据帧中存在待解码数据块对应的第二目标参考数据块的第一区域属于第一参考数据块集合，第二目标参考数据块的第二区域属于第二参考数据块集合时，暂停对多媒体数据帧中的待解码数据块进行滤波处理。也就是说，在编码端，当多媒体数据帧中的数据块均采用第二参考数据块集合中的参考数据块进行编码，则允许使用环路滤波器对多媒体数据帧进行滤波处理，有利于提高多媒体数据帧的质量。当多媒体数据帧中的数据块存在采用第一参考数据块集合中的参考数据块进行编码，或者，当多媒体数据帧中的数据块存在第一目标参考数据块部分属于第一参考数据块集合，且部分属于第二参考数据块集合时，则禁止使用环路滤波器对多媒体数据帧进行滤波处理，通过禁止使用环路滤波器对多媒体 数据帧进行处理，避免增加额外的图像存储需求，有利于节省存储空间。环路滤波器可以包括但不限于：去块滤波器(DBF，Deblocking filter)、样点自适应补偿(SAO，Sample Adaptive Offset)、自适应环路滤波器(ALF，Adaptive Loop Filter)、约束方向增强滤波器(CDEF，constrained directional enhancement)、环路恢复滤波其(LR，loop restoration)、跨分量的样点补偿(CCSO，Cross Component Sample Offset)等。
图10是本申请实施例提供的一种多媒体数据处理方法的流程示意图，如图10所示，该方法至少可以包括以下S201-S203：
S201、获取多媒体数据帧中待解码数据块的属性信息。
本申请实施例中，计算机设备可以获取该多媒体数据帧中的待解码数据块的属性信息，该待解码数据块可以是指上文中的当前编码数据块，待解码数据块可以是指多媒体数据帧中的未编码的数据块中即将编码的数据块，即待解码数据块可以是指多媒体数据帧中与已解码的数据块的距离最近的未编码数据块。
待解码数据块的属性信息包括待解码数据块所属的多媒体数据帧的媒体类型、多媒体数据帧的参考标志、待解码数据块所属的多媒体数据帧的清晰度等中的一种或多种；媒体类型包括帧内和非帧内，帧内类型可以是指具有代表意义的多媒体数据帧，帧内类型包括全帧内编码帧、Intra帧、I帧和关键帧。帧内类型的帧只允许使用帧内编码技术，不需要依赖其他帧进行编码。例如，帧内类型可以是指能够体现视频数据或图像数据中主要含义的多媒体数据帧。非帧内类型可以是指视频数据或图像数据中除帧内类型以外的多媒体数据帧，非帧内类型可以称为帧间编码帧、P帧、B帧或非关键帧。非帧内类型的帧允许使用帧间编码技术和帧内编码技术，通常当采用帧间编码技术时，需要在参考帧编码完成后才可以进行编码。多媒体数据帧的参考标志用于反映待解码数据块的参考数据块的选择范围。
S202、获取与该待解码数据块的属性信息相匹配的第二目标参考数据块；其中，第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合第一参考数据块集合和第二参考数据块集合的第三参考数据块集合，第一参考数据块集合包括与待解码数据块之间不具有相邻关系的已解码的数据块，第二参考数据块集合包括与待解码数据块之间具有相邻关系的已解码的数据块。
本申请实施例中，多媒体数据帧与待解码数据块不具有相邻关系的已解码的数据块、与待解码数据块具有相邻关系的已解码的数据块均能够用于作为参考数据块，计算机设备通过根据待解码数据块的属性信息，自适应地从具有相邻关系的已解码的数据块和不具有相邻关系的已解码的数据块中选择第二目标参考数据块，可提高获取参考数据块的灵活性以及准确度。
需要说明的是，本申请实施例中的已解码的数据块，也可以称为解码端已重建的数据块，已重建的数据块为指已解码的数据块经过恢复处理后的数据块。第二目标参考数据块可以是指与待解码数据块具有相邻关系的已解码数据块，或，该第二目标参考数据块可以是指与待解码数据块不具有相邻关系的已解码数据块，或者，该第二目标参考数据块的第二区域属于与待解码数据块具有相邻关系的已解码数的据块，且该第二目标参考数据块的第二区域可以属于与待解码数据块不具有相邻关系的已解码的数据块。
S203、根据第二目标参考数据块对待解码数据块进行解码。
本申请实施例中，计算机设备可以根据第二目标参考数据块、待解码数据块分别在多媒体数据帧的位置信息，根据该位置信息确定第二目标参考数据块与待解码数据块之间的块矢量，然后，根据该块矢量以及第二目标参考数据块预测得到待解码数据块的预测信号，将该预测信号与待解码数据块的残差信号进行相加得到该待解码数据块的原始信号。通过对待解码数据块进行解码，有利于去除多媒体数据帧进行恢复处理，提高多 媒体数据帧的质量。
需要说明的是，通常视频数据中包括多个多媒体数据帧，或者，图像数据中包括多个多媒体数据帧，但上述待解码数据块、第二目标参考数据块均属于同一多媒体数据帧。
本申请实施例中，计算机设备通过获取与待解码数据块的属性信息相匹配的第二目标参考数据块，该第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合该第一参考数据块集合和该第二参考数据块集合的第三参考数据块集合；也就是说，可供选择的参考数据块集合包括第一参考数据块集合、第二参考数据块集合以及第三参考数据块集合，扩展了参考数据块的选择范围；同时，根据待解码数据块的属性信息，自适应地选择第二目标参考数据块，根据第二目标参考数据块对待解码数据块进行解码，能够提高获取第二目标参考数据块的灵活性和准确度，提高对多媒体数据帧的解码性能。同时，通过根据第二目标参考数据块对待解码数据块进行解码，有利于对多媒体数据帧进行恢复处理，提高多媒体数据帧的质量。
在一些实施例中，计算机设备可以通过如下三种匹配方式中的一种或多种组合方式，获取与待解码数据块的属性信息匹配的第二目标参考数据块：
匹配方式一：若该多媒体数据帧的媒体类型为帧内类型，则从第一参考数据块集合中或第三参考数据块集合中获取与该待解码数据块的属性信息相匹配的第二目标参考数据块；该第二目标参考数据块是根据该第一参考数据块和该第二参考数据块确定得到；若该多媒体数据帧的媒体类型为非帧内类型，则从第二参考数据块集合中获取与该待解码数据块的属性信息相匹配的第二目标参考数据块。待解码数据块与已解码数据块之间具有相邻关系可以是指待解码数据块的解码顺序与已解码的数据块的解码顺序相邻；待解码数据块与已解码的数据块之间不具有相邻关系可以是指待解码数据块的解码顺序与已解码的数据块的解码顺序不相邻。
需要说明的是，第一参考数据块集合和第二参考数据块集合分别存储在第一存储空间和第二存储空间，第一存储空间和第二存储空间可以位于同一个计算机设备中，也可以位于不同计算机设备中。
在匹配方式一中，如果该多媒体数据帧的媒体类型为帧内类型，可以从第一参考数据块集合中搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息匹配的第二目标参考数据块。如果该多媒体数据帧的媒体类型为非帧内类型，则从该第二参考数据块集合中搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息相匹配的第二目标参考数据块。通过根据多媒体数据帧的媒体类型，自适应地选择与待解码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
匹配方式二：当该待解码数据块的属性信息包括所述待解码数据块所属的多媒体数据帧的参考标志时；计算机设备获取与该待解码数据块的属性信息相匹配的第二目标参考数据块，包括：若该多媒体数据帧的参考标志为第一参考标志，则从第一参考数据块集合或第三参考数据块集合中获取与该待解码数据块的属性信息相匹配的第二目标参考数据块；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中获取与该待解码数据块的属性信息相匹配的第二目标参考数据块。该参考标志可以是指由多媒体数据帧的发送者(或接收者)预先设置的，因此，根据参考标志自适应的选择第二目标参考数据块，相当于根据用户的编码需求自适应的选择第二目标参考数据块，有利于提高选择的参考数据块的灵活性和准确性。
第一参考标志为用于指示待解码数据块的参考数据块的选择范围为第一参考数据块集合或者第三参考数据块集合，第二参考标志为用于指示待解码数据块的参考数据块的选择范围为第二参考数据块集合。第一参考标志、第二参考标志可以是由多媒体数据 帧的发送者(或接收者)预先设置的，第一参考标志、第二参考标志可以是指数字、字母、汉字中的一种或多种构成。
在匹配方式二中，若该多媒体数据帧的参考标志为第二参考标志，则从第一参考数据块集合和第二参考数据块集合中(即第三参考数据块集合)搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息相匹配的第二目标参考数据块，这时第二目标参考数据块是根据第二参考数据块集合和第二参考数据块集合确定得到的；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息相匹配的第二目标参考数据块。通过根据多媒体数据帧的参考标志，自适应地选择与待解码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
在匹配方式二中，若该多媒体数据帧的参考标志为第二参考标志，则从第一参考数据块集合中搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息相匹配的第二目标参考数据块，这时第二目标参考数据块是指第一参考数据块集合中的任一参考数据块；若该多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中搜索与待解码数据块之间具有最强相关性的数据块，作为与该待解码数据块的属性信息相匹配的第二目标参考数据块，这时第二目标参考数据块可以是指第二参考数据块集合中任一参考数据块。通过根据多媒体数据帧的参考标志，自适应地选择与待解码数据块相匹配的参考数据块，有利于提高获取参考数据块的准确度以及灵活性。
匹配方式三，当该待解码数据块的属性信息包括待解码数据块所属的多媒体数据帧的清晰度时，当待解码数据块所属的多媒体数据帧的清晰度大于清晰度阈值，表明用户对多媒体数据帧的质量要求比较高，可以从第一参考数据块集合或第三参考数据块集合中获取与该待解码数据块的属性信息匹配的第二目标参考数据块。当待解码数据块所属的多媒体数据帧的清晰度小于或等于清晰度阈值，表明用户对多媒体数据帧的质量要求比较低，可以从第二参考数据块集合中获取与该待解码数据块的属性信息匹配的第二目标参考数据块。多媒体数据帧的清晰度可以用于反映用户对多媒体数据帧的质量要求，因此，根据多媒体数据帧的清晰度自适应的选择第二目标参考数据块，相当于，根据用户对多媒体数据帧的质量需求自适应的选择第二目标参考数据块，有利于提高选择的参考数据块的灵活性和准确性。
在一些实施例中，计算机设备可以根据多媒体数据帧的解码方式从如下两种方式中选择一种方式，获取第一参考数据块集合：方式一：当该多媒体数据帧的解码方式为串行解码方式时，该第一参考数据块集合中的第一参考数据块为多媒体数据帧中参数关联关系满足回写限制条件，且与待解码数据块属于同一个解码片的已解码的数据块；参数关联关系用于反映第一位置信息，与最大解码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息、第二位置信息分别是指已解码的数据块、待解码数据块在多媒体数据帧中的位置信息；回写限制条件用于反映将已解码的数据块写入至第一参考数据块集合的限制时延。也就是说，串行解码方式是指同一时间仅对多媒体数据帧中的一个数据块进行解码，如果多媒体数据帧的解码方式为串行解码方式时，该解码方式不会对参考数据块的确定过程造成影响。但是，已解码的数据块写入至第一参考数据块集合中会带来较大回写延时，即已解码的数据块写入至第一参考数据块集合中需要消耗较长时间，因此，已解码的数据块至少等待预设时长后，才能够被用于作为参考数据块，预设时长是根据将所述已解码的数据块写入至所述第一参考数据块集合的限制时延确定的，该限制时延可以是指将已解码的数据块写入至第一参考数据块集合中的平均时延、最大时延以及最小时延等中的一种。由此可知，当该多媒体数据帧的解码方式为串行解 码方式时，该第一参考数据块集合中的第一参考数据块需要满足以下两个条件：a、与待解码数据块属于同一个解码片；b、参数关联关系满足回写限制条件。
方式二：当该多媒体数据帧的解码方式为并行解码方式或者为串行解码方式时，该第一参考数据块集合的第一参考数据块为与该待解码数据块属于同一个解码片，且该多媒体数据帧中参数关联关系满足回写限制条件，以及该参数关联关系满足解码处理条件的已解码的数据块；该参数关联关系用于反映已解码的数据块、该待解码数据块分别在该多媒体数据帧中的位置信息，以及最大解码块的尺寸信息之间的关联关系；该回写限制条件用于反映将该已解码的数据块写入至该第一参考数据块集合的限制时延。也就是说，并行解码方式是指同一时间对多媒体数据帧中的多个数据块进行预测解码，如果多媒体数据帧的解码方式为并行解码方式时，该解码方式会对参考数据块的确定过程造成影响。同时，已解码的数据块写入至第一参考数据块集合中会带来较大回写延时，由此可知，当该多媒体数据帧的解码方式为并行解码方式时，该第一参考数据块集合中的第一参考数据块需要满足以下三个条件：a、与待解码数据块属于同一个解码片；b、参数关联关系满足回写限制条件；c、参数关联关系满足解码处理条件。特别的，当该多媒体数据帧的解码方式为串行解码方式时，也可以采用方式二获取第一参考数据块集合。此处解码处理条件可以是指并行解码处理条件。
需要说明的是，第一参考数据块集合的获取方式可以参考编码段获取第一参考数据块集合的获取方式，重复之处不再赘述。
在一些实施例中，计算机设备可以从如下两种方式中选择一种方式或至少两种组合方式，获取第二参考数据块集合：方式一：该第二参考数据块集合的第二参考数据块为该多媒体数据帧中与该待解码数据块位于同一解码处理单元的已解码的数据块。方式二：第二参考数据块集合的第二参考数据块为多媒体数据帧中与待解码数据块具有相邻关系，且与待解码数据块属于同一个解码片的已解码的数据块。其中，第二参考数据块与待解码数据块之间具有相邻关系可以是以下情况中的一种或多种组合：a、第二参考数据块与待解码数据块具有相邻关系是指：第二参考数据块与待解码数据块属于同一个最大解码块。b、第二参考数据块与待解码数据块具有相邻关系是指：第二参考数据块位于与待解码数据块相邻的N个最大解码块内，且最大解码块的尺寸小于尺寸阈值；N是根据最大解码块的尺寸信息确定的。
在一些实施例中，该第一参考数据块、第二参考数据块与待解码数据块之间的距离均小于第二距离阈值；该第二距离阈值可以是根据应用场景或用户需求动态设置的，如该第二距离阈值可以为2 14，同时，该第一参考数据块、第二参考数据块位于多媒体数据帧的整像素位置。多媒体数据帧的待解码数据块的数量为多个，获取根据多媒体数据帧的各个待解码数据块的属性信息；当根据多媒体数据帧各个待解码数据块的属性信息，确定多媒体数据帧中所有的待解码数据块对应的第二目标参考数据块均属于第二参考数据块集合时，对多媒体数据帧中的待解码数据块进行滤波处理；当根据多媒体数据帧各个待解码数据块的属性信息，确定多媒体数据帧中存在待解码数据块对应的第二目标参考数据块属于第一参考数据块集合时，暂停对多媒体数据帧中的待解码数据块进行滤波处理。当根据多媒体数据帧各个待解码数据块的属性信息，确定多媒体数据帧中存在待解码数据块对应的第二目标参考数据块第一区域属于所述第一参考数据块集合，第二目标参考数据块第二区域属于第二参考数据块集合时，暂停对多媒体数据帧中的待解码数据块进行滤波处理。
图11是本申请实施例提供的一种多媒体数据处理装置的结构示意图，该多媒体数据处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，例如该多媒体数据处理装置为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的 相应步骤。如图11所示，该多媒体数据处理装置可以包括：第一获取模块111、第一匹配模块112以及编码模块113。
第一获取模块，配置为获取多媒体数据帧中待编码数据块的属性信息；第一匹配模块，配置为获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；其中，所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，所述第二参考数据块集合包括与所述待编码数据块之间具有相邻关系的已编码的数据块；编码模块，配置为根据所述第一目标参考数据块对所述待编码数据块进行预测编码。
在一些实施例中，所述待编码数据块的属性信息包括所述待编码数据块所属的多媒体数据帧的媒体类型；所述第一匹配模块还配置为：若所述多媒体数据帧的媒体类型为帧内类型，则从所述第一参考数据块集合或所述第三参考数据块集合中中获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；若所述多媒体数据帧的媒体类型为非帧内类型，则从第二参考数据块集合中获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块。
在一些实施例中，所述待编码数据块的属性信息包括所述待编码数据块所属的多媒体数据帧的参考标志；所述第一匹配模块还配置为：若所述多媒体数据帧的参考标志为第一参考标志，则从所述第一参考数据块集合或所述第三参考数据块集合中获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；若所述多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块。
在一些实施例中，所述第一参考数据块集合包括一个或多个第一参考数据块；当所述多媒体数据帧的编码方式为串行编码方式时，第一参考数据块为所述多媒体数据帧中参数关联关系满足回写限制条件，且与所述待编码数据块属于同一个编码片的已编码的数据块；所述参数关联关系用于反映第一位置信息，与最大编码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息、第二位置信息分别是指已编码的数据块、所述待编码数据块在所述多媒体数据帧中的位置信息；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延。
在一些实施例中，第一获取模块还配置为当所述多媒体数据帧的编码方式为串行编码方式时，分别获取第一位置信息，以及所述最大编码块的尺寸信息和第二位置信息；所述第一位置信息、第二位置信息分别是指已编码的数据块、所述待编码数据块在所述多媒体数据帧中的位置信息；获取回写限制条件；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延；从所述多媒体数据帧中筛选出参数关联关系满足回写限制条件，且与所述待编码数据块属于同一个编码片的已编码的数据块，作为第一参考数据块；所述参数关联关系用于反映所述第一位置信息，与所述最大编码块的尺寸信息和所述第二位置信息之间的关联关系；将所述第一参考数据块添加至所述第一参考数据块集合。
在一些实施例中，所述第一获取模块还配置为：根据所述第一位置信息和所述第二位置信息，从所述多媒体数据帧中筛选出与所述待编码数据块属于同一编码片的已编码的数据块，作为第一候选参考数据块；确定在所述回写限制条件下所述多媒体数据帧中已编码的数据块，与所述待编码数据块之间的间隔数据块限制数量；根据所述第一位置信息、所述最大编码块的尺寸信息和所述第二位置信息，统计所述第一候选参考数据块与所述待编码数据块之间的间隔数据块数量；将间隔数据块数量大于所述间隔数据块限制数量的第一候选参考数据块，确定为第一参考数据块。
在一些实施例中，第一获取模块还配置为：根据第一位置信息以及所述最大编码块的尺寸信息确定所述第一候选参考数据块的行标识和列标识；根据所述第二位置信息以及所述最大编码块的尺寸信息确定所述待编码数据块的行标识和列标识；基于所述待编码数据块的尺寸信息确定所述多媒体数据帧中每行的数据块数量；根据所述每行的数据块数量、所述第一候选参考数据块的行标识和列标识，以及所述待编码数据块的行标识和列标识，统计所述第一候选参考数据块与所述待编码数据块之间的间隔数据块数量。
在一些实施例中，所述第一参考数据块集合包括一个或多个第二参考数据块；所述第二参考数据块为与所述待编码数据块属于同一个编码片，且所述多媒体数据帧中参数关联关系满足回写限制条件，以及所述参数关联关系满足编码处理条件的已编码的数据块；所述参数关联关系用于反映已编码的数据块、所述待编码数据块分别在所述多媒体数据帧中的位置信息，以及最大编码块的尺寸信息之间的关联关系；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延。
在一些实施例中，第一获取模块还配置为分别获取第一位置信息，以及所述最大编码块的尺寸信息和第二位置信息；所述第一位置信息、第二位置信息分别是指已编码的数据块、所述待编码数据块在所述多媒体数据帧中的位置信息；获取回写限制条件以及编码处理条件；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延；从所述多媒体数据帧中筛选出参数关联关系满足回写限制条件和所述编码处理条件，且与所述待编码数据块属于同一个编码片的已编码的数据块，作为第二参考数据块；所述参数关联关系用于反映所述第一位置信息，与所述待编码数据块的尺寸信息和所述第二位置信息之间的关联关系；将所述第二参考数据块添加至所述第一参考数据块集合。
在一些实施例中，第一获取模块还配置为：根据所述第一位置信息和所述第二位置信息，从所述多媒体数据帧中筛选出与所述待编码数据块属于同一编码片的已编码的数据块，作为第一候选参考数据块；从所述第一候选参考数据块中筛选出参数关联关系满足回写限制条件的第一候选参考数据块，作为第二候选参考数据块；确定在所述回写限制条件下所述多媒体数据帧中已编码的数据块，与所述待编码数据块之间的第一数据块限制距离；确定在所述编码处理条件下所述多媒体数据帧中已编码的数据块，与所述待编码数据块之间的第二数据块限制距离；根据所述第一位置信息、所述最大编码块的尺寸信息和所述第二位置信息、所述第一数据块限制距离和所述第二数据块限制距离，从所述第二候选参考数据块中确定出满足所述编码处理条件的第二候选参考数据块，作为第二参考数据块。
在一些实施例中，第一获取模块还配置为：根据所述第一位置信息以及所述最大编码块的尺寸信息确定所述第二候选参考数据块的行标识和列标识；根据所述第二位置信息以及所述最大编码块的尺寸信息确定所述待编码数据块的行标识和列标识；从所述第二候选参考数据块中筛选出行标识小于或等于所述待编码数据块的行标识的第二候选参考数据块，作为第三候选参考数据块；根据所述第三候选参考数据块的行标识和所述待编码数据块的行标识确定所述第三候选参考数据块与所述待编码数据块之间的行距离；根据所述第三候选参考数据块的列标识和所述待编码数据块的列标识确定所述第三候选参考数据块与所述待编码数据块之间的列距离；将所述第三候选参考数据块中列距离小于第一距离阈值的第三候选参考数据块，确定为第二参考数据块；所述第一距离阈值是根据所述第一数据块限制距离、所述第二数据块限制距离、所述第三候选参考数据块与所述待编码数据块之间的行距离确定的。
在一些实施例中，所述第二参考数据块集合包括一个或多个第三参考数据块，第三参考数据块为所述多媒体数据帧中与所述待编码数据块位于同一编码处理单元的已编 码的数据块；或者，所述第二参考数据块集合包括一个或多个第四参考数据块，所述第四参考数据块为所述多媒体数据帧中与所述待编码数据块具有相邻关系，且与所述待编码数据块属于同一个编码片的已编码的数据块。
在一些实施例中，所述第四参考数据块与所述待编码数据块具有相邻关系是指：所述第二参考数据块与所述待编码数据块属于同一个最大编码块；或者，所述第二参考数据块位于与所述待编码数据块相邻的N个最大编码块内，且所述最大编码块的尺寸小于或等于尺寸阈值；N是根据所述最大编码块的尺寸信息确定的；或者，所述第二参考数据块位于与所述待编码数据块相邻的N个最大编码块内，且所述最大编码块的尺寸大于或等于尺寸阈值；所述多媒体数据帧中所述第二参考数据块向右移动一个最大编码块后所在区域中位于左上角的数据块为未重建的数据块；所述第二参考数据块移动一个最大编码块的距离时的位置信息，与所述待编码数据块的位置信息不相同；N是根据所述最大编码块的尺寸信息确定的。
在一些实施例中，所述第一目标参考数据块与所述待编码数据块之间的距离均小于第二距离阈值；所述第一目标参考数据块位于所述多媒体数据帧的整像素位置。
在一些实施例中，所述多媒体数据帧的待编码数据块的数量为多个，第一获取模块还配置为获取根据所述多媒体数据帧的各个待编码数据块的属性信息；当根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中所有的待编码数据块对应的第一目标参考数据块均属于所述第二参考数据块集合时，对所述多媒体数据帧中的待编码数据块进行滤波处理；当根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中存在待编码数据块对应的第一目标参考数据块选自所述第一参考数据块集合时，暂停对所述多媒体数据帧中的待编码数据块进行滤波处理。当根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中存在待编码数据块对应的第一目标参考数据块的第一区域属于所述第一参考数据块集合，所述第一目标参考数据块的第二区域属于所述第二参考数据块集合时，暂停对所述多媒体数据帧中的待编码数据块进行滤波处理
根据本申请的一个实施例，图7所示的多媒体数据处理方法所涉及的步骤可由图11所示的多媒体数据处理装置中的各个模块来执行。例如，图7中所示的步骤S101可由图11中的第一获取模块111来执行，图7中所示的步骤S102可由图11中的第一匹配模块112来执行；图7中所示的步骤S103可由图11中的编码模块113来执行。
图12是本申请实施例提供的另一种多媒体数据处理装置的结构示意图，该多媒体数据处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，例如该多媒体数据处理装置为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图12所示，该多媒体数据处理装置可以包括：第二获取模块121、第二匹配模块122以及解码模块123。
第二获取模块，配置为获取多媒体数据帧中待解码数据块的属性信息；第二匹配模块，配置为获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；其中，所述第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待解码数据块之间不具有相邻关系的已解码的数据块，所述第二参考数据块集合包括与所述待解码数据块之间具有相邻关系的已解码的数据块；解码模块，配置为根据所述第二目标参考数据块对所述待解码数据块进行解码。
在一些实施例中，所述待解码数据块的属性信息包括所述待解码数据块所属的多媒体数据帧的媒体类型；所述第二匹配模块还配置为：若所述多媒体数据帧的媒体类型为帧内类型，则从第一参考数据块集合中获取与所述待解码数据块的属性信息相匹配的第 二目标参考数据块；若所述多媒体数据帧的媒体类型为非帧内类型，则从所述第一参考数据块集合或者第三参考数据块集合中获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块。
在一些实施例中，所述待解码数据块的属性信息包括所述待解码数据块所属的多媒体数据帧的参考标志；所述第二匹配模块还配置为：若所述多媒体数据帧的参考标志为第一参考标志，则从所述第一参考数据块集合或者第三参考数据块集合中获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；若所述多媒体数据帧的参考标志为第二参考标志，则从第二参考数据块集合中获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块。
在一些实施例中，所述第一参考数据块集合包括一个或多个第一参考数据块；当所述多媒体数据帧的解码方式为串行解码方式时，所述第一参考数据块为所述多媒体数据帧中参数关联关系满足回写限制条件，且与所述待解码数据块属于同一个解码片的已解码的数据块；所述参数关联关系用于反映第一位置信息，与最大解码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息、第二位置信息分别是指已解码的数据块、所述待解码数据块在所述多媒体数据帧中的位置信息；所述回写限制条件用于反映将所述已解码的数据块写入至所述第一参考数据块集合的限制时延。
在一些实施例中，所述第一参考数据块集合包括一个或多个第二参考数据块；所述第二参考数据块为与所述待解码数据块属于同一个解码片，且所述多媒体数据帧中参数关联关系满足回写限制条件，以及所述参数关联关系满足解码处理条件的已解码的数据块；所述参数关联关系用于反映已解码的数据块、所述待解码数据块分别在所述多媒体数据帧中的位置信息，以及最大解码块的尺寸信息之间的关联关系；所述回写限制条件用于反映将所述已解码的数据块写入至所述第一参考数据块集合的限制时延。
在一些实施例中，所述第二参考数据块集合包括一个或多个第三参考数据块，所述第三参考数据块为所述多媒体数据帧中与所述待解码数据块位于同一解码处理单元的已解码的数据块；或者，所述第二参考数据块集合包括一个或多个第四参考数据块，所述第四参考数据块为所述多媒体数据帧中与所述待解码数据块具有相邻关系，且与所述待解码数据块属于同一个解码片的已解码的数据块。
在一些实施例中，所述多媒体数据帧的待解码数据块的数量为多个，第二获取模块还配置为获取根据所述多媒体数据帧的各个待解码数据块的属性信息；当根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中所有的待解码数据块对应的第二目标参考数据块均属于所述第二参考数据块集合时，对所述多媒体数据帧中的待解码数据块进行滤波处理；当根据所述多媒体数据帧各个待解码数据块的属性信息，确定所述多媒体数据帧中存在待解码数据块对应的第二目标参考数据块属于所述第一参考数据块集合时，暂停对所述多媒体数据帧中的待解码数据块进行滤波处理。当根据所述多媒体数据帧各个待解码数据块的属性信息，确定所述多媒体数据帧中存在待解码数据块对应的第二目标参考数据块的第一区域属于所述第一参考数据块集合，所述第二目标参考数据块的第二区域属于所述第二参考数据块集合时，暂停对所述多媒体数据帧中的待解码数据块进行滤波处理。
根据本申请的一个实施例，图10所示的多媒体数据处理方法所涉及的步骤可由图12所示的多媒体数据处理装置中的各个模块来执行。例如，图10中所示的步骤S201可由图12中的第二获取模块121来执行，图10中所示的步骤S202可由图12中的第二匹配模块122来执行；图10中所示的步骤S203可由图12中的解码模块123来执行。
根据本申请的一个实施例，图11和图12所示的多媒体数据处理装置中的各个模块可以分别或全部合并为一个或若干个单元来构成，或者其中的某个(些)单元还可以再 拆分为功能上更小的至少两个子单元，可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述模块是基于逻辑功能划分的，在实际应用中，一个模块的功能也可以由至少两个单元来实现，或者至少两个模块的功能由一个单元实现。在本申请的其它实施例中，数据处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由至少两个单元协作实现。
根据本申请的一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算机设备上运行能够执行如图7和图10中所示的相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造如图11和图12中所示的多媒体数据处理装置，以及来实现本申请实施例的多媒体数据处理方法。上述计算机程序可以记载于例如计算机可读存储介质上，并通过计算机可读存储介质装载于上述计算机设备中，并在其中运行。
图13是本申请实施例提供的一种计算机设备的结构示意图，如图13所示，上述计算机设备1000可以包括：处理器1001，网络接口1004和存储器1005，此外，上述计算机设备1000还可以包括：媒体内容接口1003，和至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。其中，媒体内容接口1003可以包括显示屏(Display)、键盘(Keyboard)，可选媒体内容接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口(如WIFI接口)。存储器1005可以是高速RAM存储器，也可以是非易失性的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005可选的还可以是至少一个位于远离前述处理器1001的存储装置。如图13所示，作为一种计算机可读存储介质的存储器1005中可以包括操作系统、网络通信模块、媒体内容接口模块以及设备控制应用程序。在图13所示的计算机设备1000中，网络接口1004可提供网络通讯功能；而媒体内容接口1003主要用于为媒体内容提供输入的接口。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图7以及前文图10所对应实施例中对上述多媒体数据处理方法的描述，也可执行前文图11和图12所对应实施例中对上述多媒体数据处理装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
此外，这里需要指出的是：本申请实施例还提供了一种计算机可读存储介质，且上述计算机可读存储介质中存储有前文提及的多媒体数据处理装置所执行的计算机程序，且上述计算机程序包括程序指令，当上述处理器执行上述程序指令时，能够执行前文图7和图10对应实施例中对上述多媒体数据处理方法的描述，因此，这里将不再进行赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。对于本申请所涉及的计算机可读存储介质实施例中未披露的技术细节，请参照本申请方法实施例的描述。
作为示例，上述程序指令可被部署在一个计算机设备上执行，或者被部署位于一个地点的至少两个计算机设备上执行，又或者，在分布在至少两个地点且通过通信网络互连的至少两个计算机设备上执行，分布在至少两个地点且通过通信网络互连的至少两个计算机设备可以组成区块链网络。
上述计算机可读存储介质可以是前述任一实施例提供的多媒体数据处理装置或者上述计算机设备的内部存储单元，例如计算机设备的硬盘或内存。该计算机可读存储介质也可以是该计算机设备的外部存储设备，例如该计算机设备上配备的插接式硬盘，智能存储卡(SMC，Smart Media Card)，安全数字(SD，Secure Digital)卡，闪存卡(flash card)等。进一步地，该计算机可读存储介质还可以既包括该计算机设备的内部存储单元也包括外部存储设备。该计算机可读存储介质用于存储该计算机程序以及该计算机设备所需的其他程序和数据。该计算机可读存储介质还可以用于暂时地存储已经输出或者 将要输出的数据。
本申请实施例还提供了一种计算机程序产品，包括计算机程序或计算机指令，所述计算机程序或计算机指令被处理器执行时实现前文图7和图10对应实施例中对上述多媒体数据处理方法的描述，因此，这里将不再进行赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。对于本申请所涉及的计算机程序产品的实施例中未披露的技术细节，请参照本申请方法实施例的描述。
本申请实施例的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同媒体内容，而非用于描述特定顺序。此外，术语“包括”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、装置、产品或设备没有限定于已列出的步骤或模块，而是可选地还包括没有列出的步骤或模块，或可选地还包括对于这些过程、方法、装置、产品或设备固有的其他步骤单元。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例提供的方法及相关装置是参照本申请实施例提供的方法流程图和/或结构示意图来描述的，具体可由计算机程序指令实现方法流程图和/或结构示意图的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。这些计算机程序指令可提供到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或结构示意图一个方框或多个方框中指定的功能的装置。这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或结构示意图一个方框或多个方框中指定的功能。这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或结构示意一个方框或多个方框中指定的功能的步骤。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (31)

  1. 一种多媒体数据处理方法，包括：
    获取多媒体数据帧中待编码数据块的属性信息；
    获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；其中，所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，所述第二参考数据块集合包括与所述待编码数据块之间具有相邻关系的已编码的数据块；
    根据所述第一目标参考数据块对所述待编码数据块进行预测编码。
  2. 如权利要求1所述的方法，其中，所述待编码数据块的属性信息包括所述待编码数据块所属的多媒体数据帧的媒体类型；
    所述获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块，包括：
    若所述多媒体数据帧的媒体类型为帧内类型，从所述第一参考数据块集合或所述第三参考数据块集合中获取与待编码数据块的属性信息相匹配的第一目标参考数据块；
    若所述多媒体数据帧的媒体类型为非帧内类型，从第二参考数据块集合中获取与待编码数据块的属性信息相匹配的第一目标参考数据块。
  3. 如权利要求1所述的方法，其中，所述待编码数据块的属性信息包括所述待编码数据块所属的多媒体数据帧的参考标志；
    所述获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块，包括：
    若所述多媒体数据帧的参考标志为第一参考标志，从所述第一参考数据块集合或所述第三参考数据块集合中获取与待编码数据块的属性信息相匹配的第一目标参考数据块；
    若所述多媒体数据帧的参考标志为第二参考标志，从第二参考数据块集合中获取与待编码数据块的属性信息相匹配的第一目标参考数据块；
    其中，所述第一参考标志用于指示待编码数据块的参考数据块的选择范围为第一参考数据块集合或第三参考数据块集合；所述第二参考标志用于指示待编码数据块的参考数据块的选择范围为第二参考数据块集合。
  4. 如权利要求1至3任一项所述的方法，其中，所述第一参考数据块集合包括一个或多个第一参考数据块；
    若所述多媒体数据帧的编码方式为串行编码方式，所述第一参考数据块为满足以下两个条件的已编码的数据块：在所述多媒体数据帧中的参数关联关系满足回写限制条件、与所述待编码数据块属于同一个编码片；
    所述参数关联关系用于反映第一位置信息、最大编码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待编码数据块在所述多媒体数据帧中的位置信息；
    所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延。
  5. 如权利要求1至3任一项所述的方法，其中，所述第一参考数据块集合包括一个或多个第二参考数据块；
    所述第二参考数据块为满足以下三个条件的已编码的数据块：与所述待编码数据块属于同一个编码片、在所述多媒体数据帧中的参数关联关系满足回写限制条件、所述参数关联关系满足编码处理条件；
    所述参数关联关系用于反映第一位置信息、最大编码块的尺寸信息和第二位置信息 之间的关联关系，所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待编码数据块在所述多媒体数据帧中的位置信息；
    所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延。
  6. 如权利要求1至3任一项所述的方法，其中，所述方法还包括：
    若所述多媒体数据帧的编码方式为串行编码方式，分别获取第一位置信息、最大编码块的尺寸信息和第二位置信息；所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待编码数据块在所述多媒体数据帧中的位置信息；
    获取回写限制条件；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延；
    从所述多媒体数据帧中筛选出参数关联关系满足回写限制条件，且与所述待编码数据块属于同一个编码片的已编码的数据块；
    将筛选出的已编码的数据块，确定为第一参考数据块；所述参数关联关系用于反映所述第一位置信息、所述最大编码块的尺寸信息和所述第二位置信息之间的关联关系；
    将所述第一参考数据块添加至所述第一参考数据块集合中。
  7. 如权利要求6所述的方法，其中，所述从所述多媒体数据帧中筛选出参数关联关系满足回写限制条件，且与所述待编码数据块属于同一个编码片的已编码的数据块，包括：
    根据所述第一位置信息和所述第二位置信息，从所述多媒体数据帧中，将与所述待编码数据块属于同一编码片的已编码的数据块确定为第一候选参考数据块；
    在所述回写限制条件下，确定所述多媒体数据帧中已编码的数据块与所述待编码数据块之间的间隔数据块限制数量；
    根据所述第一位置信息、所述最大编码块的尺寸信息和所述第二位置信息，统计所述第一候选参考数据块与所述待编码数据块之间的间隔数据块数量；
    将间隔数据块数量大于所述间隔数据块限制数量的第一候选参考数据块，确定为筛选出的已编码的数据块。
  8. 如权利要求7所述的方法，其中，所述根据所述第一位置信息、所述最大编码块的尺寸信息和第二位置信息，统计所述第一候选参考数据块与所述待编码数据块之间的间隔数据块数量，包括：
    根据所述第一位置信息以及所述最大编码块的尺寸信息，确定所述第一候选参考数据块的行标识和列标识；
    根据所述第二位置信息以及所述最大编码块的尺寸信息，确定所述待编码数据块的行标识和列标识；
    基于所述待编码数据块的尺寸信息，确定所述多媒体数据帧中每行的数据块数量；
    根据所述每行的数据块数量、所述第一候选参考数据块的行标识和列标识，以及所述待编码数据块的行标识和列标识，统计所述第一候选参考数据块与所述待编码数据块之间的间隔数据块数量。
  9. 如权利要求1至3任一项所述的方法，其中，所述方法还包括：
    分别获取第一位置信息、最大编码块的尺寸信息和第二位置信息；所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待编码数据块在所述多媒体数据帧中的位置信息；
    获取回写限制条件以及编码处理条件；所述回写限制条件用于反映将所述已编码的数据块写入至所述第一参考数据块集合的限制时延；
    从所述多媒体数据帧中，将参数关联关系满足回写限制条件和所述编码处理条件，且与所述待编码数据块属于同一个编码片的已编码的数据块，确定为第二参考数据块；所述参数关联关系用于反映所述第一位置信息、所述待编码数据块的尺寸信息和所述第二位置信息之间的关联关系；
    将所述第二参考数据块添加至所述第一参考数据块集合。
  10. 如权利要求9所述的方法，其中，所述从所述多媒体数据帧中，将参数关联关系满足回写限制条件和所述编码处理条件，且与所述待编码数据块属于同一个编码片的已编码的数据块，确定为第二参考数据块，包括：
    根据所述第一位置信息和所述第二位置信息，从所述多媒体数据帧中，将与所述待编码数据块属于同一编码片的已编码的数据块确定为第一候选参考数据块；
    在所述第一候选参考数据块中，将参数关联关系满足回写限制条件的第一候选参考数据块确定为第二候选参考数据块；
    在所述回写限制条件下，确定所述多媒体数据帧中已编码的数据块与所述待编码数据块之间的第一数据块限制距离；
    在所述编码处理条件下，确定所述多媒体数据帧中已编码的数据块与所述待编码数据块之间的第二数据块限制距离；
    根据所述第一位置信息、所述最大编码块的尺寸信息、所述第二位置信息、所述第一数据块限制距离和所述第二数据块限制距离，从所述第二候选参考数据块中，将满足所述编码处理条件的第二候选参考数据块确定为所述第二参考数据块。
  11. 如权利要求10所述的方法，其中，所述根据所述第一位置信息、所述最大编码块的尺寸信息、所述第二位置信息、所述第一数据块限制距离和所述第二数据块限制距离，从所述第二候选参考数据块中，将满足所述编码处理条件的第二候选参考数据块确定为第二参考数据块，包括：
    根据所述第一位置信息以及所述最大编码块的尺寸信息，确定所述第二候选参考数据块的行标识和列标识；
    根据所述第二位置信息以及所述最大编码块的尺寸信息，确定所述待编码数据块的行标识和列标识；
    在所述第二候选参考数据块中，将行标识小于或等于所述待编码数据块的行标识的第二候选参考数据块，确定为第三候选参考数据块；
    根据所述第三候选参考数据块的行标识和所述待编码数据块的行标识，确定所述第三候选参考数据块与所述待编码数据块之间的行距离；
    根据所述第三候选参考数据块的列标识和所述待编码数据块的列标识，确定所述第三候选参考数据块与所述待编码数据块之间的列距离；
    将列距离小于第一距离阈值的第三候选参考数据块，确定为所述第二参考数据块；所述第一距离阈值是根据所述第一数据块限制距离、所述第二数据块限制距离、所述第三候选参考数据块与所述待编码数据块之间的行距离确定的。
  12. 如权利要求1至3任一项所述的方法，其中，
    所述第二参考数据块集合包括一个或多个第三参考数据块，
    所述第三参考数据块为所述多媒体数据帧中与所述待编码数据块位于同一编码处理单元的已编码的数据块。
  13. 如权利要求1至3任一项所述的方法，其中，所述第二参考数据块集合包括一个或多个第四参考数据块，所述第四参考数据块为所述多媒体数据帧中与所述待编码数据块具有相邻关系，且与所述待编码数据块属于同一个编码片的已编码的数据块。
  14. 如权利要求13所述的方法，其中，所述第四参考数据块与所述待编码数据块 具有相邻关系是指：所述第四参考数据块与所述待编码数据块属于同一个最大编码块。
  15. 如权利要求13所述的方法，其中，所述第四参考数据块与所述待编码数据块具有相邻关系是指：所述第四参考数据块位于与所述待编码数据块相邻的N个最大编码块内，且所述最大编码块的尺寸小于或等于尺寸阈值；N是根据所述最大编码块的尺寸信息确定的。
  16. 如权利要求13所述的方法，其中，所述第四参考数据块与所述待编码数据块具有相邻关系是指：所述第四参考数据块位于与所述待编码数据块相邻的N个最大编码块内，且所述最大编码块的尺寸大于或等于尺寸阈值；其中，在所述多媒体数据帧中，若将所述第四参考数据块向第一方向移动一个最大编码块，移动后的第四参考数据块所在区域中位于第二方向的最上角的数据块为未重建的数据块，所述第一方向和所述第二方向为相对的两个方向；所述第四参考数据块移动一个最大编码块的距离时的位置信息，与所述待编码数据块的位置信息不相同。
  17. 如权利要求1至3任一项所述的方法，其中，所述第一目标参考数据块与所述待编码数据块之间的距离均小于第二距离阈值；
    所述第一目标参考数据块位于所述多媒体数据帧的整像素位置。
  18. 如权利要求1至3任一项所述的方法，其中，所述多媒体数据帧的待编码数据块的数量为多个，所述方法还包括：
    获取所述多媒体数据帧的各个待编码数据块的属性信息；
    若根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中所有的待编码数据块对应的第一目标参考数据块均属于所述第二参考数据块集合，对所述多媒体数据帧中的待编码数据块进行滤波处理；
    若根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中存在待编码数据块对应的所述第一目标参考数据块选自所述第一参考数据块集合，暂停对所述多媒体数据帧中的待编码数据块进行滤波处理；
    若根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中存在待编码数据块对应的所述第一目标参考数据块的第一区域属于所述第一参考数据块集合，且所述第一目标参考数据块的第二区域属于所述第二参考数据块集合，暂停对所述多媒体数据帧中的待编码数据块进行滤波处理。
  19. 一种多媒体数据处理方法，包括：
    获取多媒体数据帧中待解码数据块的属性信息；
    获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；其中，所述第二目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待解码数据块之间不具有相邻关系的已解码的数据块，所述第二参考数据块集合包括与所述待解码数据块之间具有相邻关系的已解码的数据块；
    根据所述第二目标参考数据块对所述待解码数据块进行解码。
  20. 如权利要求19所述的方法，其中，所述待解码数据块的属性信息包括所述待解码数据块所属的多媒体数据帧的媒体类型；
    所述获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块，包括：
    若所述多媒体数据帧的媒体类型为帧内类型，从所述第一参考数据块集合或者所述第三参考数据块集合中确定与所述待解码数据块的属性信息相匹配的第二目标参考数据块；
    若所述多媒体数据帧的媒体类型为非帧内类型，从所述第二参考数据块集合中确定与所述待解码数据块的属性信息相匹配的第二目标参考数据块。
  21. 如权利要求19所述的方法，其中，所述待解码数据块的属性信息包括所述待解码数据块所属的多媒体数据帧的参考标志；
    所述获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块，包括：
    若所述多媒体数据帧的参考标志为第一参考标志，从所述第一参考数据块集合或所述第三参考数据块集合中确定与所述待解码数据块的属性信息相匹配的第二目标参考数据块；
    若所述多媒体数据帧的参考标志为第二参考标志，从所述第二参考数据块集合中确定与所述待解码数据块的属性信息相匹配的第二目标参考数据块；
    其中，所述第一参考标志用于指示待解码数据块的参考数据块的选择范围为第一参考数据块集合或第三参考数据块集合；所述第二参考标志用于指示待解码数据块的参考数据块的选择范围为第二参考数据块集合。
  22. 如权利要求19至21任一项所述的方法，其中，所述第一参考数据块集合包括一个或多个第一参考数据块；
    若所述多媒体数据帧的解码方式为串行解码方式，所述第一参考数据块为满足以下两个条件的已编码的数据块：在所述多媒体数据帧中的参数关联关系满足回写限制条件、与所述待解码数据块属于同一个解码片；
    所述参数关联关系用于反映第一位置信息、最大解码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待解码数据块在所述多媒体数据帧中的位置信息；
    所述回写限制条件用于反映将所述已解码的数据块写入至所述第一参考数据块集合的限制时延。
  23. 如权利要求19至21任一项所述的方法，其中，所述第一参考数据块集合包括一个或多个第二参考数据块；
    所述第二参考数据块为满足以下三个条件的已编码的数据块：与所述待解码数据块属于同一个解码片、在所述多媒体数据帧中的参数关联关系满足回写限制条件、所述参数关联关系满足解码处理条件；
    所述参数关联关系用于反映第一位置信息、最大编码块的尺寸信息和第二位置信息之间的关联关系，所述第一位置信息是指已编码的数据块在所述多媒体数据帧中的位置信息；所述第二位置信息是指所述待编码数据块在所述多媒体数据帧中的位置信息；
    所述回写限制条件用于反映将所述已解码的数据块写入至所述第一参考数据块集合的限制时延。
  24. 如权利要求19至21任一项所述的方法，其中，
    所述第二参考数据块集合包括一个或多个第三参考数据块，所述第三参考数据块为所述多媒体数据帧中与所述待解码数据块位于同一解码处理单元的已解码的数据块。
  25. 如权利要求19至21任一项所述的方法，其中，
    所述第二参考数据块集合包括一个或多个第四参考数据块，所述第四参考数据块为所述多媒体数据帧中与所述待解码数据块具有相邻关系，且与所述待解码数据块属于同一个解码片的已解码的数据块。
  26. 如权利要求19所述的方法，其中，所述多媒体数据帧的待解码数据块的数量为多个，所述方法还包括：
    获取所述多媒体数据帧的各个待解码数据块的属性信息；
    若根据所述多媒体数据帧各个待编码数据块的属性信息，确定所述多媒体数据帧中所有的待解码数据块对应的第二目标参考数据块均属于所述第二参考数据块集合，对所述多媒体数据帧中的待解码数据块进行滤波处理；
    若根据所述多媒体数据帧各个待解码数据块的属性信息，确定所述多媒体数据帧中存在待解码数据块对应的第二目标参考数据块属于所述第一参考数据块集合，暂停对所述多媒体数据帧中的待解码数据块进行滤波处理；
    若根据所述多媒体数据帧各个待解码数据块的属性信息，确定所述多媒体数据帧中存在待解码数据块对应的第二目标参考数据块第一区域属于所述第一参考数据块集合，且所述第二目标参考数据块第二区域属于所述第二参考数据块集合，暂停对所述多媒体数据帧中的待解码数据块进行滤波处理。
  27. 一种多媒体数据处理装置，包括：
    第一获取模块，配置为获取多媒体数据帧中待编码数据块的属性信息；
    第一匹配模块，配置为获取与所述待编码数据块的属性信息相匹配的第一目标参考数据块；其中，所述第一目标参考数据块选自第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待编码数据块之间不具有相邻关系的已编码的数据块，所述第二参考数据块集合包括与所述待编码数据块之间具有相邻关系的已编码的数据块；
    编码模块，配置为根据所述第一目标参考数据块对所述待编码数据块进行预测编码。
  28. 一种多媒体数据处理装置，包括：
    第二获取模块，配置为获取多媒体数据帧中待解码数据块的属性信息；
    第二匹配模块，配置为获取与所述待解码数据块的属性信息相匹配的第二目标参考数据块；其中，所述第二目标参考数据块属于第一参考数据块集合、第二参考数据块集合，或融合所述第一参考数据块集合和所述第二参考数据块集合的第三参考数据块集合，所述第一参考数据块集合包括与所述待解码数据块之间不具有相邻关系的已解码的数据块，所述第二参考数据块集合包括与所述待解码数据块之间具有相邻关系的已解码的数据块；
    解码模块，配置为根据所述第二目标参考数据块对所述待解码数据块进行解码。
  29. 一种计算机设备，包括：处理器以及存储器；
    上述处理器与存储器相连；所述存储器配置为存储程序指令，所述处理器配置为调用所述程序指令，以执行如权利要求1至26任一项所述的多媒体数据处理方法。
  30. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时使所述处理器执行如权利要求1至26任一项所述的多媒体数据处理方法。
  31. 一种计算机程序产品，所述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中；
    当计算机设备的处理器从所述计算机可读存储介质读取所述计算机程序，并执行所述计算机程序时，实现权利要求1至26任一项所述的多媒体数据处理方法。
PCT/CN2022/118502 2021-10-20 2022-09-13 多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品 WO2023065891A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2023560291A JP2024514294A (ja) 2021-10-20 2022-09-13 マルチメディアデータ処理方法、装置、機器、コンピュータ可読記憶媒体及びコンピュータプログラム
EP22882502.2A EP4395312A1 (en) 2021-10-20 2022-09-13 Multimedia data processing method and apparatus, device, computer-readable storage medium and computer program product
KR1020237037580A KR20230162989A (ko) 2021-10-20 2022-09-13 멀티미디어 데이터 프로세싱 방법, 장치, 디바이스, 컴퓨터-판독가능 저장 매체, 및 컴퓨터 프로그램 제품
US18/474,712 US20240015310A1 (en) 2021-10-20 2023-09-26 Multimedia data processing method, apparatus, device, computer-readable storage medium, and computer program product

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111221890.5 2021-10-20
CN202111221890.5A CN113992635B (zh) 2021-10-20 2021-10-20 多媒体数据处理方法、装置、设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/474,712 Continuation US20240015310A1 (en) 2021-10-20 2023-09-26 Multimedia data processing method, apparatus, device, computer-readable storage medium, and computer program product

Publications (1)

Publication Number Publication Date
WO2023065891A1 true WO2023065891A1 (zh) 2023-04-27

Family

ID=79739604

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/118502 WO2023065891A1 (zh) 2021-10-20 2022-09-13 多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品

Country Status (6)

Country Link
US (1) US20240015310A1 (zh)
EP (1) EP4395312A1 (zh)
JP (1) JP2024514294A (zh)
KR (1) KR20230162989A (zh)
CN (2) CN113992635B (zh)
WO (1) WO2023065891A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113992635B (zh) * 2021-10-20 2022-09-20 腾讯科技（深圳）有限公司 多媒体数据处理方法、装置、设备及存储介质
CN115396645B (zh) * 2022-08-18 2024-04-19 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190208217A1 (en) * 2017-12-29 2019-07-04 Microsoft Technology Licensing, Llc Constraints on locations of reference blocks for intra block copy prediction
CN110944178A (zh) * 2018-09-22 2020-03-31 上海天荷电子信息有限公司 使用多个以编码参数关联的候选集的数据压缩方法和装置
CN111050177A (zh) * 2018-10-15 2020-04-21 腾讯科技（深圳）有限公司 视频编码、视频解码方法、装置、计算机设备和存储介质
CN112369027A (zh) * 2018-06-28 2021-02-12 高通股份有限公司 具有用于帧内预测的多条参考线的位置相关帧内预测组合
US20210266557A1 (en) * 2018-06-11 2021-08-26 Samsung Electronics Co., Ltd. Encoding method and apparatus therefor, and decoding method and apparatus therefor
CN113992635A (zh) * 2021-10-20 2022-01-28 腾讯科技（深圳）有限公司 多媒体数据处理方法、装置、设备及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109688407B (zh) * 2017-10-18 2020-11-06 北京金山云网络技术有限公司 编码单元的参考块选择方法、装置、电子设备及存储介质
CN116193127A (zh) * 2018-09-22 2023-05-30 上海天荷电子信息有限公司 专用历史最近运动矢量预测候选集的图像压缩方法及装置
EP4221226A1 (en) * 2018-11-07 2023-08-02 HFI Innovation Inc. Method and apparatus of encoding or decoding video blocks by current picture referencing coding
CN110798683B (zh) * 2019-08-26 2022-05-27 腾讯科技（深圳）有限公司 数据解码方法、装置和数据编码方法、装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190208217A1 (en) * 2017-12-29 2019-07-04 Microsoft Technology Licensing, Llc Constraints on locations of reference blocks for intra block copy prediction
US20210266557A1 (en) * 2018-06-11 2021-08-26 Samsung Electronics Co., Ltd. Encoding method and apparatus therefor, and decoding method and apparatus therefor
CN112369027A (zh) * 2018-06-28 2021-02-12 高通股份有限公司 具有用于帧内预测的多条参考线的位置相关帧内预测组合
CN110944178A (zh) * 2018-09-22 2020-03-31 上海天荷电子信息有限公司 使用多个以编码参数关联的候选集的数据压缩方法和装置
CN111050177A (zh) * 2018-10-15 2020-04-21 腾讯科技（深圳）有限公司 视频编码、视频解码方法、装置、计算机设备和存储介质
CN113992635A (zh) * 2021-10-20 2022-01-28 腾讯科技（深圳）有限公司 多媒体数据处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN113992635A (zh) 2022-01-28
CN115996214A (zh) 2023-04-21
JP2024514294A (ja) 2024-04-01
EP4395312A1 (en) 2024-07-03
KR20230162989A (ko) 2023-11-29
CN113992635B (zh) 2022-09-20
US20240015310A1 (en) 2024-01-11

Similar Documents

Publication Publication Date Title
US20220408111A1 (en) Image encoding method using a skip mode, and a device using the method
WO2023065891A1 (zh) 多媒体数据处理方法、装置、设备、计算机可读存储介质及计算机程序产品
CN113498605A (zh) 编码器、解码器及使用自适应环路滤波器的相应方法
JP7439841B2 (ja) ループ内フィルタリングの方法及びループ内フィルタリングの装置
CN116208773B (zh) 编码器、解码器及用于自适应环路滤波的对应方法
WO2022166462A1 (zh) 编码、解码方法和相关设备
CN116614622A (zh) 解码预测方法、装置及计算机存储介质
US20220224888A1 (en) Video encoding method, video decoding method, and related device
WO2022116824A1 (zh) 视频解码方法、视频编码方法、相关设备及存储介质
CN111277841A (zh) 视频通信系统中的错误隐藏技术
CN116962682A (zh) 用于指示条带的图像分割信息的解码器及对应方法
WO2022022299A1 (zh) 视频编解码中的运动信息列表构建方法、装置及设备
WO2022217442A1 (zh) 系数编解码方法、编码器、解码器以及计算机存储介质
WO2023065890A1 (zh) 多媒体数据处理方法、装置、计算机设备、计算机可读存储介质及计算机程序产品
TW202234886A (zh) 具有固定濾波器的自我調整迴路濾波器
WO2023125631A1 (zh) 编码块的处理方法、视频编解码方法、装置、介质及计算机设备
WO2024104503A1 (zh) 图像编解码
WO2022037458A1 (zh) 视频编解码中的运动信息列表构建方法、装置及设备
JP2024504672A (ja) 復号プロセスのエネルギー消費を表す情報をシグナリングするためのメタデータ
CN117979031A (zh) 图像滤波方法、存储介质、电子设备及产品
JP2003078915A (ja) 画像符号化装置、画像符号化方法、記録媒体及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22882502

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023560291

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 20237037580

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2022882502

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022882502

Country of ref document: EP

Effective date: 20240327

NENP Non-entry into the national phase

Ref country code: DE