WO2024067736A1 - 视频编码方法和装置、计算设备、存储介质及程序产品 - Google Patents

视频编码方法和装置、计算设备、存储介质及程序产品 Download PDF

Info

Publication number
WO2024067736A1
WO2024067736A1 PCT/CN2023/122197 CN2023122197W WO2024067736A1 WO 2024067736 A1 WO2024067736 A1 WO 2024067736A1 CN 2023122197 W CN2023122197 W CN 2023122197W WO 2024067736 A1 WO2024067736 A1 WO 2024067736A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
computing device
encoding
pixels
coding unit
Prior art date
Application number
PCT/CN2023/122197
Other languages
English (en)
French (fr)
Inventor
张涛
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2024067736A1 publication Critical patent/WO2024067736A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component

Definitions

  • the present disclosure relates to the technical field of video coding and decoding, and in particular, to a video coding method, a video coding device applying the method, and also to a computing device, a computer-readable storage medium, and a computer program product for implementing the method.
  • screen video content is widely used, including PPT presentations, word document sharing, etc.
  • Screen video is a video obtained by capturing images of the screen display content of electronic devices such as computers and mobile phones. It has many obvious differences from traditional natural videos. For example, the images in screen videos have discontinuous tones, while the images in natural videos have continuous tones. In addition, the images in screen videos have characteristics such as fewer colors in local areas, a large number of sharp boundaries, a large number of flat areas, high-contrast text, and a large number of repeated textures.
  • SCC screen compression - Screen Content Coding
  • HEVC High Efficiency Video Coding
  • the SCC standard adds tools suitable for screen compression, such as intra-frame coding mode, intra-frame block copy (IBC) coding mode, and palette coding mode. Therefore, SCC can be used to encode natural videos as well as screen videos or videos with similar characteristics.
  • the appropriate encoding mode can be selected according to the characteristics of each coding unit of the video to be encoded. Among these modes, the encoding based on the palette mode itself is more complex, and not all coding units in the video are suitable for encoding using the palette mode.
  • a video encoding method that is capable of determining whether a coding unit of a video is suitable for using a palette encoding mode, and quickly skipping the palette encoding mode when the coding unit is not suitable for the palette encoding mode, thereby improving encoding efficiency.
  • a video encoding method comprising: a computing device divides a video frame image to be encoded into a plurality of encoding units; the computing device performs the following steps for each of the plurality of encoding units: the computing device determines a color composition of the encoding unit, wherein the color composition includes the number of colors contained in the encoding unit; and the computing device determines an encoding mode of the encoding unit based on the color composition of the encoding unit.
  • a video encoding device including: a coding unit determination module, which is configured to: divide the video frame image to be encoded into multiple coding units; a coding mode determination module, which is configured to: for each of the multiple coding units: determine the color composition of the coding unit, wherein the color composition includes the number of colors contained in the coding unit; based on the color composition of the coding unit, determine the encoding mode of the coding unit.
  • a computing device which includes a processor and a memory, wherein the memory is configured to store computer-executable instructions, and the computer-executable instructions are configured to, when executed on the processor, cause the processor to perform the video encoding method according to the first aspect of the present disclosure and its various exemplary embodiments.
  • a computer-readable storage medium configured to store computer-executable instructions, and the computer-executable instructions are configured to, when executed on a processor, cause the processor to execute the video encoding method according to the first aspect of the present disclosure and its various exemplary embodiments.
  • a computer program product which includes computer executable instructions, and the computer executable instructions are configured to, when executed on a processor, cause the processor to perform the video encoding method according to the first aspect of the present disclosure and its various exemplary embodiments.
  • the computing device can predict the encoding mode for each unit to be encoded in the video frame image to be encoded, that is, the computing device can quickly and accurately select an appropriate encoding mode or skip a specific encoding mode (for example, only determine the encoding mode as the palette encoding mode in the encoding unit that is suitable for using the palette encoding mode).
  • the speed of video encoding decision-making can be improved, thereby significantly improving the video encoding efficiency.
  • FIG1 schematically shows an application scenario in which video coding is applied
  • FIG2 schematically shows a frame of an image in a video to be encoded
  • FIG3 schematically illustrates a video encoding method according to some exemplary embodiments of the present disclosure in the form of a flowchart
  • FIG4 shows more details of the video encoding method shown in FIG3 according to some exemplary embodiments of the present disclosure
  • FIG5 schematically shows a clustering process for colors included in a coding unit and a color index table generated based on the result of the clustering process
  • FIG6 shows more details of the video encoding method shown in FIG3 according to some exemplary embodiments of the present disclosure
  • FIG. 7a and FIG. 7b respectively illustrate more details of the video encoding method shown in FIG. 6 according to some exemplary embodiments of the present disclosure
  • FIG8 shows more details of the video encoding method shown in FIG3 according to some exemplary embodiments of the present disclosure
  • FIG9 schematically shows the structure of a video encoding device according to some exemplary embodiments of the present disclosure in the form of a block diagram
  • FIG. 10 schematically illustrates, in the form of a block diagram, a structure of a computing device according to some embodiments of the present disclosure.
  • Coding Unit is also called coding block.
  • the encoder will first divide the current frame of the video into multiple Coding Tree Units (CTU) according to brightness and chrominance. Then, each Coding Tree Unit will be further divided into multiple Coding Units of the same or different sizes.
  • Each Coding Unit has a certain size, such as an 8 ⁇ 8 pixel array, a 16 ⁇ 16 pixel array, or a 32 ⁇ 32 pixel array. As needed, each Coding Unit can be further divided into one or more Coding Sub-Units.
  • Palette mode is also called color index mode. It establishes a color index by assigning a fixed color value to each pixel and storing these color values in a table.
  • the table storing color values is also called a palette.
  • Palette mode is very efficient for image blocks containing only a few colors. This is because compared to the traditional prediction transformation processing of blocks, the palette mode only needs to transmit its color index for each pixel. The decoder can reconstruct the image block using the color index and the table storing color values.
  • Scene 100 may include multiple devices, for example, a first device 110, a second device 120, a third device 130, and a fourth device 140, which may communicate with each other through a network 150.
  • the first device 110 and the second device 120 may perform unidirectional data transmission.
  • the first device 110 may encode video data (for example, a video picture stream collected by the first device 110) to transmit to the second device 120 through the network 150, and the encoded video data is transmitted in the form of one or more encoded video code streams.
  • the second device 120 may receive the encoded video data from the network 150, decode the encoded video data to restore the video data, and display the video picture according to the restored video data.
  • Unidirectional data transmission is more common in applications such as media services.
  • the third device 130 and the fourth device 140 may perform bidirectional transmission of encoded video data. Bidirectional transmission may occur during, for example, video conferencing, online education, remote desktop control, etc.
  • each of the third device 130 and the fourth device 140 may encode video data (e.g., video picture streams captured by these devices) and transmit it to the other of the third device 130 and the fourth device 140 through the network 150.
  • Each of the third device 130 and the fourth device 140 may also receive encoded video data transmitted by the other of the third device 130 and the fourth device 140, decode the encoded video data to restore the video data, and may display a video image on a display device based on the restored video data.
  • first device 110, the second device 120, the third device 130, and the fourth device 140 in FIG. 1 may be a server, a personal computer, a smart phone, a laptop computer, a tablet computer, a media player, a video conferencing device, a vehicle-mounted smart terminal, or any other device capable of providing video services, and the present disclosure does not impose any restrictions on this.
  • the network 150 may be any suitable network for transmitting encoded video data between these devices, including a wired and/or wireless communication network.
  • the communication network may be any suitable communication network for exchanging data in a circuit switching and/or packet switching channel, including but not limited to a telecommunications network, a local area network, a wide area network, and/or the Internet, and the present disclosure does not impose any restrictions on this.
  • the video data transmitted between the various devices may be a video obtained by capturing an image of the screen display content of the device.
  • the image in this video may have one or more of the following characteristics, namely: having discontinuous tones; having a small number of colors in a local area; having a large number of sharp boundaries; having a large number of flat areas; containing text with high contrast; having a large number of repeated textures, etc.
  • the video data transmitted between the various devices in the application scenario 100 may also be a natural video, whose image has a continuous tone. It should be understood that the video encoding method according to the present disclosure can be applied to screen videos or videos with similar characteristics, as well as natural videos.
  • Image 200 can be, for example, content on a device screen shared with the other party through screen sharing during a video call, such as a document, a web page, etc.
  • many areas of image 200 have a small number of colors.
  • the content in the text area of image 200 is mainly
  • there are obvious contrasting boundaries in multiple regions of the image 200 such as the boundaries between different color blocks and the boundaries between each sub-block and the background.
  • I frame is an intra-frame prediction frame
  • the traditional intra-frame coding mode including DC, planar mode, and direction-based prediction mode, referred to as IPM here
  • IBC coding mode and palette coding mode can be traversed in turn, and the mode with the lowest coding cost is selected as the best coding mode for the current coding unit.
  • the coding complexity based on the palette coding mode is high, and not all coding units in the video are suitable for coding with the palette coding mode, for the coding units in the video that are not suitable for the palette coding mode, the coding using this mode can be quickly skipped, so as to increase the speed of video coding decisions and thus improve the user experience.
  • the present disclosure provides a video encoding method, which can be executed by a computing device. Specifically, the method includes: dividing a video frame image to be encoded into multiple encoding units; for each of the multiple encoding units, performing the following steps: determining the color composition of the encoding unit, wherein the color composition includes the number of colors contained in the encoding unit; based on the color composition of the encoding unit, determining the encoding mode of the encoding unit.
  • the encoding mode in determining the encoding mode of the encoding unit may include but is not limited to one or more of the following encoding modes: inter-frame encoding mode, intra-frame encoding mode, intra-frame block copy encoding mode, and palette encoding mode.
  • the video encoding method 300 includes steps 310 , 320 , 330 , 340 , 350 , 360 and 370 .
  • the video encoding method 300 starts at step 310.
  • the computing device divides the video frame image to be encoded into a plurality of encoding units.
  • the computing device divides a frame image to be encoded obtained from the video to be encoded into a plurality of encoding units to be encoded.
  • the video to be encoded includes multiple frames of images. Therefore, after obtaining the video to be encoded, the computing device will sequentially obtain a frame image from the multiple frames of images for encoding.
  • a frame image obtained from the multiple frames of images is also referred to as a video frame image to be encoded, and unless otherwise stated, the term "video frame image to be encoded" will be used hereinafter.
  • the computing device will divide the frame image into a plurality of coding tree units according to brightness and chrominance, and then each encoding tree unit will be encoded.
  • the tree unit is further divided into a plurality of coding units of the same or different sizes.
  • Each coding unit has a certain size, such as an 8 ⁇ 8 pixel array, a 16 ⁇ 16 pixel array, or a 32 ⁇ 32 pixel array, etc. It should be understood that any suitable method of dividing the video frame image to be encoded into a plurality of coding units can be applied to step 320, and the present disclosure does not limit this.
  • the computing device may include an encoding device.
  • the computing device performs steps 330 and 340 for each of the plurality of encoding units:
  • Step 330 is used to determine the color composition of the coding unit, wherein the color composition includes the number of colors contained in the coding unit.
  • the computing device determines that the color composition of the coding unit includes the following S1-S2:
  • the computing device determines the color used for each pixel in the coding unit
  • any suitable method for determining the color composition of the coding unit can be applied to step 330, and the present disclosure is not limited thereto.
  • the color number of a color described in this application refers to the number of color types.
  • step 330 may include steps 330-1 and 330-2, specifically:
  • step 330-1 the computing device performs clustering processing on the colors of each pixel in the coding unit to determine the type of color contained in the coding unit;
  • step 330 - 2 the computing device generates a color index table based on the types of colors included in the encoding unit, wherein the number of color indexes included in the color index table corresponds to the number of colors.
  • Step 330-1 is essentially that the computing device determines the color of each pixel in the coding unit, and classifies the colors of each determined pixel to determine the type of color contained in the coding unit.
  • the color coding value corresponding to each pixel in the coding unit can be determined based on the color coding used, and then the type of color contained in the coding unit can be determined based on the color coding value corresponding to each pixel in the coding unit.
  • the color coding can be any suitable color coding, for example, the YUV color coding or RGB color coding known in the field of video coding and decoding technology can be used.
  • the computing device can assign corresponding index values to various colors based on the types of colors contained in the determined coding unit to generate a color index table of the various colors.
  • the term "number of colors" refers to the number of types of colors, in other words, the number of colors. Therefore, the color index table includes The number of color indexes corresponds to the number of colors contained in the coding unit.
  • the color index table can also be called a palette, which establishes an index for each color used. Therefore, once the coding mode of the coding unit is determined to be the palette coding mode, the color index table can be conveniently used in the subsequent coding process.
  • FIG. 5 schematically shows the clustering process of the colors contained in a coding unit and the color index table generated based on the result of the clustering process.
  • the coding unit is a 4 ⁇ 4 pixel array, and the colors of each pixel therein can be determined based on the YUV color coding, and the distribution of these colors in all pixels of the coding unit is shown.
  • a color index is established for each color, namely Index0, Index1, Index2 and Index3, wherein each index corresponds to a color.
  • the color index Index0 represents the color whose Y, U, and V values are 240, 80, and 80, respectively, in the coding unit.
  • the number on each pixel corresponds to a color index value in the color index table on the right.
  • the color index table not only describes the types of colors used in the coding unit, but also records the number of colors of the colors contained in the coding unit (which corresponds to the number of color indexes included in the color index table).
  • the computing device determines the encoding mode of the encoding unit based on the color composition of the encoding unit.
  • the color composition of the encoding unit may include various information, including not only the number of colors included in the encoding unit, but also the number of pixels corresponding to each color included in the encoding unit, etc. The present disclosure is not limited to this.
  • step 340 may include steps 340-1, 340-2, 340-3, and 340-4. Specifically:
  • step 340-1 the computing device sorts the various colors contained in the coding unit from large to small according to the number of pixels corresponding to each color;
  • step 340-2 the computing device adds the number of pixels corresponding to the first M colors in the color sorting to generate the number of main color pixels, where M is a preset number of main colors, which is a positive integer and is less than the number of colors;
  • step 340-3 the computing device determines a palette encoding mode threshold based on the total number of pixels of the coding unit.
  • step 340 - 4 in response to the number of main color pixels being greater than or equal to the palette encoding mode threshold, the computing device determines the encoding mode of the encoding unit as the palette encoding mode.
  • the color composition of the coding unit also includes the respective colors contained in the coding unit.
  • the number of pixels corresponding to each color in the coding unit can be counted. For example, in the coding unit shown in FIG5 , the number of pixels corresponding to color Index0 is 6, the number of pixels corresponding to color Index1 is 6, the number of pixels corresponding to color Index2 is 2, and the number of pixels corresponding to color Index3 is 2.
  • steps 340-1, 340-2, 340-3, and 340-4 are performed.
  • Step 340-1 is used to sort the colors according to the number of pixels corresponding to the various colors that have been determined.
  • the color sorting is established in the order of the number of pixels from large to small.
  • the color sorting established in the order of the number of pixels from large to small can be: Index0: 6; Index1: 6; Index2: 2; Index3: 2.
  • it is also possible to establish a sequence in the order of small to large which will be described in detail below.
  • step 340-2 the number of pixels corresponding to the main colors that are used more is accumulated to generate the number of main color pixels.
  • the number of main color pixels essentially reflects the proportion of the main color in the colors contained in the coding unit.
  • the number of main colors M can be pre-set according to actual needs and based on experience, where M is a positive integer and is less than the number of colors contained in the coding unit.
  • M is a positive integer and is less than the number of colors contained in the coding unit.
  • the number of main colors M can be set, the number of pixels corresponding to the first M colors can be accumulated starting from the maximum value of the color sorting to generate the number of main color pixels. Still taking the situation shown in FIG. 5 as an example, if the number of primary colors M is preset to 2, then the number of pixels corresponding to the two colors Index0:6 and Index1:6 in the color sorting is added together, so that the number of primary color pixels in this example is 12.
  • Step 340-3 is used to determine the palette coding mode threshold based on the total number of pixels in the coding unit. Because the sizes of the coding units obtained by dividing the video frame image to be encoded may be different (that is, they may include pixel arrays of different sizes), the palette coding mode threshold is associated with the total number of pixels in the coding unit.
  • step 340-3 may include steps 340-3-a1 and 340-3-a2, specifically:
  • step 340-3-a1 the computing device obtains a preset threshold coefficient, wherein the threshold coefficient is associated with the total number of pixels and is less than 1;
  • step 340-3-a2 the computing device multiplies the total number of pixels by the threshold coefficient to generate the palette encoding mode threshold.
  • the threshold coefficient may be pre-set according to actual needs and based on experience, and is associated with the total number of pixels in the coding unit.
  • the threshold coefficient may be 0.7
  • the threshold coefficient may be 0.8
  • the threshold coefficient may be 0.8
  • the threshold coefficient may be 0.8.
  • the palette coding mode threshold corresponding to the coding unit can be obtained.
  • step 340-3 may include steps 340-3-b1 and 340-3-b2, specifically:
  • step 340-3-b1 the computing device obtains a preset number of pixel deviations, wherein the number of pixel deviations is associated with the total number of pixels and is less than the total number of pixels;
  • step 340-3-b2 the computing device subtracts the number of pixel deviations from the total number of pixels to generate the palette encoding mode threshold.
  • the number of pixel deviations can also be pre-set according to actual needs and based on experience, and corresponds to the total number of pixels of the coding unit. For example, when the total number of pixels of the coding unit is 64, the number of pixel deviations can be 10, when the total number of pixels of the coding unit is 256, the number of pixel deviations can be 51, and when the total number of pixels of the coding unit is 1024, the number of pixel deviations can be 204.
  • the palette coding mode threshold corresponding to the coding unit can be obtained.
  • using the pre-set number of pixel deviations and the total number of pixels of the coding unit only a subtraction operation is needed to obtain the corresponding palette coding mode threshold, which is simpler to operate and therefore more efficient.
  • the computing device determines the encoding mode of the encoding unit as the palette encoding mode in response to the number of main color pixels being greater than or equal to the palette encoding mode threshold.
  • the number of main color pixels reflects the proportion of the main color in the colors contained in the encoding unit. Therefore, when it is determined that the main color occupies a large proportion of the colors contained in the encoding unit (that is, when the requirements of the palette encoding mode threshold are met), the encoding mode of the encoding unit can be determined as the palette encoding mode.
  • the encoding method shown in Figure 6 may further include the step of: the computing device determines that the encoding unit skips the palette encoding mode in response to the number of main color pixels being less than the palette encoding mode threshold.
  • step 340 may include steps 340-1', 340-2', 340-3', 340-4', 340-5' and 340-6'. These steps can be divided into two branches, namely a first branch (including steps 340-1', 340-2') and a second branch (including steps 340-1', 340-3', 340-4', 340-5', 340-6'). The method shown in FIG8 is described below with respect to these two branches respectively.
  • step 340-1' the computing device determines whether the number of colors contained in the coding unit is less than or equal to a preset color number threshold, if so, step 340-2' is executed, if not, step 340-3' is executed.
  • step 340-2' the computing device determines that the coding unit applies the palette coding mode, that is, the computing device determines the coding mode of the coding unit as the palette coding mode in response to the number of colors being less than or equal to the preset color number threshold.
  • the color number threshold can be preset according to actual needs and based on experience. For example, the color number threshold can be set to 2, which means that only two colors are contained in the coding unit, and thus the palette coding mode is applicable.
  • the coding mode of the coding unit can be determined as the palette coding mode. It can be seen that in the method shown in this branch, it is possible to more quickly determine whether the coding unit is suitable for applying the palette coding mode based on the number of colors contained in the coding unit and using the preset color number threshold.
  • the second branch includes steps 340-1', 340-3', 340-4', 340-5' and 340-6', specifically:
  • step 340-1' the computing device determines whether the number of colors contained in the coding unit is less than or equal to a preset color number threshold; and in response to the number of colors contained in the coding unit being greater than the preset color number threshold, the computing device executes steps 340-3', 340-4', 340-5' and 340-6':
  • step 340-3' the computing device sorts the various colors contained in the coding unit from small to large according to the number of pixels corresponding to each color;
  • step 340-4' the computing device adds the number of pixels corresponding to the first N colors in the color order to generate the number of pixels of the secondary color, wherein N is a preset number of secondary colors, which is a positive integer and is less than the number of colors;
  • step 340-5' the computing device determines a palette encoding mode threshold based on the total number of pixels of the coding unit.
  • step 340 - 6 ′ in response to the number of secondary color pixels being less than or equal to the palette encoding mode threshold, the computing device determines the encoding mode of the encoding unit as the palette encoding mode.
  • Step 340-3' is similar to step 340-1 described in detail above, and will not be described again here.
  • Step 340-4' is used to establish a color sorting for these colors according to the number of pixels corresponding to the various colors determined in the coding unit.
  • the color sorting is established in the order of the number of pixels from small to large. Taking the case shown in FIG. 5 as an example, in the coding unit shown in FIG. 5, the color order established in the order of the number of pixels from small to large can be: Index3:2; Index2:2; Index0:6; Index1:6.
  • the number of pixels corresponding to the less used secondary colors is accumulated to generate the number of secondary color pixels.
  • the number of secondary color pixels substantially reflects the proportion of the secondary colors in the colors contained in the coding unit.
  • the number of secondary colors N can be preset according to actual needs and based on experience, where N is a positive integer and is less than the number of colors contained in the coding unit.
  • the number of secondary colors N can be set, the number of pixels corresponding to the first N colors can be accumulated starting from the minimum value of the color ordering to generate the number of secondary color pixels. Still taking the case shown in FIG. 5 as an example, if the number of secondary colors N is preset to 2, then the two values of Index3:2 and Index2:2 are taken from the color ordering and added, so that the number of secondary color pixels in this example is 4.
  • Step 340-5' is used to determine the palette coding mode threshold based on the total number of pixels in the coding unit. Similar to step 340-3 described in detail above, step 340-5' can also apply the steps described in Figures 7a and 7b to determine the palette coding mode threshold, which will not be repeated here. However, it should be understood that because the secondary colors contained in the coding unit are considered in the second branch, both the threshold coefficient and the number of pixel deviations are different from the situation where the primary colors contained in the coding unit are considered (as shown in Figure 6).
  • the threshold coefficient when the palette encoding mode threshold is determined based on the threshold coefficient, taking into account the secondary colors contained in the coding unit, when the total number of pixels of the coding unit is 64, the threshold coefficient can be 0.3, when the total number of pixels of the coding unit is 256, the threshold coefficient can be 0.2, and when the total number of pixels of the coding unit is 1024, the threshold coefficient can be 0.2; when based on the number of pixel deviations, when the total number of pixels of the coding unit is 64, the number of pixel deviations can be 54, when the total number of pixels of the coding unit is 256, the number of pixel deviations can be 205, and when the total number of pixels of the coding unit is 1024, the number of pixel deviations can be 820.
  • the computing device determines the encoding mode of the encoding unit as the palette encoding mode in response to the number of secondary color pixels being less than or equal to the palette encoding mode threshold.
  • the number of secondary color pixels reflects the proportion of secondary colors in the colors contained in the encoding unit. Therefore, when it is determined that the secondary colors occupy a small proportion of the colors contained in the encoding unit (that is, when the requirements of the palette encoding mode threshold are met), the encoding mode of the encoding unit can be determined as the palette encoding mode.
  • the encoding method shown in FIG8 may further include the steps of: calculating In response to the number of secondary color pixels being greater than the palette encoding mode threshold, the device determines that the encoding unit skips the palette encoding mode.
  • steps 340-1', 340-2' can also be used in combination with the method shown in Figure 6. That is, it is possible to first determine whether the coding unit is suitable for applying the palette coding mode based on the number of colors contained in the coding unit and using a preset color number threshold; and when the number of colors contained in the coding unit is greater than the color number threshold, steps 340-1, 340-2, 340-3 and 340-4 of the method shown in Figure 6 are executed. Further, the method in this combination may also include the step of: the computing device determines that the coding unit skips the palette coding mode in response to the number of main color pixels being less than the palette coding mode threshold.
  • the video encoding method 300 determines at step 350 whether the encoding mode has been determined for all encoding units. If the encoding mode has been determined for all encoding units, the video encoding method 300 ends at step 370. If the encoding mode has not been determined for all encoding units, the video encoding method 300 will repeat steps 330 and 340 for the next encoding unit (step 360) to determine the encoding mode for the encoding unit. The video encoding method 300 will loop through steps 330 to 360 until the encoding mode has been determined for all encoding units, and then the video encoding method 300 ends at step 370.
  • each coding unit in the video frame image to be encoded can be predicted, that is, the appropriate coding mode can be quickly and accurately selected or a specific coding mode can be skipped according to the color composition of the unit to be encoded (for example, only in the coding unit that is suitable for using the palette coding mode, its coding mode is determined as the palette coding mode).
  • the speed of video coding decisions can be improved, thereby significantly improving the video coding efficiency.
  • the structure of a video encoding device is schematically shown in the form of a block diagram.
  • the video encoding device 400 can be applied to the various devices shown in FIG. 1 , and can be applied to the various scenarios described in the present disclosure.
  • the video encoding device 400 includes: a coding unit determination module 410 and a coding mode determination module 420.
  • the coding unit determination module 410 is configured to: divide the video frame image to be encoded into multiple coding units.
  • the coding mode determination module 420 is configured to: for each of the multiple coding units: determine the color composition of the coding unit, wherein the color composition includes the number of colors contained in the coding unit; and, based on the color composition of the coding unit, determine the coding mode of the coding unit.
  • the above modules involve the operations of steps 320, 330, 340, 350 and 360 described above with respect to FIG. 3, and thus will not be described in detail here.
  • Each module described above with respect to FIG. 9 may be implemented in hardware or in hardware combined with software and/or firmware.
  • these modules may be implemented as computer executable code/instructions configured to be executed in one or more processors and stored in a computer readable storage medium.
  • these modules may be implemented as hardware logic/circuits.
  • one or more of these modules may be implemented together in a system on a chip (SoC).
  • SoC system on a chip
  • the SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and/or one or more components in other circuits), and may optionally execute the received program code and/or include embedded firmware to perform the functions.
  • a processor e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.
  • DSP digital signal processor
  • FIG. 10 a block diagram schematically illustrates a structure of a computing device 500 according to some embodiments of the present disclosure.
  • the computing device 500 can be used in various application scenarios described in the present disclosure.
  • the computing device 500 may include at least one processor 502, memory 504, communication interface(s) 506, a display device 508, other input/output (I/O) devices 510, and one or more mass storage devices 512 that can communicate with each other, such as via a system bus 514 or other appropriate connection.
  • processor 502 memory 504, communication interface(s) 506, a display device 508, other input/output (I/O) devices 510, and one or more mass storage devices 512 that can communicate with each other, such as via a system bus 514 or other appropriate connection.
  • the processor 502 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores.
  • the processor 502 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operating instructions.
  • the processor 502 may be configured to retrieve and execute computer-readable instructions stored in the memory 504, mass storage device 512, or other computer-readable media, such as program code of an operating system 516, program code of an application program 518, program code of other programs 520, and the like.
  • the memory 504 and the mass storage device 512 are examples of computer-readable storage media for storing instructions that can be executed by the processor 502 to implement the various functions described above.
  • the memory 504 can generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.).
  • the mass storage device 512 can generally include a hard drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc.
  • the memory 504 and the mass storage device 512 can all be collectively referred to herein as computer-readable memory or computer-readable storage media, and can be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer-executable code, which can be executed by the processor 502 as a specific machine configured to implement the operations and functions described in the exemplary embodiments of the present disclosure.
  • a plurality of program modules may be stored on the mass storage device 512.
  • These program modules include an operating system 516, one or more application programs 518, other programs 520, and program data 522, and they may be executed by the processor 502.
  • Examples of such application programs or program modules may include, for example, computer programs for implementing the following components/functions: Sequential logic (eg, computer executable code or instructions): an encoding unit determination module 410 and an encoding mode determination module 420.
  • the coding unit determination module 410 and the coding mode determination module 420 or a portion thereof may be implemented using any form of computer-readable media accessible by the computing device 500.
  • the term "computer-readable media” means at least two types of computer-readable media, namely computer storage media and communication media.
  • Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.
  • Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, tapes, disk storage devices or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computing device.
  • communication media can specifically implement computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism.
  • the computer storage media defined in the present disclosure do not include communication media.
  • the computing device 500 may also include one or more communication interfaces 506 for exchanging data with other devices, such as through a network, direct connection, etc.
  • the communication interfaces 506 may facilitate communications within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc.
  • the communication interfaces 506 may also provide for communications with external storage devices (not shown), such as in a storage array, network attached storage, storage area network, etc.
  • the computing device 500 may also include a display device 508 such as a display for displaying information and images.
  • Other I/O devices 510 may be devices that receive various inputs from a target object and provide various outputs to the target object, including but not limited to a touch input device, a gesture input device, a camera, a keyboard, a remote control, a mouse, a printer, an audio input/output device, and the like.
  • the present disclosure also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium.
  • a processor of a computing device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computing device executes the method for determining an associated application or the method for determining recommended content provided in the above various optional implementations.
  • the description of the terms “one embodiment”, “some embodiments”, “example”, “specific example”, or “some examples” etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure.
  • the schematic representations of the above terms do not necessarily refer to the same embodiment or example.
  • the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
  • those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
  • modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • module routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • module generally refer to software, firmware, hardware or a combination thereof.
  • the features of the techniques described herein are platform-independent, meaning that these techniques can be implemented on a variety of computing platforms with a variety of processors.
  • a plurality of steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented by hardware, it may be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.
  • data such as videos to be encoded and images to be encoded may be involved.
  • data such as videos to be encoded and images to be encoded may be involved.
  • user permission or consent is required, and the collection, use and processing of the relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Abstract

提供了一种视频编码方法，其包括：计算设备将待编码视频帧图像划分为多个编码单元；计算设备针对所述多个编码单元中的每一个编码单元，执行下述步骤：计算设备确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；计算设备基于所述编码单元的颜色构成，确定所述编码单元的编码模式。

Description

视频编码方法和装置、计算设备、存储介质及程序产品
本申请基于申请号为：202211197925.0，申请日为2022年09月29日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本公开涉及视频编解码技术领域，具体地，涉及一种视频编码方法，以及应用该方法的视频编码装置，并且还涉及用于实现该方法的计算设备、计算机可读存储介质以及计算机程序产品。
背景技术
随着计算机应用的发展，诸如视频会议、在线教育、远程桌面等应用场景变得越来越普遍。在这些应用场景中，大量地使用了屏幕视频内容形式，包括PPT演示、word文档共享等。屏幕视频是通过截取诸如计算机、手机等电子设备的屏幕显示内容的图像而得到的视频，它与传统的自然视频具有很多明显的区别。例如，屏幕视频中的图像具有非连续色调，而自然视频中的图像具有连续色调。此外，屏幕视频中的图像的局部区域中的颜色数较少、存在大量尖锐的边界、存在大量平坦区域、包含对比度高的文字、存在大量重复的纹理等特性。
由于屏幕视频的固有特性，国际标准组织制定了屏幕压缩的标准——屏幕内容编码(Screen Content Coding，即SCC)，SCC是基于高效视频编码标准(High Efficiency Video Coding，即HEVC)的扩展版本。相比HEVC，SCC标准增加了适合屏幕压缩的工具，例如，帧内编码模式、帧内块拷贝(Intra Block Copy，即IBC)编码模式、调色板(palette)编码模式。因此，SCC既能够用于对自然视频编码，也能够用于屏幕视频或者具有类似特征的视频进行编码。在编码过程中，可以根据待编码的视频的各编码单元的特点来选择合适的编码模式。在这些模式中，基于调色板模式的编码本身复杂度较高，而且并非视频中的所有编码单元都适合用调色板模式进行编码。
因此，期望提供一种视频编码方法，其能够针对视频的编码单元确定是否适合使用调色板编码模式，并且在编码单元不适合调色板编码模式时快速跳过该模式，从而提高编码效率。
发明内容
根据本公开的第一个方面，提供了一种视频编码方法，包括：计算设备将待编码视频帧图像划分为多个编码单元；所述计算设备针对所述多个编码单元中的每一个编码单元，执行下述步骤：所述计算设备确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；以及，所述计算设备基于所述编码单元的颜色构成，确定所述编码单元的编码模式。
根据本公开的第二个方面，提供了一种视频编码装置，包括：编码单元确定模块，其被配置成：将待编码视频帧图像划分为多个编码单元；编码模式确定模块，其被配置成：针对所述多个编码单元中的每一个编码单元：确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；基于所述编码单元的颜色构成，确定所述编码单元的编码模式。
根据本公开的第三个方面，提供了一种计算设备，其包括处理器和存储器，所述存储器被配置成存储计算机可执行指令，所述计算机可执行指令被配置成当在所述处理器上执行时，使所述处理器执行根据本公开的第一个方面及其各示例性实施例所述的视频编码方法。
根据本公开的第四个方面，提供了一种计算机可读存储介质，其被配置成存储计算机可执行指令，所述计算机可执行指令被配置成当在处理器上执行时，使所述处理器执行根据本公开的第一个方面及其各示例性实施例所述的视频编码方法。
根据本公开的第五个方面，提供了一种计算机程序产品，其包括计算机可执行指令，所述计算机可执行指令被配置成当在处理器上执行时，使所述处理器执行根据本公开的第一个方面及其各示例性实施例所述的视频编码方法。
在根据本公开一些实施例的视频编码方法和装置中，计算设备可以针对待编码视频帧图像中的各个待编码单元进行编码模式预判，即计算设备根据待编码单元的颜色构成情况快速准确地选择适当的编码模式或跳过特定编码模式(例如只在适合使用调色板编码模式的编码单元中将其编码模式确定为调色板编码模式)，由此，能够提高视频编码决策的速度，从而显著提升了视频编码效率。
附图说明
下面将结合附图对本公开的具体实施例进行详细的描述，以便能够对本公开的更多细节、特征和优点具有更加充分的认识和理解；在附图中：
图1示意性地示出了一种应用了视频编码的应用场景；
图2示意性地示出了待编码视频中的一帧图像；
图3以流程图的形式示意性地示出了根据本公开的一些示例性实施例的视频编码方法；
图4根据本公开的一些示例性实施例，示出了图3所示的视频编码方法的更多细节；
图5示意性地示出了对一个编码单元包含的颜色的聚类处理以及基于该聚类处理的结果生成的颜色索引表；
图6根据本公开的一些示例性实施例，示出了图3所示的视频编码方法的更多细节；
图7a和图7b根据本公开的一些示例性实施例，分别示出了图6所示的视频编码方法的更多细节；
图8根据本公开的一些示例性实施例，示出了图3所示的视频编码方法的更多细节；
图9其以框图的形式示意性地示出了根据本公开的一些示例性实施例的视频编码装置的结构；
图10以框图的形式示意性地示出了根据本公开的一些实施例的一种计算设备的结构。
应理解的是，附图中显示的内容都仅仅是示意性的，因此，其不必按照比例进行绘制。此外，在全部附图中，相同或相似的特征由相同或相似的附图标记指示。
具体实施方式
下文中的描述提供了本公开的各示例性实施例的具体细节，以使本领域的技术人员能够充分理解以及实施根据本公开的各示例性实施例描述的技术方案。
编码单元：编码单元(Coding Unit，即CU)也被称作编码块。在获取到待编码的视频之后，编码器会首先根据亮度和色度将视频的当前帧划分成多个编码树单元(Coding Tree Unit，即CTU)，然后，每个编码树单元会被进一步划分为尺寸相同或不同的多个编码单元。每个编码单元都具有一定尺寸，例如为8×8的像素阵列、16×16的像素阵列或者32×32的像素阵列等等。根据需要，每个编码单元还可以被进一步划分为一个或多个编码子单元。
调色板模式：调色板模式也被称为是颜色索引模式，其通过给每一个像素分配一个固定的颜色值，并将这些颜色值储存在表中，从而建立起颜色索引。该存储有颜色值的表也被称作调色板。调色板模式对于只含有少数几种颜色的图像块非常高效，这是因为相比于传统的对块进行预测变换处理，调色板模式只需要对每个像素传输其颜色索引，解码器利用颜色索引和存储有颜色值的表就能够重建该图像块。
参见图1，示意性地示出了一种应用了视频编码的应用场景。如图1所示，应用场 景100可以包括多个设备，例如，可以包括第一设备110、第二设备120、第三设备130以及第四设备140，这些设备可以通过网络150彼此通信。在应用场景100中，第一设备110和第二设备120可以执行单向数据传输。例如，第一设备110可以对视频数据(例如，由第一设备110采集的视频图片流)进行编码，以通过网络150传输到第二设备120，已编码的视频数据以一个或多个已编码视频码流的形式传输。第二设备120则可以从网络150接收已编码的视频数据，对已编码的视频数据进行解码以恢复视频数据，并根据恢复的视频数据显示视频图片。单向数据传输在媒体服务等应用中是较常见的。此外，应用场景100中，第三设备130和第四设备140可以执行已编码视频数据的双向传输。双向传输可以在例如视频会议、在线教育、远程桌面控制等期间发生。对于双向数据传输，第三设备130和第四设备140中的每个设备可以对视频数据(例如，由这些设备采集的视频图片流)进行编码，并通过网络150传输到第三设备130和第四设备140中的另一设备。第三设备130和第四设备140中的每个设备还可接收由第三设备130和第四设备140中的另一设备传输的已编码视频数据，对已编码视频数据进行解码以恢复视频数据，并且可根据恢复的视频数据在显示装置上显示视频图像。
应理解的是，图1中的第一设备110、第二设备120、第三设备130以及第四设备140可以是服务器、个人计算机、智能电话、膝上型计算机、平板电脑、媒体播放器、视频会议设备、车载智能终端，或者任何其他能够提供视频服务的设备，本公开对此不作任何限制。网络150可以是在这些设备之间传送已编码视频数据的任何合适的网络，包括有线和/或无线通信网络。通信网络可以是在电路交换和/或分组交换信道中交换数据的任何合适的通信网络，包括但不限于电信网络、局域网、广域网和/或因特网，本公开对此也不作任何限制。
在应用场景100中，各个设备之间传输的视频数据可以是对设备的屏幕显示内容进行截取的图像而得到的视频。这种视频中的图像可以具有下述特性中的一种或多种，即：具有非连续色调；局部区域中的颜色数较少；存在大量尖锐的边界；存在大量平坦区域；包含对比度高的文字；存在大量重复的纹理等等。当然，应用场景100中的各个设备之间传输的视频数据也可以是自然视频，其图像具有连续色调。应理解的是，根据本公开的视频编码方法既可应用于屏幕视频或具有类似特征的视频，也可应用于自然视频。
参见图2，其示意性地示出了待编码视频中的一帧图像。图像200例如可以是视频通话过程中通过屏幕共享来向对方共享设备屏幕上的内容，例如文档、网页等等。从图2中可见，图像200的许多区域中的颜色数较少，例如，图像200的文字区域中的内容主 要包含两种颜色；此外，图像200的多个区域中存在对比明显的边界，例如，不同色块之间的边界，以及各个子图块与背景之间的边界等等。同时，图像200中也存在包含较多颜色的区域，例如各个子图块中显示的内容。因此，在对图像200进行视频编码时，针对其中对应于不同区域的编码单元，应当选择合适的编码方式，以获得最佳的视频编码性能。
作为非限制性示例，在相关技术中，在屏幕视频或具有类似特征的视频进行编码的过程中，当进行I帧编码时(I帧为帧内预测帧)，可以依次对传统的帧内编码模式(包括DC，planar模式，以及基于方向的预测模式，这里简称为IPM)、IBC编码模式以及调色板编码模式进行遍历，从中选择编码代价最小的模式作为当前编码单元的最佳编码模式。在这些编码模式中，因为基于调色板编码模式的编码的复杂度较高，而且并非视频中的所有编码单元都适合用调色板编码模式进行编码，所以，针对视频中不适合使用调色板编码模式的编码单元，可以快速跳过采用该模式的编码，以便能够提高视频编码决策的速度，从而改善用户的体验。
在本申请中的一些实施例中，本公开提供一种视频编码方法，该方法可以由计算设备执行，具体地，该方法包括：将待编码视频帧图像划分为多个编码单元；针对所述多个编码单元中的每一个编码单元，执行下述步骤：确定所述编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；基于所述编码单元的颜色构成，确定所述编码单元的编码模式。其中，确定所述编码单元的编码模式中的编码模式可以包括但不限于以下多种编码模式中的一种或多种：帧间编码模式、帧内编码模式、帧内块拷贝编码模式、调色板编码模式。
参见图3，其以流程图的形式示意性地示出了根据本公开的一些示例性实施例的视频编码方法。如图3所示，视频编码方法300包括步骤310、320、330、340、350、360和370。
具体而言，视频编码方法300在步骤310处开始。在步骤320中，计算设备将待编码视频帧图像划分为多个编码单元，换言之，计算设备将从待编码视频中获取的待编码的一帧图像划分为待编码的多个编码单元。待编码视频包括多帧图像，因此，在获取到待编码视频之后，计算设备会顺序地从所述多帧图像中获取一帧图像进行编码处理。应理解的是，在本公开中，从所述多帧图像中获取的一帧图像也被称为待编码视频帧图像，并且如无相反的声明，下文中将会一直使用术语“待编码视频帧图像”。针对待编码视频帧图像，计算设备会根据亮度和色度将该帧图像划分为多个编码树单元，然后每个编码 树单元会被进一步划分为尺寸相同或不同的多个编码单元。每个编码单元都具有一定尺寸，例如为8×8的像素阵列、16×16的像素阵列或者32×32的像素阵列等等。应理解的是，将待编码视频帧图像划分为多个编码单元的任何合适的方式都可以被应用于步骤320，本公开对此不作限制。其中，计算设备可以包括编码装置。
如图3所示，在视频编码方法300中，所述计算设备针对所述多个编码单元中的每一个编码单元，执行步骤330和340：
步骤330用于确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数。换言之，在前述330中，计算设备确定编码单元的颜色构成包括以下S1-S2：
S1、计算设备针对编码单元中每一个像素确定其用到的颜色；
S2、基于各个像素的颜色，确定该编码单元的颜色构成。
应理解的是，用于确定编码单元的颜色构成的任何合适的方法都可以应用于步骤330，本公开对此不作限制。
本申请所述的颜色的颜色数指颜色种类的数量。
参见图4，其根据本公开的一些示例性实施例，进一步示出了图3所示的视频编码方法300的步骤330的细节。如图4所示，根据本公开的一些示例性实施例，步骤330可以包括步骤330-1和330-2，具体而言：
在步骤330-1，计算设备对所述编码单元中各个像素的颜色进行聚类处理，以确定所述编码单元包含的颜色的种类；
在步骤330-2，计算设备基于所述编码单元包含的颜色的种类生成颜色索引表，其中，所述颜色索引表包括的颜色索引的数量对应于所述颜色数。
步骤330-1实质上是计算设备确定编码单元中各个像素的颜色，并且对所确定的各个像素的颜色进行分类，以确定编码单元包含的颜色的种类。在本公开的一些示例性实施例中，可以基于所用的颜色编码，确定所述编码单元中各个像素各自对应的颜色编码值，然后基于所述编码单元中各个像素各自对应的颜色编码值，确定所述编码单元包含的颜色的种类。应理解的是，颜色编码可以是任何合适的颜色编码，例如，可以采用视频编解码技术领域中已知的YUV颜色编码或者RGB颜色编码。
步骤330-2中计算设备基于所确定的编码单元包含的颜色的种类，可以给各种颜色分别赋予对应的索引值，以生成所述各种颜色的颜色索引表。应理解的是，在本公开中，术语“颜色数”是指颜色的种类的数量，换言之，具有几种颜色。因此，颜色索引表包括 的颜色索引的数量对应于编码单元包含的颜色的颜色数。在本公开中，颜色索引表也可以被称作调色板，其针对所用到的每一种颜色都建立了索引。因此，一旦编码单元的编码模式被确定为调色板编码模式，那么颜色索引表就可以很方便地被用于后续的编码环节中。
参见图5，其示意性地示出了对一个编码单元包含的颜色的聚类处理以及基于该聚类处理的结果生成的颜色索引表。如图5中的左侧视图所示，该编码单元是一个4×4的像素阵列，基于YUV颜色编码可以确定其中各个像素具有的颜色，并且显示了这些颜色在编码单元的全部像素中的分布情况。在图5的右侧视图所示的表中，针对每一种颜色建立了颜色索引，即Index0、Index1、Index2和Index3，其中，每一个索引对应于一种颜色。例如，颜色索引Index0表示该编码单元中Y、U、V值分别为240、80、80的颜色。从图中的左侧视图可见，每一个像素上的数字对应于右侧的颜色索引表中的一个颜色索引值。由此可见，颜色索引表不仅描述了该编码单元中用到的颜色的种类，而且还记录了该编码单元包含的颜色的颜色数(其与颜色索引表包括的颜色索引的数量对应)。
继续参见图3，视频编码方法300在步骤340处，计算设备基于所述编码单元的颜色构成，确定所述编码单元的编码模式。应理解的是，编码单元的颜色构成可以包括各种信息，不仅可以包括编码单元包含的颜色的颜色数，还可以包括编码单元包含的各种颜色各自对应的像素数量等等。本公开对此不作限制。
参见图6，其根据本公开的一些示例性实施例，进一步示出了图3所示的视频编码方法的细节。如图6所示，根据本公开的一些示例性实施例，步骤340可以包括步骤340-1、340-2、340-3和340-4，具体而言：
在步骤340-1，计算设备将所述编码单元包含的各种颜色按照各自对应的像素数量从大到小地进行颜色排序；
在步骤340-2，计算设备将所述颜色排序中前M个颜色各自对应的像素数量相加，以生成主要颜色像素数量，其中，M为预先设定的主要颜色数，其为正整数并且小于所述颜色数；
在步骤340-3，计算设备基于所述编码单元的像素总数确定调色板编码模式阈值；以及
在步骤340-4，计算设备响应于所述主要颜色像素数量大于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。
在图6所示的方法中，编码单元的颜色构成还包括该编码单元包含的各种颜色各自 对应的像素数量，也就是说，可以分别统计编码单元包含的各种颜色各自对应的像素数量。以图5所示的情形为例，在图5所示的编码单元中，颜色Index0对应的像素数量为6，颜色Index1对应的像素数量为6，颜色Index2对应的像素数量为2，颜色Index3对应的像素数量为2。在确定了编码单元包含的各种颜色各自对应的像素数量的情况下，执行步骤340-1、340-2、340-3和340-4。
步骤340-1用于按照已经确定的各种颜色对应的像素数量，对这些颜色进行颜色排序。在图6所示的示例性实施例中，颜色排序是按照像素数量的从大到小的顺序建立的。仍然以图5所示的情形为例，在图5所示的编码单元中，按照像素数量的从大到小的顺序建立的颜色排序可以是：Index0:6；Index1:6；Index2:2；Index3:2。然而，按照从小到大的顺序建立序列也是可能的，这一点将在下文中再详细描述。在步骤340-2，对使用较多的主要颜色对应的像素数量进行累加，从而生成主要颜色像素数量。主要颜色像素数量实质上反映了主要颜色在编码单元包含的颜色中的占比。作为非限制性示例，可以根据实际需要并且基于经验来预先设定主要颜色数M，其中，M为正整数并且小于编码单元包含的颜色的颜色数。当主要颜色数M被设定后，可以从颜色排序的最大值开始取前M个颜色对应的像素数量进行累加，从而生成主要颜色像素数量。还是以图5所示的情形为例，如果主要颜色数M被预先设置为2，那么从颜色排序中取Index0:6和Index1:6两个颜色对应的像素数量相加，从而得到该示例中的主要颜色像素数量为12。步骤340-3用来基于编码单元的像素总数确定调色板编码模式阈值。因为从待编码视频帧图像中划分得到的编码单元的尺寸会存在不同(也就是说，可能包括不同尺寸大小的像素阵列)，所以调色板编码模式阈值与编码单元的像素总数相关联。
参见图7a和图7b，其根据本公开的示例性实施例，分别进一步示出了图6所示的视频编码方法的细节。如图7a所示，根据本公开的一些示例性实施例，步骤340-3可以包括步骤340-3-a1和340-3-a2，具体而言：
在步骤340-3-a1，计算设备获取预先设定的阈值系数，其中，所述阈值系数与所述像素总数相关联并且小于1；
在步骤340-3-a2，计算设备将所述像素总数乘以所述阈值系数，以生成所述调色板编码模式阈值。
应理解的是，阈值系数可以是根据实际需要并且基于经验而预先设定的，并且其与编码单元的像素总数相关联。作为非限制示例，当编码单元的像素总数为64个时，阈值系数可以为0.7，当编码单元的像素总数为256个时，阈值系数可以为0.8，并且当编码 单元的像素总数为1024个时，阈值系数可以为0.8。由此，基于各种编码单元的像素总数和相关联的预先设定的阈值系数，就能够获得与该编码单元对应的调色板编码模式阈值。
此外，如7b所示，根据本公开的一些示例性实施例，步骤340-3可以包括步骤340-3-b1和340-3-b2，具体而言：
在步骤340-3-b1，计算设备获取预先设定的像素偏差数量，其中，所述像素偏差数量与所述像素总数相关联并且小于所述像素总数；
在步骤340-3-b2，计算设备将所述像素总数减去所述像素偏差数量，以生成所述调色板编码模式阈值。
同样应理解的是，像素偏差数量也可以是根据实际需要并且基于经验而预先设定的，并且与编码单元的像素总数对应。例如，当编码单元的像素总数为64个时，像素偏差数量可以为10，当编码单元的像素总数为256个时，像素偏差数量可以为51，并且当编码单元的像素总数为1024个时，像素偏差数量可以为204。由此，基于各种编码单元的像素总数和相关联的预先设定的像素偏差数量，就能够获得与该编码单元对应的调色板编码模式阈值。此外，在本示例中，利用预先设定的像素偏差数量和编码单元的像素总数，只需要使用减法运算就能够得到对应的调色板编码模式阈值，其运算更加简单，因此效率更高。
继续参见图6，在步骤340-4中，计算设备响应于所述主要颜色像素数量大于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。主要颜色像素数量反映了主要颜色在编码单元包含的颜色中的占比。因此，当确定编码单元中包含的颜色中主要颜色占有较大比例时(即，满足调色板编码模式阈值的要求时)，那么可以将编码单元的编码模式确定为调色板编码模式。
由此可见，利用图3、图4、图6、图7a、图7b所示的方法，可以针对视频的待编码视频帧图像中的各个编码单元进行预判，并且只在适合使用调色板编码模式的编码单元中将其编码模式确定为调色板编码模式，由此，能够提高视频编码决策的速度，从而改善用户的体验。
此外，在根据本公开的一些示例性实施例中，图6所示的编码方法还可以包括步骤：计算设备响应于所述主要颜色像素数量小于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
参见图8，其根据本公开的一些示例性实施例，进一步示出了图3所示的视频编码 方法的细节。如图8所示，根据本公开的一些示例性实施例，步骤340可以包括步骤340-1'、340-2'、340-3'、340-4'、340-5'和340-6'。这些步骤可以分为两个分支，即第一分支(包括步骤340-1'、340-2')和第二分支(包括步骤340-1'、340-3'、340-4'、340-5'、340-6')。下面分别就这两个分支来描述图8所示的方法。
在第一分支中：在步骤340-1'，计算设备判断所述编码单元中包含的颜色的颜色数是否小于或等于预先设定的颜色数阈值，若是，则执行步骤340-2'，若否，则执行步骤340-3'。在步骤340-2'，计算设备确定所述编码单元应用调色板编码模式，即计算设备响应于所述颜色数小于或等于预先设定的颜色数阈值，将所述编码单元的编码模式确定为调色板编码模式。颜色数阈值可以根据实际需要并且基于经验而预先设定，例如，颜色数阈值可以被设置为2，这意味着编码单元中仅包含两种颜色，因而适用调色板编码模式。因此，当确定编码单元中包含的颜色的颜色数小于或等于预先设定的颜色数阈值时，可以将编码单元的编码模式确定为调色板编码模式。由此可见，在该分支所示的方法中，可以基于编码单元包含的颜色的颜色数，利用预先设定的颜色数阈值，更加快捷地确定该编码单元是否适于应用调色板编码模式。
第二分支包括步骤340-1'、340-3'、340-4'、340-5'和340-6'，具体而言：
在步骤340-1'，计算设备判断所述编码单元中包含的颜色的颜色数是否小于或等于预先设定的颜色数阈值；并且计算设备响应于所述编码单元中包含的颜色的颜色数大于预先设定的颜色数阈值时，执行步骤340-3'、340-4'、340-5'和340-6'：
在步骤340-3'，计算设备将所述编码单元包含的各种颜色按照各自对应的像素数量从小到大地进行颜色排序；
在步骤340-4'，计算设备将所述颜色排序中前N个颜色各自对应的像素数量相加，以生成次要颜色像素数量，其中，N为预先设定的次要颜色数，其为正整数并且小于所述颜色数；
在步骤340-5'，计算设备基于所述编码单元的像素总数确定调色板编码模式阈值；以及
在步骤340-6'，计算设备响应于所述次要颜色像素数量小于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。
步骤340-3'与前面已经详细描述的步骤340-1类似，在此不再赘述。步骤340-4'用于按照编码单元中已经确定的各种颜色对应的像素数量，为这些颜色建立颜色排序，但是在本示例性实施例中，该颜色排序按照像素数量的从小到大的顺序来建立。以图5所 示的情形为例，在图5所示的编码单元中，按照像素数量的从小到大的顺序建立的颜色排序可以是：Index3:2；Index2:2；Index0:6；Index1:6。在步骤340-4'，对使用较少的次要颜色对应的像素数量进行累加，以生成次要颜色像素数量。次要颜色像素数量实质上反映了次要颜色在编码单元包含的颜色中的占比。作为非限制性示例，可以根据实际需要并且基于经验来预先设定次要颜色数N，其中，N为正整数并且小于编码单元包含的颜色的颜色数。当次要颜色数N被设定后，可以从颜色排序的最小值开始依次取前N个颜色对应的像素数量进行累加，从而生成次要颜色像素数量。还是以图5所示的情形为例，如果次要颜色数N被预先设置为2，那么从颜色排序中取Index3:2和Index2:2两个值相加，从而得到该示例中的次要颜色像素数量为4。
步骤340-5'用来基于编码单元的像素总数确定调色板编码模式阈值。与前面已经详细描述的步骤340-3类似，步骤340-5'同样可以应用图7a和图7b中描述的步骤来确定调色板编码模式阈值，在此不再赘述。但是，应当理解的是，因为在第二分支中是针对编码单元中包含的次要颜色进行考虑，所以无论是阈值系数还是像素偏差数量，其与针对编码单元中包含的主要颜色进行考虑的情形(如图6所示的情形)会有所区别。作为非限制性，在针对编码单元包含的次要颜色进行考虑的情况下，当基于阈值系数来确定调色板编码模式阈值时，当编码单元的像素总数为64个时，阈值系数可以为0.3，当编码单元的像素总数为256个时，阈值系数可以为0.2，并且当编码单元的像素总数为1024个时，阈值系数可以为0.2；当基于像素偏差数量时，当编码单元的像素总数为64个时，像素偏差数量可以为54，当编码单元的像素总数为256个时，像素偏差数量可以为205，并且当编码单元的像素总数为1024个时，像素偏差数量可以为820。
在步骤340-6'，计算设备响应于所述次要颜色像素数量小于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。次要颜色像素数量反映了次要颜色在编码单元包含的颜色中的占比。因此，当确定编码单元包含的颜色中次要颜色占有较小比例时(即，满足调色板编码模式阈值的要求时)，那么可以将编码单元的编码模式确定为调色板编码模式。
由此可见，利用图3、图4、图7a、图7b、图8所示的方法，可以针对视频的待编码视频帧图像中的各个编码单元进行预判，并且只在适合使用调色板编码模式的编码单元中将其编码模式确定为调色板编码模式，由此，能够提高视频编码决策的速度，从而改善用户的体验。
在根据本公开的一些示例性实施例中，图8所示的编码方法还可以包括步骤：计算 设备响应于所述次要颜色像素数量大于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
此外，还应理解的是，上述第一分支中包括的步骤(即，步骤340-1'、340-2')也可以与图6所示的方法结合在一起使用。也就是说，可以先基于编码单元包含的颜色的颜色数，利用预先设定的颜色数阈值，确定该编码单元是否适于应用调色板编码模式；并且，当编码单元包含的颜色的颜色数大于颜色数阈值阈值时，执行图6所示的方法的步骤340-1、340-2、340-3和340-4。进一步地，在这种组合方式下的方法还可以包括步骤：计算设备响应于所述主要颜色像素数量小于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
继续参见图3，视频编码方法300在步骤350处判断是否已经为所有编码单元确定了编码模式。如果已经为所有编码单元确定了编码模式，那么视频编码方法300在步骤370处结束。如果还没有为所有编码单元确定编码模式，那么视频编码方法300将对下一个编码单元(步骤360)重复执行步骤330、340，以便为该编码单元确定编码模式。视频编码方法300将循环执行步骤330至360，直到已经为所有编码单元确定了编码模式，然后视频编码方法300在步骤370处结束。
由此可见，利用图3所示的方法，可以针对视频的待编码视频帧图像中的各个编码单元进行预判，即根据待编码单元的颜色构成情况快速准确地选择适当的编码模式或跳过特定编码模式(例如只在适合使用调色板编码模式的编码单元中将其编码模式确定为调色板编码模式)，由此，能够提高视频编码决策的速度，从而显著提升了视频编码效率。
参见图9，其以框图的形式示意性地示出了根据本公开的一些示例性实施例的视频编码装置的结构。视频编码装置400可以被应用于图1所示的各种设备中，并且可以应用于本公开描述的各种场景。如图9所示，视频编码装置400包括：编码单元确定模块410和编码模式确定模块420。
编码单元确定模块410被配置成：将待编码视频帧图像划分为多个编码单元。编码模式确定模块420被配置成：针对所述多个编码单元中的每一个编码单元：确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；以及，基于所述编码单元的颜色构成，确定所述编码单元的编码模式。上述各个模块涉及在上文中关于图3描述的步骤320、330、340、350和360的操作，因而在此不再赘述。
上面关于图9描述的各个模块均可以在硬件中或在结合软件和/或固件的硬件中实 现。例如，这些模块可以被实现为计算机可执行代码/指令，该计算机可执行代码/指令被配置为在一个或多个处理器中执行并存储在计算机可读存储介质中。可替换地，这些模块可以被实现为硬件逻辑/电路。例如，在一些实施例中，这些模块中的一个或多个可以一起在片上系统(SoC)中实现。SoC可以包括集成电路芯片(其包括处理器(例如，中央处理单元(CPU)、微控制器、微处理器、数字信号处理器(DSP)等)、存储器、一个或多个通信接口、和/或其他电路中的一个或多个部件)，并且可以可选地执行所接收的程序代码和/或包括嵌入式固件以执行功能。
参见图10，其以框图的形式示意性地示出了根据本公开的一些实施例的一种计算设备500的结构。计算设备500可以被用于本公开描述的各种应用场景。
计算设备500可以包括能够诸如通过系统总线514或其他适当的方式连接彼此通信的至少一个处理器502、存储器504、(多个)通信接口506、显示设备508、其他输入/输出(I/O)设备510以及一个或多个大容量存储装置512。
处理器502可以是单个处理单元或多个处理单元，所有处理单元可以包括单个或多个计算单元或者多个核心。处理器502可以被实施成一个或多个微处理器、微型计算机、微控制器、数字信号处理器、中央处理单元、状态机、逻辑电路和/或基于操作指令来操纵信号的任何设备。除了其他能力之外，处理器502可以被配置成获取并且执行存储在存储器504、大容量存储装置512或者其他计算机可读介质中的计算机可读指令，诸如操作系统516的程序代码、应用程序518的程序代码、其他程序520的程序代码等。
存储器504和大容量存储设备512是用于存储指令的计算机可读存储介质的示例，指令可以由处理器502执行来实施前面所描述的各种功能。举例来说，存储器504一般可以包括易失性存储器和非易失性存储器二者(例如RAM、ROM等等)。此外，大容量存储设备512一般可以包括硬盘驱动器、固态驱动器、可移除介质、包括外部和可移除驱动器、存储器卡、闪存、软盘、光盘(例如CD、DVD)、存储阵列、网络附属存储、存储区域网等等。存储器504和大容量存储设备512在本文中都可以被统称为计算机可读存储器或计算机可读存储介质，并且可以是能够把计算机可读、处理器可执行程序指令存储为计算机可执行代码的非瞬时性介质，计算机可执行代码可以由处理器502作为被配置成实施在本公开的各示例性实施例中所描述的操作和功能的特定机器来执行。
多个程序模块可以存储在大容量存储设备512上。这些程序模块包括操作系统516、一个或多个应用程序518、其他程序520和程序数据522，并且它们可以被处理器502执行。这样的应用程序或程序模块的示例可以包括例如用于实现以下部件/功能的计算机程 序逻辑(例如，计算机可执行代码或指令)：编码单元确定模块410和编码模式确定模块420。
虽然在图10中被图示成存储在计算设备500的存储器504中，但是编码单元确定模块410和编码模式确定模块420或者它们的一部分可以使用可由计算设备500访问的任何形式的计算机可读介质来实施。如本文所使用的，术语“计算机可读介质”的含义至少包括两种类型的计算机可读介质，也就是计算机存储介质和通信介质。计算机存储介质包括通过用于存储信息的任何方法或技术实施的易失性和非易失性、可移除和不可移除介质，信息诸如是计算机可读指令、数据结构、程序模块或者其他数据。计算机存储介质包括而不限于RAM、ROM、EEPROM、闪存或其他存储器技术，CD-ROM、数字通用盘(DVD)、或其他光学存储装置，磁盒、磁带、磁盘存储装置或其他磁性存储设备，或者可以被用来存储信息以供计算设备访问的任何其他非传送介质。与此相对，通信介质可以在诸如载波或其他传送机制之类的已调数据信号中具体实现计算机可读指令、数据结构、程序模块或其他数据。本公开所定义的计算机存储介质不包括通信介质。
计算设备500还可以包括一个或更多通信接口506，以用于诸如通过网络、直接连接等等与其他设备交换数据。通信接口506可以促进在多种网络和协议类型内的通信，其中包括有线网络(例如LAN、电缆等等)和无线网络(例如WLAN、蜂窝、卫星等等)、因特网等等。通信接口506还可以提供与诸如存储阵列、网络附属存储、存储区域网等等中的外部存储装置(未示出)的通信。
在一些示例中，计算设备500还可以包括诸如显示器之类的显示设备508，以用于显示信息和图像。其他I/O设备510可以是接收来自目标对象的各种输入并且向目标对象提供各种输出的设备，包括但不限于触摸输入设备、手势输入设备、摄影机、键盘、遥控器、鼠标、打印机、音频输入/输出设备等等。
本公开还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算设备执行上述各种可选实现方式中提供的用于确定关联应用的方法或者用于确定推荐内容的方法。
本公开中使用的术语仅用于描述本公开中的各示例性实施例，并不意图限制本公开。如本文中使用的，单数形式“一个”、“一”和“该”旨在也包括复数形式，除非上下文清楚地另有指示。还要理解的是，术语“包括”和“包含”当在本公开中使用时，是指所述及的特征的存在，但不排除一个或多个其他特征的存在或者添加一个或多个 其他特征。如本文中使用的，术语“和/或”包括相关联的列出项目中的一个或多个的任意和全部组合。将理解的是，尽管术语“第一”、“第二”、“第三”等在本文中可以用来描述各种特征，但是这些特征不应当由这些术语限制。这些术语仅用来将一个特征与另一个特征相区分。
除非另有定义，本公开中使用的所有术语(包括技术术语和科学术语)具有与本公开所属领域的普通技术人员所通常理解的相同含义。还要理解的是，诸如那些在通常使用的字典中定义的之类的术语应当被解释为具有与其在相关领域和/或本说明书上下文中的含义相一致的含义，并且将不在理想化或过于正式的意义上进行解释，除非本公开中明确地如此定义。
在本说明书的描述中，术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点被包含于本公开的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
本文在软件硬件元件或程序模块的一般上下文中描述了各种技术。一般地，这些模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、元素、组件、数据结构等。本文所使用的术语“模块”，“功能”和“组件”一般表示软件、固件、硬件或其组合。本文描述的技术的特征是与平台无关的，意味着这些技术可以在具有各种处理器的各种计算平台上实现。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。此外，还应理解的是，在流程图中表示或在此以其他方式描述的方法的各个步骤仅仅示例性的，并不意味着所图示或描述的方法的步骤必须按照所示或所述的步骤执行。相反，在流程图中表示或在此以其他方式描述的方法的各个步骤可以按照与本公开中不同的顺序被执行，或者可以同时被执行。此外，根据需要，在流程图中表示或在此以其他方式描述的方法还可以包括其他附加的步骤。
应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在 上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，则可用本领域公知的下列技术中的任一项或它们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路、具有合适的组合逻辑门电路的专用集成电路、可编程门阵列、现场可编程门阵列等。
本技术领域的普通技术人员可以理解上述实施例方法的全部或部分步骤可以通过程序指令相关的硬件完成，所述程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括执行方法实施例的步骤之一或其组合。
应当理解，在本公开的各示例性实施方式中，可能涉及到待编码视频、待编码图像等数据。当本公开中所描述的涉及这样的数据的示例性实施例被应用到具体产品或技术中时，需要获得用户许可或者同意，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
尽管已经结合一些示例性实施例详细地描述了本公开，但是其不旨在被限制于在本文中所阐述的特定形式。相反，本公开的范围仅由所附权利要求来限定。

Claims (17)

  1. 一种视频编码方法，由计算设备执行，所述方法包括：
    将待编码视频帧图像划分为多个编码单元；
    针对所述多个编码单元中的每一个编码单元，执行下述步骤：
    确定所述编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；
    基于所述编码单元的颜色构成，确定所述编码单元的编码模式。
  2. 根据权利要求1所述的视频编码方法，其中，所述计算设备确定编码单元的颜色构成包括：
    所述计算设备对所述编码单元中各个像素的颜色进行聚类处理，以确定所述编码单元包含的颜色的种类；
    所述计算设备基于所述编码单元包含的颜色的种类生成颜色索引表，其中，所述颜色索引表包括的颜色索引的数量对应于所述颜色数。
  3. 根据权利要求1所述的视频编码方法，其中，所述颜色构成还包括所述编码单元包含的各种颜色各自对应的像素数量，并且所述计算设备基于所述编码单元的颜色构成，确定所述编码单元的编码模式包括：
    所述计算设备将所述编码单元包含的各种颜色按照各自对应的像素数量从大到小地进行颜色排序；
    所述计算设备将所述颜色排序中前M个颜色各自对应的像素数量相加，以生成主要颜色像素数量，其中，M为预先设定的主要颜色数，其为正整数并且小于所述颜色数；
    所述计算设备基于所述编码单元的像素总数确定调色板编码模式阈值；
    所述计算设备响应于所述主要颜色像素数量大于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。
  4. 根据权利要求1或2所述的视频编码方法，其中，所述计算设备基于所述编码单元的颜色构成，确定所述编码单元的编码模式包括：
    所述计算设备响应于所述颜色数小于或等于预先设定的颜色数阈值，将所述编码单元的编码模式确定为调色板编码模式。
  5. 根据权利要求2或4所述的视频编码方法，其中，所述计算设备对所述编码单元中各个像素的颜色进行聚类处理，以确定所述编码单元包含的颜色的种类包括：
    所述计算设备基于所用的颜色编码，确定所述编码单元中各个像素各自对应的颜色 编码值；
    所述计算设备基于所述编码单元中各个像素各自对应的颜色编码值，确定所述编码单元包含的颜色的种类。
  6. 根据权利要求5所述的视频编码方法，其中，所述颜色编码是YUV颜色编码或者RGB颜色编码。
  7. 根据权利要求3所述的视频编码方法，其中，所述视频编码方法还包括：
    所述计算设备响应于所述主要颜色像素数量小于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
  8. 根据权利要求4-6中任一项所述的视频编码方法，其中，所述颜色构成还包括所述编码单元包含的各种颜色各自对应的像素数量，并且所述视频编码方法还包括：所述计算设备响应于所述颜色数大于所述颜色数阈值，执行下述步骤：
    所述计算设备将所述编码单元包含的各种颜色按照各自对应的像素数量从小到大地进行颜色排序；
    所述计算设备将所述颜色排序中前N个颜色各自对应的像素数量相加，以生成次要颜色像素数量，其中，N为预先设定的次要颜色数，其为正整数并且小于所述颜色数；
    所述计算设备基于所述编码单元的像素总数确定调色板编码模式阈值；
    所述计算设备响应于所述次要颜色像素数量小于或等于所述调色板编码模式阈值，将所述编码单元的编码模式确定为调色板编码模式。
  9. 根据权利要求8所述的视频编码方法，其中，所述视频编码方法还包括：
    所述计算设备响应于所述次要颜色像素数量大于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
  10. 根据权利要求4-6中任一项所述的视频编码方法，其中，所述颜色构成还包括所述编码单元包含的各种颜色各自对应的像素数量，并且所述视频编码方法还包括：所述计算设备响应于所述颜色数大于所述颜色数阈值，执行下述步骤：
    所述计算设备将所述编码单元包含的各种颜色按照各自对应的像素数量从大到小地进行颜色排序；
    所述计算设备将所述颜色排序中前M个颜色各自对应的像素数量相加，以生成主要颜色像素数量，其中，M为预先设定的主要颜色数，其为正整数并且小于所述颜色数；
    所述计算设备基于所述编码单元的像素总数确定调色板编码模式阈值；
    所述计算设备响应于所述主要颜色像素数量大于或等于所述调色板编码模式阈值， 将所述编码单元的编码模式确定为调色板编码模式。
  11. 根据权利要求10所述的视频编码方法，其中，所述视频编码方法还包括：
    所述计算设备响应于所述主要颜色像素数量小于所述调色板编码模式阈值，确定所述编码单元跳过调色板编码模式。
  12. 根据权利要求3、8和10中任一项所述的视频编码方法，其中，所述计算设备基于所述编码单元的像素总数确定调色板编码模式阈值包括：
    所述计算设备获取预先设定的阈值系数，其中，所述阈值系数与所述像素总数相关联并且小于1；
    所述计算设备将所述像素总数乘以所述阈值系数，以生成所述调色板编码模式阈值。
  13. 根据权利要求3、8和10中任一项所述的视频编码方法，其中，所述计算设备基于所述编码单元的像素总数确定调色板编码模式阈值包括：
    所述计算设备获取预先设定的像素偏差数量，其中，所述像素偏差数量与所述像素总数相关联并且小于所述像素总数；
    所述计算设备将所述像素总数减去所述像素偏差数量，以生成所述调色板编码模式阈值。
  14. 一种视频编码装置，包括：
    编码单元确定模块，其被配置成：将待编码视频帧图像划分为多个编码单元；
    编码模式确定模块，其被配置成：针对所述多个编码单元中的每一个编码单元：
    确定编码单元的颜色构成，其中，所述颜色构成包括所述编码单元包含的颜色的颜色数；
    基于所述编码单元的颜色构成，确定所述编码单元的编码模式。
  15. 一种计算设备，其包括处理器和存储器，所述存储器被配置成存储计算机可执行指令，所述计算机可执行指令被配置成当在所述处理器上执行时，使所述处理器执行根据权利要求1至13中任一项所述的视频编码方法。
  16. 一种计算机可读存储介质，其被配置成存储计算机可执行指令，所述计算机可执行指令被配置成当在处理器上执行时，使所述处理器执行根据权利要求1至13中任一项所述的视频编码方法。
  17. 一种计算机程序产品，其包括计算机可执行指令，所述计算机可执行指令被配置成当在处理器上执行时，使所述处理器执行根据权利要求1至13中任一项所述的视频编码方法。
PCT/CN2023/122197 2022-09-29 2023-09-27 视频编码方法和装置、计算设备、存储介质及程序产品 WO2024067736A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211197925.0 2022-09-29
CN202211197925.0A CN117834892A (zh) 2022-09-29 2022-09-29 视频编码方法和装置、计算设备、存储介质及程序产品

Publications (1)

Publication Number Publication Date
WO2024067736A1 true WO2024067736A1 (zh) 2024-04-04

Family

ID=90476286

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/122197 WO2024067736A1 (zh) 2022-09-29 2023-09-27 视频编码方法和装置、计算设备、存储介质及程序产品

Country Status (2)

Country Link
CN (1) CN117834892A (zh)
WO (1) WO2024067736A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150189302A1 (en) * 2014-01-02 2015-07-02 Qualcomm Incorporated Color index coding for palette-based video coding
US20150281703A1 (en) * 2014-03-26 2015-10-01 Qualcomm Incorporated Determining palette size, palette entries and filtering of palette coded blocks in video coding
US20150341643A1 (en) * 2014-05-23 2015-11-26 Futurewei Technologies, Inc. Advanced screen content coding with improved palette table and index map coding methods
CN115118964A (zh) * 2022-06-24 2022-09-27 腾讯科技（深圳）有限公司 视频编码方法、装置、电子设备及计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150189302A1 (en) * 2014-01-02 2015-07-02 Qualcomm Incorporated Color index coding for palette-based video coding
US20150281703A1 (en) * 2014-03-26 2015-10-01 Qualcomm Incorporated Determining palette size, palette entries and filtering of palette coded blocks in video coding
US20150341643A1 (en) * 2014-05-23 2015-11-26 Futurewei Technologies, Inc. Advanced screen content coding with improved palette table and index map coding methods
CN115118964A (zh) * 2022-06-24 2022-09-27 腾讯科技（深圳）有限公司 视频编码方法、装置、电子设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN117834892A (zh) 2024-04-05

Similar Documents

Publication Publication Date Title
US11200426B2 (en) Video frame extraction method and apparatus, computer-readable medium
US20220038724A1 (en) Video stream decoding method and apparatus, terminal device, and storage medium
CN112040233B (zh) 视频编码、视频解码方法和装置、电子设备及存储介质
CN105163127A (zh) 视频分析方法及装置
CN113473126B (zh) 视频流的处理方法、装置、电子设备及计算机可读介质
US11277619B2 (en) Rate control for video splicing applications
WO2021196994A1 (zh) 编码的方法及装置、终端和存储介质
CN107172376B (zh) 一种基于屏幕共享的视频编码方法和装置
CN112333448A (zh) 视频编码、解码方法和装置、电子设备和存储介质
US8620096B2 (en) Virtualization server for presentation virtualization and image data encoding method
CN112351285A (zh) 视频编码、解码方法和装置、电子设备和存储介质
CN115118964A (zh) 视频编码方法、装置、电子设备及计算机可读存储介质
CN101057503A (zh) 用于视频重放系统的正常播放和特技模式播放的胶片颗粒模拟
US20170134454A1 (en) System for cloud streaming service, method for still image-based cloud streaming service and apparatus therefor
WO2024067736A1 (zh) 视频编码方法和装置、计算设备、存储介质及程序产品
JP2022526547A (ja) 画像処理方法および電子デバイス
TWI700918B (zh) 具有初始化片段之視訊及音訊之寫碼
CN107426611B (zh) 基于视频转码的多路输出方法及系统
CN110969672A (zh) 图像压缩方法及装置
CN115243101B (zh) 视频动静率识别方法、装置、电子设备及存储介质
Kumar et al. Macro-block mode decision in MPEG-2 video compression using machine learning
WO2023029689A1 (zh) 多媒体数据的共享方法、媒体共享服务器、终端、电子设备和计算机可读存储介质
WO2023185806A1 (zh) 一种图像编解码方法、装置、电子设备及存储介质
WO2024098821A1 (zh) Av1的滤波方法及装置
US9092790B1 (en) Multiprocessor algorithm for video processing