CN117979015A

CN117979015A - Video encoding and decoding method, video encoder and video decoder

Info

Publication number: CN117979015A
Application number: CN202211315812.6A
Authority: CN
Inventors: 屈小刚
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2024-05-03

Abstract

The application discloses a video encoding and decoding method, a video encoder and a video decoder. The video encoding method includes: acquiring an image sequence, wherein the image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, and the video stream comprises a plurality of images which are continuous in time; acquiring a first coding rule, wherein the first coding rule is used for indicating the corresponding relation between Q encoders and the K images, each encoder in the Q encoders is used for processing the corresponding image, the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1; and according to the first coding rule, coding the image corresponding to the qth coder in the image sequence by the qth coder in the qth coder, wherein Q is [1, Q ]. The method provided by the embodiment of the application can improve the frame rate of video encoding and decoding.

Description

Video encoding and decoding method, video encoder and video decoder

Technical Field

The present application relates to the field of video encoding and decoding technology, and more particularly, to a video encoding and decoding method, a video encoder, and a video decoder.

Background

In the scenes of live broadcasting, cloud desktop, cloud games and the like, images are generally collected at an encoding end and are encoded into video streams to be transmitted to terminal equipment, and the higher the live broadcasting encoding frame rate is, the better the user experience is.

In the prior art, the frame rate of video encoding and decoding is improved by a method of end side frame insertion. From a technical perspective, only part of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) frame insertion algorithms can achieve a relatively seamless frame insertion experience, but this method requires a relatively high-performance graphics processor (graphic processing unit, GPU) or neural network processor (neural network processing unit, NPU), and most mobile terminals cannot provide such high computational power.

Therefore, how to improve the frame rate of video encoding and decoding without wasting computation is a technical problem that needs to be solved at present.

Disclosure of Invention

The application provides a video encoding and decoding method, a video encoder and a video decoder, which are used for improving the frame rate of encoding/decoding.

In a first aspect, a video encoding method is provided, the method comprising: acquiring an image sequence, wherein the image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, the video stream comprises a plurality of images which are continuous in time, and K is an integer greater than or equal to 1; acquiring a first coding rule, wherein the first coding rule is used for indicating the corresponding relation between Q encoders and the K images, each encoder in the Q encoders is used for processing the corresponding image, the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1; and according to the first coding rule, coding the image corresponding to the qth coder in the image sequence by the qth coder in the qth coder, wherein Q is [1, Q ].

The video coding method provided by the embodiment of the application can fully utilize the software resource and the hardware resource of the computer equipment, so that the software and the hardware of the computer equipment can cooperatively code the acquired image sequence, and the frame rate can be improved by at least one time.

It should be understood that Q encoders are encoders created by the encoding end device, and the Q encoders may encode corresponding images in the image sequence in parallel according to a preset first encoding rule. The correspondence between the Q encoders and the K images is indicated by a preset first encoding rule.

Illustratively, the Q encoders include one software encoder and one hardware encoder. As an example, the first encoding rule in an embodiment of the present application may be that odd frames in the image sequence are encoded by a software encoder and even frames are encoded by a hardware encoder. Illustratively, the images numbered 1, 3, 5 … … are encoded by a software encoder and the images numbered 2,4,6 … … are encoded by a hardware encoder. Alternatively, it is also possible to provide that even frames in the image sequence are encoded by a software encoder and that odd frames are encoded by a hardware encoder.

Illustratively, the Q encoders include two software encoders and one hardware encoder. As an example, the first encoding rule in the embodiment of the present application may be that the images numbered 1, 4, 7 … … are encoded by a first software encoder, the images numbered 2, 5, 8 … … are encoded by a second software encoder, and the images numbered 3, 6, 9 … … are encoded by a hardware encoder.

The person skilled in the art can set different first coding rules according to the requirements, and the application is not limited to specific video coding rules. Other coding rules are used on the basis of the video coding method provided by the application, and the protection scope of the application is also provided.

For example, if the encoding device includes a hardware encoder and a software encoder, the image sequence is encoded to form a 2-way code stream, and the 2-way code stream is decoded at the decoding end simultaneously and then is synchronized to form a reconstructed image sequence.

The computer equipment in the embodiment of the application comprises, but is not limited to, mobile phones, tablet computers, notebook computers and other equipment.

With reference to the first aspect, in certain implementation manners of the first aspect, before the acquiring the first coding rule, the method further includes: the Q encoders are created according to the computing power of the computer device, the image resolution and the frame rate.

It should be understood that the computing power of the computer device refers to the capability of a central processing unit (central processing unit, CPU), a graphics processor (graphic processing unit, GPU), a tensor processing unit (tensor processing unit, TPU), a field programmable gate array (field program gate way, FPGA), an Application SPECIFIC INTEGRATED Circuit (ASIC), and other types of processors of the computer to perform the number of data operations per second by depending on a carrier device such as a computer server, a high-performance computing cluster, and various types of intelligent terminals. The computing power of a computer device may also be referred to simply as computing power.

It should be appreciated that the number of encoders created and managed by a computer device is related to the computational power of the device, the preset image resolution and the frame rate. Illustratively, on a cell phone where the software is maximally capable of encoding 1080p, 30 frames per second (framespersecond, fps) of video streams and the hardware is maximally capable of encoding 1080p, 30fps of video streams, 1 software encoder and 1 hardware encoder can be created, and maximally capable of encoding 1080p, 60fps of video streams. Alternatively, 2 software encoders may be created, each encoding a 1080p, 15fps video stream, and 1 hardware encoder encoding a 1080p, 30fps video stream. The number of software encoders and the number of hardware encoders specifically created by the computer device are not specifically defined by the present application.

With reference to the first aspect, in certain implementations of the first aspect, the encoding parameters of the Q encoders are the same.

The N software encoders and the M hardware encoders need to set the same encoding parameters, including, but not limited to, encoding format (h.264, h.265, etc.), pixel format (yuv 420p, etc.), bit depth, resolution, frame rate, code rate, I-frame interval, B-frame interval, maximum reference frame number, slice mode (single slice or multiple slices), profile type (base profile, main profile, etc.), encapsulation format (rtmp, hls, etc.), etc.

With reference to the first aspect, in certain implementations of the first aspect, the video stream is a live video stream.

The video coding method provided by the embodiment of the application can fully utilize the software resource and the hardware resource of the computer equipment, so that the software and the hardware of the computer equipment can cooperatively code the acquired image sequence, the real-time performance of live video playing can be ensured, and the frame rate can be improved by at least one time.

With reference to the first aspect, in certain implementation manners of the first aspect, the method further includes: when the resolution of the first image is larger than a preset threshold, dividing the first image into Y sub-images, wherein the first image is any one image in the image sequence, and Y is E [2, Q ]; acquiring a second coding rule, wherein the second coding rule is used for indicating the corresponding relation between Y encoders and Y sub-images, the Y encoders are any Y encoders in the Q encoders, and the Y sub-images are in one-to-one correspondence with the Y encoders; and according to the second coding rule, coding the sub-image corresponding to the Y encoder in the Y sub-images through the Y encoder in the Y encoders, wherein Y is [1, Y ].

When the method provided by the embodiment of the application is used for encoding the image with super-resolution, the image can be firstly divided, and then the divided sub-images are encoded in parallel by using a software encoder and a hardware encoder, namely, each sub-image corresponds to one encoder.

The second encoding rule is similar to the first encoding rule, and the object encoded by the encoder is replaced by a sub-image, which is not described in detail in the present application.

In some possible implementations, the method provided by the embodiments of the present application may use software and hardware resources of multiple devices to encode in parallel when encoding a sequence of images with high resolution and ultra-high frame rate.

The video coding method provided by the embodiment of the application can divide the image when coding the image with super-large resolution, and then uses the software coder and the hardware coder to cooperatively code the divided sub-images, thereby improving the frame rate by at least one time.

In a second aspect, there is provided a video decoding method, the method comprising: acquiring a Q-channel code stream, wherein the Q-channel code stream comprises a code stream generated after Q encoders encode corresponding images in K images in a video stream according to a first encoding rule, the first encoding rule is used for indicating the corresponding relation between the Q encoders and the K images, the video stream comprises a plurality of images which are continuous in time, the Q encoders comprise N software encoders and M hardware encoders, K is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1; decoding the Q-path code stream through Q decoders to obtain a plurality of decoded images, wherein the Q decoders are in one-to-one correspondence with the Q-path code stream; the plurality of images are displayed according to a temporal order of the plurality of images.

It should be appreciated that Q decoders may decode the Q-way code stream in parallel, resulting in decoded multiple pictures, each of which may carry temporal information related to the video stream. The display unit of the playback end may display the decoded images according to the time sequence of the images.

The video decoding method provided by the embodiment of the application can decode the multipath code streams coded by the software coder and the hardware coder at the same time, and can improve the video playing frame rate by at least one time.

With reference to the second aspect, in certain implementations of the second aspect, before the decoding of the Q-way code stream by the Q decoders, the method further includes: and creating the Q decoders according to the Q-channel code stream.

The decoding end can create a software encoder and/or a hardware encoder according to the composition rule of the obtained code stream. For example, the decoding end may determine that the obtained code stream is Q paths according to the content description about the code stream in the transmission unit, so as to correspondingly create Q decoders.

With reference to the second aspect, in certain implementations of the second aspect, decoding parameters of the Q decoders are the same.

It should be appreciated that the decoding parameters of the plurality of decoders created by the decoding scheduling module are the same, including, but not limited to, encoding formats (h.264, h.265, etc.), encapsulation formats (e.g., rtmp, hls, etc.), and the like.

With reference to the second aspect, in certain implementations of the second aspect, the Q decoders include at least one hardware decoder and at least one software decoder.

For example, q=2, the decoding end may create one software encoder and one hardware encoder to decode the first and second path code streams, respectively, or in case of sufficient computational power at the decoding end, two software encoders may be created to decode the first and second path code streams, respectively, or two hardware encoders may be created to decode the first and second path code streams, respectively.

With reference to the second aspect, in some implementations of the second aspect, the acquiring a Q-way code stream includes: and downloading the Q-channel code stream according to the uniform resource locator URL of the live video stream.

For example, q=2, the decoding side includes a software decoder and a hardware decoder, where the software decoder may download the first path of code stream according to the URL of the live broadcast to perform subsequent decoding, and the hardware decoder may download the second path of code stream according to the URL of the live broadcast to perform subsequent decoding. It should be appreciated that the downloading of the first path code stream and the downloading of the second path code stream may be performed simultaneously.

With reference to the second aspect, in certain implementations of the second aspect, the displaying the plurality of images according to a temporal order of the plurality of images includes: displaying the plurality of images according to the order of presentation time stamps of the plurality of images.

It should be understood that each of the decoded plurality of pictures includes a presentation time stamp (presentation timestamp, PTS), and the display unit at the decoding end may display the decoded plurality of pictures in the PTS order.

In a third aspect, a video encoding apparatus is provided, the video encoding apparatus comprising a number of functional units for implementing any of the methods of the first aspect.

For example, the video encoding apparatus may include an acquisition unit and an encoding unit.

In a fourth aspect, a video decoding apparatus is provided, comprising a number of functional units for implementing any of the methods of the second aspect.

For example, the video decoding apparatus may include a stream receiving unit, a decoding unit, and a display unit.

In a fifth aspect, there is provided a video encoder comprising: the image acquisition unit is used for acquiring an image sequence, wherein the image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, the video stream comprises a plurality of images which are continuous in time, and K is an integer greater than or equal to 1; the encoding unit is used for acquiring a first encoding rule, wherein the first encoding rule is used for indicating the corresponding relation between Q encoders and the K images, each encoder in the Q encoders is used for processing the corresponding image, the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1; the coding unit is further configured to code, according to the first coding rule, a picture corresponding to a qth encoder in the picture sequence by the qth encoder in the qth encoder, where Q is e [1, Q ].

With reference to the fifth aspect, in certain implementations of the fifth aspect, the encoding unit is further configured to create the Q encoders according to a computing capability of a computer device, the image resolution, and the frame rate.

With reference to the fifth aspect, in certain implementations of the fifth aspect, the encoding parameters of the Q encoders are the same.

With reference to the fifth aspect, in certain implementations of the fifth aspect, the video stream is a live video stream.

With reference to the fifth aspect, in certain implementation manners of the fifth aspect, the encoding unit is further configured to, when a resolution of a first image is greater than a preset threshold, divide the first image into Y sub-images, where the first image is any image in the image sequence, Y e [2, q ]; the coding unit is further configured to obtain a second coding rule, where the second coding rule is used to indicate a correspondence between Y encoders and the Y sub-images, the Y encoders are any Y encoders in the Q encoders, and the Y sub-images are in one-to-one correspondence with the Y encoders; the coding unit is further configured to code, according to the second coding rule, a sub-image corresponding to a Y-th encoder in the Y sub-images by a Y-th encoder in the Y encoders, where Y is e [1, Y ].

The advantages of the fifth aspect and any possible implementation manner of the fifth aspect correspond to those of the first aspect and any possible implementation manner of the first aspect, and are not described in detail.

In a sixth aspect, there is provided a video decoder comprising: the receiving unit is used for obtaining a Q-channel code stream, wherein the Q-channel code stream comprises a code stream generated after Q encoders encode corresponding images in K images in a video stream according to a first encoding rule, the first encoding rule is used for indicating the corresponding relation between the Q encoders and the K images, the video stream comprises a plurality of images which are continuous in time, the Q encoders comprise N software encoders and M hardware encoders, K is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1; the decoding unit is used for decoding the Q-path code streams through Q decoders to obtain a plurality of decoded images, and the Q decoders are in one-to-one correspondence with the Q-path code streams; and a display unit for displaying the plurality of images according to a time sequence of the plurality of images.

With reference to the sixth aspect, in certain implementation manners of the sixth aspect, the decoding unit is further configured to create the Q decoders according to the Q-way code stream.

With reference to the sixth aspect, in certain implementations of the sixth aspect, decoding parameters of the Q decoders are the same.

With reference to the sixth aspect, in certain implementations of the sixth aspect, the Q decoders include at least one hardware decoder and at least one software decoder.

With reference to the sixth aspect, in some implementations of the sixth aspect, the receiving unit is further configured to download the Q-way code stream according to a URL of a live video stream.

With reference to the sixth aspect, in certain implementation manners of the sixth aspect, the display unit is further configured to display the plurality of images according to an order of presentation time stamps of the plurality of images.

Advantageous effects of any one of the possible implementation manners of the sixth aspect and the sixth aspect correspond to advantageous effects of any one of the possible implementation manners of the second aspect and the second aspect, and are not described in detail.

In a seventh aspect, an embodiment of the present application provides an apparatus for encoding video data, the apparatus comprising: a memory for storing video data, the video data comprising one or more images; a video encoder for implementing any of the methods of the first aspect.

In an eighth aspect, an embodiment of the present application provides an apparatus for decoding video data, the apparatus comprising: a memory for storing video data in the form of a code stream; a video decoder for implementing any of the methods of the second aspect.

In a ninth aspect, an embodiment of the present application provides an encoding apparatus, including: a memory and a processor that invokes program code stored in the memory to perform part or all of the steps of any one of the methods of the first aspect.

Optionally, the memory is a nonvolatile memory.

Optionally, the memory and the processor are coupled to each other.

In a tenth aspect, an embodiment of the present application provides a decoding apparatus, including: a memory and a processor that invokes program code stored in the memory to perform part or all of the steps of any one of the methods of the second aspect.

Optionally, the memory is a nonvolatile memory.

Optionally, the memory and the processor are coupled to each other.

In an eleventh aspect, embodiments of the present application provide a computer-readable storage medium storing program code, wherein the program code includes instructions for performing part or all of the steps of any one of the methods of the first or second aspects.

In a twelfth aspect, embodiments of the present application provide a computer program product which, when run on a computer, causes the computer to perform part or all of the steps of any one of the methods of the first or second aspects.

Drawings

Fig. 1 is a schematic diagram of a video encoding and decoding system according to an embodiment of the present application.

Fig. 2 is an application schematic diagram of a video encoding and decoding method according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a video encoding rule according to an embodiment of the present application.

Fig. 4 is an exemplary flowchart of a video encoding method according to an embodiment of the present application.

Fig. 5 is an exemplary flowchart of another video encoding method provided by an embodiment of the present application.

Fig. 6 is an exemplary flowchart of a video decoding method according to an embodiment of the present application.

Fig. 7 is a schematic block diagram of a video encoder provided by an embodiment of the present application.

Fig. 8 is a schematic block diagram of a video decoder according to an embodiment of the present application.

Fig. 9 is a schematic block diagram of another video encoder provided by an embodiment of the present application.

Fig. 10 is a schematic block diagram of another video decoder provided by an embodiment of the present application.

Detailed Description

Video coding generally refers to processing a sequence of pictures that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" may be used as synonyms. Video encoding is performed on the source side, typically including processing (e.g., by compression) the original video picture to reduce the amount of data required to represent the video picture, thereby more efficiently storing and/or transmitting. Video decoding is performed on the destination side, typically involving inverse processing with respect to the encoder to reconstruct the video pictures.

In the scenes of live broadcasting, cloud desktop, cloud games and the like, images are generally collected at an encoding end and are encoded into video streams to be transmitted to terminal equipment, and the higher the live broadcasting encoding frame rate is, the better the user experience is. In the prior art, the frame rate of video encoding and decoding is improved by a method of end side frame insertion. From a technical perspective, only part of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) frame insertion algorithms can achieve a relatively seamless frame insertion experience, but this method requires a relatively high-performance graphics processor (graphic processing unit, GPU) or neural network processor (neural network processing unit, NPU), and most mobile terminals cannot provide such high computational power. And the coding end of the prior technical proposal is coded by hardware or software, and the equipment resources can not be fully utilized. The method provided by the embodiment of the application can fully utilize the software resource and the hardware resource of the equipment, and the software and the hardware can cooperatively encode the image, so that the frame rate can be improved by at least one time.

To facilitate an understanding of the embodiments of the application, some definitions referred to in the present application will first be briefly described.

1. Frame rate (FRAME RATE): is the frequency (rate) at which bitmap images in frames appear continuously on the display.

2. Software codec: during information transmission, signals such as video and the like are directly encoded/decoded by a central processing unit (central processing unit, CPU).

3. Hardware codec: the video stream is encoded/decoded by using a non-CPU or a CPU with little occupation, wherein the non-CPU may include a graphics processor (graphic processing unit, GPU), a field programmable gate array (field program gate way, FPGA), or an Application Specific Integrated Circuit (ASIC), etc. independent hardware modules, the video encoding/decoding operation with high CPU usage is separated from the CPU, so that the use load of the CPU is reduced, and the platform can efficiently and smoothly perform the encoding/decoding operation of the video.

I frame: the key frame is represented, which can be understood as the complete reservation of the frame picture, and the decoding can be completed only by the frame data due to the complete picture.

5.P frames: representing the forward reference frame, representing the difference between the frame and a previous key frame or P frame, the difference defined by the frame is overlapped by the previously buffered frame when decoding, and the final frame is generated, and the P frame has no complete frame data and only has the data of the difference with the frame of the previous frame.

B frame: bi-predictive frames compress encoded pictures of a transmitted data volume taking into account both the encoded frames preceding the original picture sequence and the temporal redundancy information between the encoded frames following the original picture sequence.

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. It will be apparent that the described embodiments are some, but not all, embodiments of the application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.

The present application will present various aspects, embodiments, or features about a system comprising a plurality of devices, components, modules, etc. It is to be understood and appreciated that the various systems may include additional devices, components, modules, etc. and/or may not include all of the devices, components, modules etc. discussed in connection with the figures. Furthermore, combinations of these schemes may also be used.

In addition, in the embodiments of the present application, words such as "exemplary," "for example," and the like are used to indicate an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term use of an example is intended to present concepts in a concrete fashion.

The service scenario described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application, and as a person of ordinary skill in the art can know that, with the evolution of the network architecture and the appearance of a new service scenario, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: including the case where a alone exists, both a and B together, and B alone, where a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

The video encoding and decoding system provided by the embodiment of the application comprises an acquisition end 110, a transmission unit 150 and a playing end 120. The acquisition end 110 includes an acquisition unit 130 and an encoding unit 140, and the playing end 120 includes a stream receiving unit 160, a decoding unit 170 and a display unit 180. Optionally, the collecting end 110 may also include a streaming unit, which is not limited in the present application.

The capturing unit 130 is configured to capture images at a preset frame rate, including but not limited to capturing images from a camera, capturing images from a graphics card, capturing images from a graphics engine, and the like.

The encoding unit 140 is configured to perform image encoding, and encode each frame of image into P-channel video code streams in parallel according to a software and hardware allocation rule and a time sequence, where common encoding formats are h.264, h.265, AV1, VP8, VP9, and the like.

The transmission unit 150 is configured to transmit the code stream, and transmit the code stream output by the acquisition end 110 to the playing end 120.

The receiving unit 160 is configured to download and buffer the code stream, and download the code stream from the transmitting unit 150 to the local buffer.

The decoding unit 170 is configured to decode the video code stream, and may decode the P-channel video code stream at the same time, sort the video code stream according to the coding rule of the coding unit 140 according to the image acquisition order, and then transmit the video code stream to the display unit 180.

The display unit 180 is used for displaying the image, and displaying the image sent from the decoding unit 170 on a display screen.

The acquisition end 210 includes an encoding scheduling module 130, a software encoder 240, and a hardware encoder 250. The playback end 220 includes a display unit 180, a software decoder 260, and a hardware decoder 270.

The code scheduling module 230: for creating and managing the software encoder 240 and the hardware encoder 250, the number of encoders that the encoding scheduling module 230 can create and manage is related to the computational power of the device, the image resolution preset when capturing the image sequence, the frame rate of the capturing. Illustratively, on a mobile phone, the software can maximally encode 1080p and 30 transmission frame number (framespersecond, fps) video streams, and the hardware can maximally encode 1080p and 30fps video streams, and then on the mobile phone, a 1-path software encoder and a 1-path hardware encoder can be created, and the maximum can encode 1080p and 60fps video streams.

Original image sequence 235: a set of images consisting of images that do not change much over a continuous period of time is called a sequence, where 1 to 6 are the numbers of the images.

Software encoder 240: an encoder for video encoding using a CPU. It should be understood that the number of software encoders in the acquisition end 210 is related to the computational effort of the device, the resolution of the acquired image, and the frame rate, and that for convenience only one software encoder is shown in fig. 2, this illustration should not be construed as limiting the application.

Hardware encoder 250: the encoder for video coding by using hardware generally has a special hardware module on the device, and generally occupies no or little CPU when the hardware is coded. It should be understood that the number of hardware encoders in the acquisition end 210 is related to the computational effort of the device, the resolution of the acquired image, and the frame rate, and that for convenience only one hardware encoder is shown in fig. 2, this illustration should not be construed as limiting the application.

Software decoder 260: a decoder for video decoding using a CPU.

Hardware decoder 270: decoders that use hardware for video decoding typically have dedicated hardware modules on the device that typically do not occupy or occupy very little CPU during hardware decoding.

The display unit 180: for displaying the image, the image sent from the decoding unit 170 is displayed on a display screen.

Recombinant image sequence 285: and reconstructing the image sequence according to the code stream composition rule.

The playing end 220 further includes a decoding scheduling module (not shown in the figure), where the decoding scheduling module is configured to create and manage a decoder according to a composition rule of the code stream, and the decoder includes two types of software decoders and hardware decoders. Alternatively, the decode scheduling module may create either one or more of a software decoder or a hardware decoder, the two decoders included in FIG. 2 are for illustration only and should not be construed as limiting the application.

An image in video may also be referred to as a frame, I in software encoder 240, hardware encoder 250, software decoder 260, and hardware decoder 270 refers to an I frame, and P refers to a P frame.

As an example, the encoding rules in embodiments of the present application are that odd frames in the original image sequence 235 are encoded by the software encoder 240 and even frames are encoded by the hardware encoder 250. Illustratively, the images numbered 1,3, 5 are encoded by software encoder 240 and the encoders numbered 2,4, 6 are encoded by hardware encoder 250.

Alternatively, a person skilled in the art may set different encoding rules according to the requirements, and the present application is not limited to a specific video encoding rule. Other coding rules are used on the basis of the video coding and decoding method provided by the application, and the protection scope of the application is also provided.

410, A sequence of images is acquired.

The image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, wherein the video stream comprises a plurality of images which are continuous in time, and K is an integer greater than or equal to 1.

It should be appreciated that the preset image resolution and frame rate may be related to the capabilities of the encoding device. For example, if the software can maximally encode 1080p and 30fps video streams and the hardware can maximally encode 1080p and 30fps video streams on one mobile phone, the resolution of the acquired image can be set to 1080p and the frame rate can be set to 60fps.

420, A first encoding rule is obtained.

The first encoding rule is used for indicating the corresponding relation between Q encoders and K images, each encoder in the Q encoders is used for processing the corresponding image, wherein the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1.

Fig. 3 is an example of a first encoding rule. The Q encoders include a software encoder and a hardware encoder. Odd frames in the image sequence are encoded by a software encoder and even frames are encoded by a hardware encoder. Illustratively, the images numbered 1, 3, 5 … … are encoded by a software encoder and the images numbered 2, 4, 6 … … are encoded by a hardware encoder. Alternatively, it is also possible to provide that even frames in the image sequence are encoded by a software encoder and that odd frames are encoded by a hardware encoder.

430, Encoding the image sequence by N software encoders and M hardware encoders.

The Q encoders may encode the image sequence according to a first encoding rule to generate a Q-way code stream, and a qth encoder in the Q encoders encodes an image corresponding to the qth encoder in the image sequence, Q e [1, Q ]. The Q-way code stream corresponds to N software encoders and M hardware encoders one by one.

510, Encoder resources are calculated.

The acquisition of one or more images in the video stream is started at a preset image resolution and frame rate to form an image sequence 235. It should be appreciated that the preset image resolution and frame rate may be related to the capabilities of the encoding device. For example, if the software can maximally encode 1080p and 30fps video streams and the hardware can maximally encode 1080p and 30fps video streams on one mobile phone, the resolution of the acquired image can be set to 1080p and the frame rate can be set to 60fps.

The encoding scheduling module 230 calculates the required encoder resources based on the resolution, frame rate and computational power of the device for the images in the acquired sequence of original images 235. The embodiment of the application can fully utilize the software and hardware resources of the equipment, and the software and the hardware are cooperatively coded on the image level, so that the frame rate can be improved by at least one time.

A software encoder is created 520.

The code scheduling module 230 creates at least one software encoder from the calculated encoder resources.

At 530, a hardware encoder is created.

The encoding scheduling module 230 creates at least one hardware encoder from the calculated encoder resources. Taking the example of the code scheduling module 230 creating 1 software encoder and 1 hardware encoder, the two encoders need to set the same coding parameters, including, but not limited to, coding format (h.264, h.265, etc.), pixel format (yuv 420p, etc.), bit depth, resolution, frame rate, code rate, I-frame interval, B-frame interval, maximum reference frame number, slice mode (single slice or multiple slices), profile type (base profile, main profile, etc.), encapsulation format (rtmp, hls, etc.).

540, Creating an encoding rule.

As shown in fig. 3, the encoding rule in the embodiment of the present application is that the odd frames are encoded by the software encoder 240 and the even frames are encoded by the hardware encoder 250. Alternatively, it is also possible to set that the odd frames are encoded by the hardware encoder 250 and the even frames are encoded by the software encoder 240.

550, Cyclic encoding.

The software encoder 240 and the hardware encoder 250 loop through encoding the corresponding images in the sequence of images.

560, Acquiring an image.

The images in the video stream are continuously collected according to the preset image resolution and frame rate to form an image sequence, and the image sequence is sent to the coding scheduling module 230 for coding.

570, The encoding rules are executed.

The code scheduling module 230 invokes the interfaces of the software encoder 240 and the hardware encoder 250, respectively, to code according to the coding rules. After the image is encoded by the software encoder 240 and the hardware encoder 250, the image is encapsulated in a designated encapsulation format, such as rtmp, hls, etc., and the encoded live code stream is uploaded to the transmission unit 150. The transmission unit 150 buffers the live code stream.

The method provided by the application can also be extended to a plurality of devices or a plurality of hardware for parallel coding. In some possible implementations, when encoding an image with a super resolution, the method provided by the embodiment of the application may divide the image first, and then use a software encoder and a hardware encoder to encode the divided sub-images in parallel. For example, when the resolution of the image is greater than a preset threshold, the image is divided into Y sub-images, the corresponding sub-images in the Y sub-images are encoded by Y encoders, the Y encoders are any Y of the Q encoders, the Y sub-images are in one-to-one correspondence with the Y encoders, and Y is an integer greater than or equal to 2 and less than or equal to Q.

A decoder is created 610.

The decoding scheduling module creates a software encoder and/or a hardware encoder according to the composition rule of the code stream. The decoding and scheduling module determines a composition rule of the code stream according to the content description about the transmission code stream in the transmission unit 150, for example, if the code stream output by the acquisition end 110 includes a first path code stream and a second path code stream, the decoding and scheduling module may create a software encoder and/or a hardware encoder to decode the first path code stream and the second path code stream simultaneously. For example, the decoding scheduling module may create one software encoder and one hardware encoder to decode the first and second path code streams, respectively, or in case of sufficient computational power at the decoding end, the decoding scheduling module may create two software encoders to decode the first and second path code streams, respectively, or the decoding scheduling module may create two hardware encoders to decode the first and second path code streams, respectively.

The simultaneous creation of a software decoder and creation of a hardware decoder in fig. 6 is only an example, and only one of them may be created, and the illustration should not be construed as limiting the application.

In some possible application scenarios, the decoding scheduling module may also create only one software encoder to decode the first path code stream and not the second path code stream. Or the decoding scheduling module may create only one hardware encoder to decode the second path code stream and not decode the first path code stream.

620, The software decoder downloads the first stream according to the live play URL.

Taking the example of the decoding scheduling module creating a software encoder and a hardware decoder to decode the first and second streams respectively, the software decoder 260 downloads the first stream for subsequent decoding according to the uniform resource locator (uniform resource locator, URL) for live broadcast.

The hardware decoder downloads the second stream according to the live play URL 630.

The hardware decoder 270 downloads the second stream of codes for subsequent decoding according to the URL of the live broadcast. It should be understood that steps 620 and 630 may be performed simultaneously in embodiments of the present application, and the serial numbers should not be construed as limiting the application.

621, The software decoder decodes the first path code stream frame by frame.

After decoding the first path code stream frame by frame, the software decoder 260 sends the decoded image and the presentation time stamp (presentation timestamp, PTS) of the image to the display unit 180.

631, The hardware decoder decodes the second way code stream frame by frame.

After decoding the second code stream frame by frame, the hardware decoder 270 sends the decoded image and the PTS of the image to the display unit 180.

It should be appreciated that the software decoder 260 and the hardware decoder 270 may decode the 2-way code stream in parallel, i.e., 621 and 631 steps may be performed simultaneously.

640, Displayed in PTS timing.

The display module of the playing end displays the images according to the PTS sequence.

It should be understood that, in the method described in the above embodiment, only the original image sequence is taken as an example to form 2 paths of streams, the process of forming R paths of streams by the original image sequence is similar to that of 2 paths of streams, and multiple software encoders or multiple hardware encoders may be created.

The video coding and decoding method provided by the embodiment of the application can simultaneously create a software coder and a hardware coder at the coding end, and alternately send the image sequence to the software coder and the hardware coder; at the decoding end, a software decoder and a hardware decoder can be simultaneously created, and the images are rendered in sequence after decoding. The frame rate is improved end to end by combining software and hardware, so that the CPU occupation rate is not increased basically, and the frame rate can be doubled.

The video encoding method and the video encoding method according to the embodiments of the present application are described in detail above with reference to fig. 1 to 6, the video encoder according to the embodiments of the present application is described below with reference to fig. 7, the video decoder shown in fig. 7 is capable of performing the respective steps in the video encoding method according to the embodiments of the present application, the above-mentioned limitations of the video encoder method according to the embodiments of the present application are equally applicable to the video encoder shown in fig. 7, and the description of the video encoder apparatus according to the embodiments of the present application is omitted when the description is given below to avoid unnecessary repetition.

Fig. 7 is a schematic block diagram of a video encoder provided by an embodiment of the present application. The video encoder 700 shown in fig. 7 includes: an acquisition unit 710, configured to acquire an image sequence, where the image sequence includes K images in a video stream acquired according to a preset image resolution and frame rate, where the video stream includes a plurality of images that are consecutive in time, and K is an integer greater than or equal to 1; an encoding unit 720, configured to obtain a first encoding rule, where the first encoding rule is used to indicate a correspondence between Q encoders and the K images, each of the Q encoders is used to process the corresponding image, and the Q encoders include N software encoders and M hardware encoders, where N and M are integers greater than or equal to 1; the encoding unit 720 is further configured to encode, according to the first encoding rule, a picture corresponding to a qth encoder in the sequence of pictures by the qth encoder in the qth encoder, where qe [1, Q ].

Optionally, as an embodiment, the encoding unit 720 is further configured to create the Q encoders according to a computing capability of a computer device, the image resolution, and the frame rate.

Optionally, as an embodiment, the coding parameters of the Q encoders are the same.

Optionally, as an embodiment, the video stream is a live video stream.

Optionally, as an embodiment, when the resolution of the first image is greater than a preset threshold, the encoding unit 720 segments the first image into Y sub-images, where the first image is any one image in the image sequence, Y e [2, q ]; the encoding unit 720 is further configured to obtain a second encoding rule, where the second encoding rule is used to indicate a correspondence between Y encoders and the Y sub-images, the Y encoders are any Y encoders in the Q encoders, and the Y sub-images are in one-to-one correspondence with the Y encoders; the encoding unit 720 is further configured to encode, according to the second encoding rule, a sub-image corresponding to a Y-th encoder from the Y sub-images by a Y-th encoder from the Y encoders, where Y is e [1, Y ].

It should be appreciated that the video encoder 700 herein is embodied in the form of functional units. The term "unit" herein may refer to an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (e.g., a shared, dedicated, or group processor, etc.) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that support the described functionality. In an alternative example, it will be appreciated by those skilled in the art that the video encoder 700 may be configured to perform the respective processes and/or steps corresponding to the video encoding method in the above method embodiments, and are not repeated herein.

Fig. 8 is a schematic block diagram of a video decoder according to an embodiment of the present application. The video decoder 800 shown in fig. 8 includes: a receiving unit 810, configured to obtain a Q-way code stream, where the Q-way code stream includes a code stream generated by encoding, by Q encoders, corresponding images among K images in a video stream according to a first encoding rule, where the first encoding rule is used to indicate a correspondence between the Q encoders and the K images, the video stream includes a plurality of images that are continuous in time, the Q encoders include N software encoders and M hardware encoders, K is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1; a decoding unit 820, configured to decode the Q-way code stream by Q decoders, to obtain a plurality of decoded images, where the Q decoders are in one-to-one correspondence with the Q-way code stream; and a display unit 830 for displaying the plurality of images according to their time sequence.

Optionally, as an embodiment, the decoding unit 820 is further configured to create the Q decoders according to the Q-way code stream.

Optionally, as an embodiment, decoding parameters of the Q decoders are the same.

Optionally, as an embodiment, the Q decoders include at least one hardware decoder and at least one software decoder.

Optionally, as an embodiment, the receiving unit 810 is further configured to download the Q-way code stream according to a URL of a live video stream.

Optionally, as an embodiment, the display unit 830 is further configured to display the plurality of images according to an order of presentation time stamps of the plurality of images.

It should be appreciated that the video decoder 800 herein is embodied in the form of functional units. The term "unit" herein may refer to an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (e.g., a shared, dedicated, or group processor, etc.) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that support the described functionality. In an alternative example, it will be appreciated by those skilled in the art that the video decoder 800 may be configured to perform the respective processes and/or steps corresponding to the video decoding method in the above method embodiment, and are not repeated herein.

Fig. 9 is a schematic block diagram of another video encoder 900 provided by an embodiment of the present application. As shown in fig. 9, the video encoder 900 may perform various processes and/or steps corresponding to the video encoding method in the above-described method embodiments.

The video encoder 900 may include a processor 910 and a memory 920. The memory 920 is configured to store instructions, and the processor 910 is configured to execute the instructions stored in the memory 920, so that the video encoder 900 implements respective processes and/or steps corresponding to the video encoding methods shown in fig. 4 and 5, and are not described herein for avoiding repetition.

Further, the video encoder 900 may also include an input 930 and an output 940. Further, the processor 910, memory 920, input 930, and output 940 may communicate with each other via internal connection paths to transfer control and/or data signals. The memory 920 is used for storing a computer program, which the processor 910 may use to call and run from the memory 920. The memory 920 may be integrated into the processor 910 or may be separate from the processor 910.

Alternatively, if the video encoder is a chip or a circuit, the input port 930 is an input interface, and the output port 940 is an output interface.

As an implementation, the functions of the input port 930 and the output port 940 may be considered to be implemented by a transceiver circuit or a dedicated chip for transceiving. Processor 910 may be considered to be implemented by a special purpose processing chip, a processing circuit, a processor, or a general purpose chip.

Alternatively, if the video encoder 900 is a chip or a circuit, the video encoder 900 may not include the memory 920, and the processor 910 may read the instructions in the memory outside the chip.

As another implementation, a manner of implementing the video encoder 900 provided by the embodiment of the present application using a general-purpose computer may be considered. I.e., program code that implements the functions of the processor 910, input 930, and output 940 are stored in the memory 920. A general purpose processor implements the functions of the processor 910, input 930, and output 940 by executing the code in the memory 920.

The processor 910 is mainly configured to process communication data and control the entire video encoder 900, execute a software program, and process the data of the software program, for example, encode, by a qth encoder of the Q encoders, an image corresponding to the qth encoder in the image sequence according to the first encoding rule, where Q is e [1, Q ].

In an embodiment of the present application, fig. 9 may be a schematic diagram of a video encoder 900. May be used to implement the functions of the video encoder in the method described above. The processor 910 may perform the functions of the encoding unit 720 in the video encoder 700 in fig. 7, which is not limited in this regard.

Those skilled in the art will appreciate that fig. 9 shows only one memory 920 and processor 910 for ease of illustration. In an actual video encoder 900, there may be multiple processors 910 and memories 920. Memory 920 may also be referred to as a storage medium or storage device, etc., and embodiments of the present application are not limited in this regard.

Fig. 10 is a schematic block diagram of another video decoder 1000 provided by an embodiment of the present application. As shown in fig. 10, the video decoder 1000 may be a video decoder, or may be a chip or a circuit, such as a chip or a circuit that may be disposed in the video decoder. The video decoder 1000 may perform the respective processes and/or steps corresponding to the video decoding method in the above-described method embodiment.

The video decoder 1000 may include a processor 1010 and a memory 1020. The memory 1020 is configured to store instructions, and the processor 1010 is configured to execute the instructions stored in the memory 1020, so that the video decoder 1000 implements the respective processes and/or steps corresponding to the video decoding method shown in fig. 6, and the repetition is avoided, which is not repeated herein.

Further, the video decoder 1000 may also include an input port 1030 and an output port 1040. Still further, the processor 1010, memory 1020, input 1030, and output 1040 may communicate with each other via internal connection paths to transfer control and/or data signals. The memory 1020 is used for storing a computer program, and the processor 1010 may be used to call and run the computer program from the memory 1020 to control the input port 1030 to receive data and the output port 1040 to transmit data. The memory 1020 may be integrated into the processor 1010 or may be separate from the processor 1010.

Alternatively, if the video decoder 1000 is a chip or a circuit, the input port 1030 is an input interface and the output port 1040 is an output interface.

Alternatively, if the video decoder 1000 is a chip or a circuit, the video decoder 1000 may not include the memory 1020, and the processor 1010 may read the instructions in the memory outside the chip.

As one implementation, the functions of input port 1030 and output port 1040 may be considered to be implemented by transceiving circuitry or dedicated chips for transceiving. The processor 1010 may be considered to be implemented by a dedicated processing chip, a processing circuit, a processor, or a general-purpose chip.

As another implementation manner, the video decoder 1000 provided by the embodiment of the present application may be considered to be implemented by using a general-purpose computer. I.e., program code that implements the functions of the processor 1010, input 1030, and output 1040 are stored in memory, and a general purpose processor implements the functions of the processor 1010, input 1030, and output 1040 by executing code in memory.

The processor 1010 is mainly configured to process communication data, control the entire video decoder 1000, execute a software program, and process data of the software program, for example, decode a Q-way code stream by Q decoders, to obtain decoded multiple images.

In an embodiment of the present application, fig. 10 may be a schematic diagram of a structure of a video decoder 1000. May be used to implement the functions of the video decoder in the above method. The processor 1010 may perform the function of the decoding unit 820 in the video decoder 800 in fig. 8, which is not limited in this regard.

Those skilled in the art will appreciate that for ease of illustration, fig. 10 shows only one memory 1020 and processor 1010. In an actual video decoder 1000, there may be multiple processors 1010 and memories 1020. Memory 1020 may also be referred to as a storage medium or storage device or the like, as embodiments of the application are not limited in this regard.

The video encoding and decoding method in the embodiment of the present application described above may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general purpose processor, a digital signal processor (DIGITAL SIGNAL processor, DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (field programmable GATE ARRAY, FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, a system on chip (SoC), a central processor (central processor unit, CPU), a network processor (network processor, NP), a digital signal processing circuit (DIGITAL SIGNAL processor, DSP), a microcontroller (micro controller unit, MCU), a programmable controller (programmable logic device, PLD) or other integrated chip. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a memory medium well known in the art such as random access memory (random access memory, RAM), flash memory, read-only memory (ROM), programmable read-only memory (ROM), or electrically erasable programmable memory (eeprom), registers, etc. The storage medium is located in a memory, and the processor reads instructions from the memory and, in combination with its hardware, performs the steps of the method described above.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video encoding method, comprising:

acquiring an image sequence, wherein the image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, the video stream comprises a plurality of images which are continuous in time, and K is an integer greater than or equal to 1;

Acquiring a first coding rule, wherein the first coding rule is used for indicating the corresponding relation between Q encoders and the K images, each encoder in the Q encoders is used for processing the corresponding image, the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1;

and according to the first coding rule, coding the image corresponding to the qth coder in the image sequence by the qth coder in the qth coder, wherein Q is [1, Q ].

2. The method of claim 1, wherein prior to the obtaining the first encoding rule, the method further comprises:

The Q encoders are created according to the computing power of the computer device, the image resolution and the frame rate.

3. The method according to claim 1 or 2, wherein the coding parameters of the Q encoders are the same.

4. A method according to any one of claims 1 to 3, wherein the video stream is a live video stream.

5. The method according to any one of claims 1 to 4, further comprising:

When the resolution of the first image is larger than a preset threshold, dividing the first image into Y sub-images, wherein the first image is any one image in the image sequence, and Y is E [2, Q ];

acquiring a second coding rule, wherein the second coding rule is used for indicating the corresponding relation between Y encoders and Y sub-images, the Y encoders are any Y encoders in the Q encoders, and the Y sub-images are in one-to-one correspondence with the Y encoders;

and according to the second coding rule, coding the sub-image corresponding to the Y encoder in the Y sub-images through the Y encoder in the Y encoders, wherein Y is [1, Y ].

6. A video decoding method, comprising:

Acquiring a Q-channel code stream, wherein the Q-channel code stream comprises a code stream generated after Q encoders encode corresponding images in K images in a video stream according to a first encoding rule, the first encoding rule is used for indicating the corresponding relation between the Q encoders and the K images, the video stream comprises a plurality of images which are continuous in time, the Q encoders comprise N software encoders and M hardware encoders, K is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1;

Decoding the Q-path code stream through Q decoders to obtain a plurality of decoded images, wherein the Q decoders are in one-to-one correspondence with the Q-path code stream;

the plurality of images are displayed according to a temporal order of the plurality of images.

7. The method of claim 6, wherein prior to decoding the Q-way code stream by Q decoders, the method further comprises:

And creating the Q decoders according to the Q-channel code stream.

8. The method according to claim 6 or 7, wherein the decoding parameters of the Q decoders are the same.

9. The method according to any one of claims 6 to 8, wherein the Q decoders comprise at least one hardware decoder and at least one software decoder.

10. The method according to any one of claims 6 to 9, wherein the acquiring the Q-way code stream comprises:

and downloading the Q-channel code stream according to the uniform resource locator URL of the live video stream.

11. The method according to any one of claims 6 to 10, wherein said displaying the plurality of images according to a temporal order of the plurality of images comprises:

displaying the plurality of images according to the order of presentation time stamps of the plurality of images.

12. A video encoder, comprising:

The image acquisition unit is used for acquiring an image sequence, wherein the image sequence comprises K images in a video stream acquired according to preset image resolution and frame rate, the video stream comprises a plurality of images which are continuous in time, and K is an integer greater than or equal to 1;

The encoding unit is used for acquiring a first encoding rule, wherein the first encoding rule is used for indicating the corresponding relation between Q encoders and the K images, each encoder in the Q encoders is used for processing the corresponding image, the Q encoders comprise N software encoders and M hardware encoders, and N and M are integers which are larger than or equal to 1;

The coding unit is further configured to code, according to the first coding rule, a picture corresponding to a qth encoder in the picture sequence by the qth encoder in the qth encoder, where Q is e [1, Q ].

13. The video encoder of claim 12, wherein the encoding unit is further configured to create the Q encoders based on a computing power of a computer device, the image resolution, and the frame rate.

14. The video encoder of claim 12 or 13, wherein the encoding parameters of the Q encoders are the same.

15. The video encoder of any of claims 12 to 14, wherein the video stream is a live video stream.

16. The video encoder according to any of claims 12 to 15, wherein the encoding unit is further configured to divide a first image into Y sub-images when a resolution of the first image is greater than a preset threshold, the first image being any one of the image sequences, Y e [2, q ];

The coding unit is further configured to obtain a second coding rule, where the second coding rule is used to indicate a correspondence between Y encoders and the Y sub-images, the Y encoders are any Y encoders in the Q encoders, and the Y sub-images are in one-to-one correspondence with the Y encoders;

the coding unit is further configured to code, according to the second coding rule, a sub-image corresponding to a Y-th encoder in the Y sub-images by a Y-th encoder in the Y encoders, where Y is e [1, Y ].

17. A video decoder, comprising:

The receiving unit is used for obtaining a Q-channel code stream, wherein the Q-channel code stream comprises a code stream generated after Q encoders encode corresponding images in K images in a video stream according to a first encoding rule, the first encoding rule is used for indicating the corresponding relation between the Q encoders and the K images, the video stream comprises a plurality of images which are continuous in time, the Q encoders comprise N software encoders and M hardware encoders, K is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1;

The decoding unit is used for decoding the Q-path code streams through Q decoders to obtain a plurality of decoded images, and the Q decoders are in one-to-one correspondence with the Q-path code streams;

and a display unit for displaying the plurality of images according to a time sequence of the plurality of images.

18. The video decoder of claim 17, characterized in that said decoding unit is further configured to create said Q decoders from said Q-way code stream.

19. The video decoder of claim 17 or 18, characterized in that the decoding parameters of the Q decoders are identical.

20. The video decoder of any of claims 17 to 19, characterized in that the Q decoders include at least one hardware decoder and at least one software decoder.

21. The video decoder of any of claims 17 to 20, characterized in that the streaming unit is further configured to download the Q-way code stream according to a uniform resource locator URL of a live video stream.

22. The video decoder of any of claims 17 to 21, characterized in that the display unit is further configured to display the plurality of images according to an order of presentation time stamps of the plurality of images.

23. A video codec device, comprising:

a memory;

A processor invoking program code stored in the memory to perform the method of any of claims 1-11.