WO2020237466A1

WO2020237466A1 - Video transmission method and apparatus, and aircraft, playback device, and storage medium

Info

Publication number: WO2020237466A1
Application number: PCT/CN2019/088596
Authority: WO
Inventors: 赵亮; 周士贞; 陈颖
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-05-27
Filing date: 2019-05-27
Publication date: 2020-12-03
Also published as: CN111758262A

Abstract

Disclosed are a video transmission method and apparatus, and an aircraft, a playback device, and a storage medium. The method comprises: obtaining the current image frame, encoding the current image frame, and then sending the encoded current image frame to a receiving terminal (S110); if error feedback information sent by the receiving terminal is received, determining an error correction refreshing frame according to the current image frame (S120); and performing intra-frame encoding on a partial region of the error correction refreshing frame, and sending the encoded error correction refreshing frame to the receiving terminal (S130), so as to enable a transmission error to recover faster.

Description

Video transmission method, device, aircraft, playback equipment and storage medium

Technical field

This application relates to the field of video transmission technology, and in particular to a video transmission method, device, aircraft, playback device, and storage medium.

Background technique

At present, video transmissions that require high real-time performance usually inter-encode more video frames into prediction frames (P frames) to improve channel utilization and have clearer picture quality under the same channel conditions. .

The coding and decoding of the predicted frame needs to refer to the information of other video frames. In the prior art, when the receiving end fails to decode the predicted frame, the sending end re-encodes a complete key frame (I frame) and sends it to the receiving end, and the receiving end decodes the key frame. Frame gets new reference information to make the subsequent prediction frame decode correctly; but because the compression efficiency of the key frame is relatively low, it is easy to have a burst of coding rate, sending and receiving the key frame takes more time, which affects the subsequent video For the transmission and reception of frames, there are often increased delays and freezes near the key frame; therefore, this method is not very friendly to the wireless transmission physical channel, especially in scenarios where the channel conditions are not very good or the channel changes quickly. The image transmission delay and the feeling of freezing will be more obvious.

Although there are technologies that periodically perform Gradual Decoder Refresh (GDR) decoding of the reference information needed, if the reference information needs to be updated in time to ensure that the predicted frame is decoded correctly, the refresh cycle needs to be shortened, thereby encoding redundant information Increased, channel utilization will also decrease, and image quality at the same bit rate will decrease; and when the refresh period increases, the reference information will not be updated in time, and the error recovery speed will be slower after the prediction frame decodes errors.

Summary of the invention

Based on this, this application provides a video transmission method, device, aircraft, playback device, and storage medium, aiming to solve the problem of encoding rate bursts when video frame receiving and decoding errors occur in the existing video transmission process. , There are technical problems such as increased delay and freeze, which affect the transmission of video frames.

In the first aspect, this application provides a video sending method applied to the sending end, including:

Acquiring a current image frame, encoding the current image frame, and sending the encoded current image frame to the receiving end;

If error feedback information sent by the receiving end is received, determine an error correction refresh frame according to the current image frame;

Perform intra-frame coding on a part of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.

In the second aspect, this application provides a video receiving method applied to the receiving end, including:

Receiving the encoded current image frame from the sending end, and decoding the current image frame;

If there is an error in receiving the current image frame or the decoding of the current image frame, error feedback information is sent to the sending end, and the receiving end performs error correction of intra-frame coding on the partial area sent by the sending end in response to the error feedback information Refresh frame

The error correction refresh frame is decoded, and the reference data used for decoding is refreshed according to the decoding result.

In the third aspect, this application provides a video transmission method, including:

The sending end encodes the acquired current image frame, and sends the encoded current image frame to the receiving end;

The receiving end receives and decodes the current image frame, and if there is an error in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end;

The sending end determines an error correction refresh frame according to the error feedback information, performs intra-frame coding on a part of the error correction refresh frame, and sends the coded error correction refresh frame to the receiving end;

The receiving end decodes the error correction refresh frame, and refreshes the reference data used for decoding according to the decoding result.

In a fourth aspect, the present application provides a video sending device, including a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program, and when executing the computer program, implement the following steps:

Perform intra-frame coding on a partial area of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.

In the fifth aspect, this application provides an aircraft, including:

Image acquisition component, used to acquire images;

The aforementioned video sending device is used to acquire the image collected by the image collection component, and send it to the receiving end after encoding;

Flight components, used for flight.

In a sixth aspect, the present application provides a video receiving device, including a memory and a processor;

The memory is used to store computer programs;

In a seventh aspect, this application provides a video playback device, including:

The aforementioned video receiving device is used to receive images from the sending end and decode them;

The display component is used to display the data decoded by the video receiving device.

In an eighth aspect, the present application provides a computer-readable storage medium that stores a computer program that, when executed by a processor, causes the processor to implement the aforementioned video sending method; or

Implement the aforementioned video receiving method.

The embodiments of the application provide a video transmission method, device, aircraft, playback device, and storage medium. By performing intra-frame coding on part of the error correction refresh frame when a certain image frame is transmitted incorrectly, the network resources can be reduced. Resource occupancy, avoiding the burst of code rate caused by intra-encoding of the complete image frame and increasing delay and jitter, and can provide the receiving end with reference data required for decoding in time, so that transmission errors can be recovered quickly.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a video sending method provided by an embodiment of the present application;

Fig. 2 is a schematic diagram of a scene in which the sender sends image frames to the receiver;

3 to 5 are schematic flowcharts of other implementation manners of the video sending method of this application;

Fig. 6 and Fig. 7 are schematic diagrams of the intra-frame coding process of part of the error correction refresh frame in Fig. 1;

8 and 9 are schematic flowcharts of other implementation manners of the video sending method of this application;

Fig. 10 and Fig. 11 are schematic diagrams of the intra-frame coding process of part of the error correction refresh frame in Fig. 1;

FIG. 12 is a schematic flow chart of intra-encoding a part of the error correction refresh frame in FIG. 1;

FIG. 13 is a schematic diagram of determining an intra-frame coding region according to region index information in error feedback information;

Fig. 14 and Fig. 15 are schematic diagrams of the intra-frame coding process of part of the error correction refresh frame in Fig. 1;

16 and 17 are schematic diagrams of the flow of encoding the current image frame in FIG. 1;

FIG. 18 is a schematic diagram of performing intra-frame coding on interval refresh frames in a refresh period;

Fig. 19 is a schematic diagram of performing intra-frame coding on a partial area of an interval refresh frame in a refresh period;

20 is a schematic flowchart of another implementation manner of a video sending method according to the present application;

FIG. 21 is a schematic diagram of performing intra-frame encoding on one image frame in FIG. 20 and performing inter-frame encoding on subsequent image frames;

FIG. 22 is a schematic flowchart of a video receiving method provided by an embodiment of the present application;

Fig. 23 and Fig. 24 are schematic diagrams of the flow of decoding the current image frame in Fig. 22;

25 is a schematic diagram of the flow of decoding error correction refresh frame and refreshing reference data in FIG. 22;

FIG. 26 is a schematic flowchart of a video transmission method provided by an embodiment of the present application;

FIG. 27 is a schematic diagram of image frame transmission in an embodiment of a video transmission method.

FIG. 28 is a schematic block diagram of a video sending device according to an embodiment of the present application;

FIG. 29 is a schematic block diagram of an aircraft according to an embodiment of the present application;

FIG. 30 is a schematic block diagram of a video receiving device according to an embodiment of the present application;

FIG. 31 is a schematic block diagram of a video playback device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is merely an illustration, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a video sending method for the sending end according to an embodiment of the present application. The video sending method can be applied to a terminal device or a server for sending video to the receiving end, etc. Process: The terminal device can be a mobile phone, tablet computer, notebook computer, desktop computer, personal digital assistant, wearable device or drone, etc. The server can be an independent server or a server cluster. Furthermore, the UAV can be a rotary-wing UAV, such as a quadrotor UAV, a hexarotor UAV, an eight-rotor UAV, or a fixed-wing UAV, and the UAV has Camera equipment.

As shown in FIG. 1, the video sending method of this embodiment includes steps S110 to S130.

S110. Obtain a current image frame, encode the current image frame, and send the encoded current image frame to the receiving end.

In this embodiment, the sending end sends image frames to the receiving end frame by frame, as shown in FIG. 2; exemplary, the camera on the sending end acquires image frames in real time, encodes the acquired image frames frame by frame, and sends them to the receiving end .

For example, the current image frame is an image frame acquired in real time by a camera set on the sending end, which represents an image frame acquired and to be sent by the sending end at the current moment.

Exemplarily, the sending end performs interframe encoding on the current image frame to obtain a predictive-coded picture frame (Predictive-coded Picture), that is, a P frame; by referring to the previous image frame, the interframe is performed by motion prediction Predictive coding.

As shown in Figure 2, the sending end sequentially takes the second image frame to the n-1th image frame as the current image frame, performs inter-frame coding to obtain the P frame and then sends it to the receiving end.

For example, refer to the first image frame and perform inter-frame coding on the second image frame; refer to the first image frame or the second image frame and perform inter-frame coding on the third image frame.

S120: If error feedback information sent by the receiving end is received, determine an error correction refresh frame according to the current image frame.

In this embodiment, the sending end sends the encoded image frame to the receiving end, and if the receiving end receives an error in an image frame or the decoded image frame has an error, it sends error feedback information to the sending end.

Exemplarily, after receiving and decoding any image frame, the receiving end sends success feedback information to the sending end, and decodes the image frame; the sending end obtains a new image frame after receiving the success feedback information, performs encoding and sends it to the receiving end; If the receiving end receives a certain image frame error or the decoded image frame has an error, it sends error feedback information to the transmitting end.

Exemplarily, the receiving end sends success feedback information and error feedback information to the sending end through an uplink channel between the receiving end and the sending end. For example, the uplink channel can also be used to transmit flight control instructions to the sending end, such as drones, and various control instructions, such as taking pictures, videos, and returning to home.

Exemplarily, the error feedback information includes the feedback information of the receiving end in response to the packet loss event, or includes the feedback information of the receiving end in response to an integrity check error.

When sending an image frame to the receiving end, the sending end receives error feedback information sent by the receiving end, which indicates that the image frame has failed to be sent or decoded, and the error correction refresh frame is determined according to the image frame corresponding to the error feedback information.

S130. Perform intra-frame coding on a partial area of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.

Different from performing intraframe (Intraframe) encoding on the entire image frame in the prior art, in this embodiment, the partial area of the error correction refresh frame is intraframe encoded, and the encoded error correction refresh frame is sent to the receiving end send. As shown in FIG. 2, intra-frame coding is performed on a partial region from the nth image frame to the n+2th image frame.

When performing intra-frame coding, there is no need to refer to other image frames. It can completely describe the image background and moving subject information of the image frame or part of the image frame. The data volume after intra-frame coding is relatively large, which is occupied when transmitting to the receiving end. There are a lot of network resources; when the receiving end is decoding an intra-encoded image frame or part of an image frame, it can decode this part of the data without referring to other image frames, and the decoded data is required when decoding other image frames Reference data.

The video transmission method provided in this embodiment performs intra-frame coding on part of the error correction refresh frame when a certain image frame is transmitted incorrectly, which can reduce the resource occupation of network resources and avoid the intra-frame coding of the complete image frame. The burst of code rate causes delay and jitter to increase, and can provide the receiving end with reference data required for decoding in time, so that transmission errors can be recovered quickly.

In some embodiments, as shown in FIG. 3, performing intra-frame coding on a partial area of the error correction refresh frame in step S130, and sending the coded error correction refresh frame to the receiving end includes:

S1301: Perform intra-frame coding on a partial area of the error correction refresh frame, perform inter-frame coding on the remaining area of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.

The data volume after inter-frame encoding is relatively small, and the network resources occupied when transmitting to the receiving end is relatively small; therefore, the data volume of the error correction refresh frame after encoding can be small, and it includes the area of intra-frame encoding, which can be used at the receiving end Refresh the reference data.

In some embodiments, as shown in FIG. 4, if error feedback information sent by the receiving end is received in step S120, determining an error correction refresh frame according to the current image frame includes:

S121: If error feedback information sent by the receiving end is received, determine a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame.

If the error feedback information sent by the receiving end is received, a preset number of error correction refresh frames are determined in the image frame corresponding to the error feedback information, that is, the current image frame and/or the image frames subsequent to the current image frame.

For example, the sending end obtains the nth image frame in S110, encodes the image frame, and sends it to the receiving end; but receiving error feedback information sent by the receiving end in S120, it can be used in the nth image frame and/or The preset number is determined in the image frames after n image frames, such as M error correction refresh frames, and n and M are natural numbers that are not 0. As shown in Figure 2, M is equal to 3.

Exemplarily, as shown in FIG. 5, step S121 determining a preset number of error correction refresh frames in the current image frame and/or the image frames subsequent to the current image frame includes:

S1201. Use a preset number of image frames starting from the current image frame as error correction refresh frames.

For example, as shown in FIG. 2, M image frames starting from the nth image frame, that is, the nth image frame to the n+M-1th image frame are determined as M error correction refresh frames. To respond more quickly to the error feedback information from the receiving end, send an error correction refresh frame to the receiving end.

In some embodiments, as shown in FIG. 6, step S130 performs intra-frame coding on a part of the error correction refresh frame, and sends the coded error correction refresh frame to the receiving end, including steps S131 to S134.

S131. Use the current image frame as the first error correction refresh frame, perform intra-frame encoding on a partial area of the first error correction refresh frame, and send the encoded first error correction refresh frame to the receiving Send at the end.

In this embodiment, a preset number of image frames starting from the current image frame corresponding to the error feedback information are used as error correction refresh frames.

Exemplarily, the receiving end makes an error when receiving and decoding the nth image frame, and sends error feedback information to the sending end, and the sending end determines the nth image frame to the n+M-1th image frame as M errors. Error refresh frame; the first error correction refresh frame is the current image frame corresponding to the error feedback information, that is, the nth image frame.

Exemplarily, intra-frame coding is performed on a partial area of the n-th image frame, and the remaining areas are subjected to inter-frame coding, and then the coded n-th image frame is sent to the receiving end.

S132. Obtain a next error correction refresh frame.

Exemplarily, the camera on the drone acquires image frames frame by frame, and while or after a certain image frame is encoded and transmitted by the sending end, the sending end may obtain the next image frames in real time. Since the speed at which the camera acquires the image frame is approximately constant, the transmission time of the image frame by the sender needs to be limited to a small range so that the video transmission will not be stuck. In this embodiment, intra-encoding is performed on a part of the error correction refresh frame, the bit rate of the transmission is small, and the time is shorter; it can be realized that the error correction refresh frame is transmitted at least before the next image frame is taken to ensure the video Real-time transmission, reducing delay.

Exemplarily, an image frame newly captured by the camera is acquired, for example, the n+sth image frame is used as the next error correction refresh frame, where s is less than M, and s is a natural number that is not 0.

S133: Perform intra-frame coding on a partial area of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.

Exemplarily, intra-encoding is performed on a part of the n+sth image frame, and the remaining areas are inter-encoded, and then the encoded n+sth image frame is sent to the receiving end.

In some embodiments, different error correction refresh frames corresponding to the same error feedback information perform intra-frame coding in different regions. Therefore, the receiving end can update different parts of the reference data according to the decoding of different error correction refresh frames.

S134. Return to the acquisition of the next error correction refresh frame, and perform intra-frame coding on a partial area of the error correction refresh frame, and continue to execute the step of sending the encoded error correction refresh frame to the receiving end. Until the last error correction refresh frame is coded, it is sent to the receiving end.

Exemplarily, as shown in FIG. 7, step S134 returns to the acquisition of the next error correction refresh frame, and performs intra-frame encoding on a part of the error correction refresh frame, and transfers the encoded error correction refresh frame to The steps sent by the receiving end continue to be executed until the last error correction refresh frame is coded and sent to the receiving end, including:

S1341. Determine whether the last error correction refresh frame has been encoded and sent to the receiving end.

Exemplarily, if the last error correction refresh frame is sent to the receiving end after it has not been encoded, return to the acquisition of the next error correction refresh frame, and perform intra-frame encoding on a part of the error correction refresh frame, and The step of sending the encoded error correction refresh frame to the receiving end is continued; if the last error correction refresh frame has been encoded and sent to the receiving end, it can return to step S110 to continue execution.

Exemplarily, if s is less than M-1, return to step S132 to obtain the n+s+1th image frame, perform intra-frame coding on a partial area of the image frame, and send the coded image frame to the receiving end; if s is equal to M-1, then the n+M-1 image frame, that is, the last error correction refresh frame has been partially intra-coded and sent to the receiving end, and there is no need to return to obtain a new error correction refresh frame.

Exemplarily, the preset number of error correction refresh frames determined in step S121 are sequentially coded in partial regions, and the coded error correction refresh frames are sent to the receiving end.

In some embodiments, for any error feedback information, the union of the regions where the corresponding preset number of error correction refresh frames is intra-coded is equal to the range of any error correction refresh frame. After the preset number of error correction refresh frames are intra-encoded in some areas, it is equivalent to that all areas of an entire image frame are intra-encoded; therefore, the receiving end decodes the preset number of error corrections Refresh the frame, you can completely refresh the reference data to improve the accuracy of the receiver's decoding of subsequent image frames, and prevent the received video from blurring.

Exemplarily, step S130 performs intra-frame coding on a partial area of the error correction refresh frame, and after sending the coded error correction refresh frame to the receiving end, if error feedback information sent by the receiving end is received, It means that there is an error in the error correction refresh frame currently received and decoded by the receiving end; the sending end returns to step S120 and step S130 to continue execution, for example, determines the error correction refresh frame according to the error correction refresh frame, that is, the current image frame, and performs another round of correction The determination of the wrong refresh frame and the intra-frame coding of some areas.

In some embodiments, as shown in FIG. 8, step S130 performs intra-frame coding on a part of the error correction refresh frame, and after sending the coded error correction refresh frame to the receiving end, it includes:

S140. Return to the step of acquiring the current image frame, encoding the current image frame, and sending the encoded current image frame to the receiving end until the error feedback information sent by the receiving end is received.

Exemplarily, after the preset number of error correction refresh frames determined in step S121 are subjected to partial area intra-coding and sent to the receiving end, the reference data of the receiving end is updated, and normal decoding can be performed, such as inter-frame decoding. Then the sending end may return to step S110 to obtain, for example, an image frame newly captured by the camera, that is, a current image frame, encode the current image frame, and send the encoded current image frame to the receiving end. If the error feedback information sent by the receiving end is received later, steps S120 to S140 are executed to implement the update of the reference data by the receiving end.

In other embodiments, as shown in FIG. 9, if error feedback information sent by the receiving end is received in step S120, determining an error correction refresh frame according to the current image frame includes:

S122: If error feedback information sent by the receiving end is received, determine an error correction sequence according to the current image frame.

Specifically, the error correction sequence includes a preset number of error correction refresh frames and a predicted frame after the preset number of error correction refresh frames.

Exemplarily, the receiving end receives the error correction refresh frame in which the transmitting end performs partial intra-frame coding, and decoding the error correction refresh frame can update the reference data; for the sending end, you can refer to the error correction refresh frame to the error correction refresh frame after the error correction refresh frame. Several image frames are inter-encoded; for the receiving end, several image frames after the error correction refresh frame can be inter-decoded based on the reference data.

Exemplarily, the receiving end makes an error when receiving and decoding the nth image frame, and sends error feedback information to the sending end, and the sending end determines the nth image frame to the n+M-1th image frame as M errors. Error refresh frame, determine the n+M image frame to n+M+p-1 image frame as p prediction frames; the error correction refresh frame and the prediction frame after the error correction refresh frame constitute the error feedback information corresponding Error correction sequence.

In some embodiments, as shown in FIG. 9, step S140 returns to the acquisition of the current image frame, encodes the current image frame, and continues the steps of sending the encoded current image frame to the receiving end until it is received Before the error feedback information sent by the receiving end, it includes:

S150. If the correct feedback information sent by the receiving end when the error correction refresh frame is decoded is received, perform inter-frame encoding on the predicted frame in the error correction sequence, and send the encoded predicted frame to the receiving Send at the end.

Exemplarily, step S150 is performed after performing intra-frame coding on a partial area of the error correction refresh frame in step S130, and the coded error correction refresh frame is sent to the receiving end.

Exemplarily, if the receiving end receives each error correction refresh frame and decodes each error correction refresh frame, the reference data is updated, and correct feedback information can be sent to the sending end. After receiving the correct feedback information, the sending end performs inter-frame encoding on the predicted frame in the error correction sequence, and sends the encoded predicted frame to the receiving end.

Exemplarily, when performing inter-frame coding on a certain prediction frame, any prediction frame before the prediction frame in the error correction sequence may be referred to. For example, when inter-coding the n+Mth image frame, you can refer to the n+M-1th image frame; when inter-coding the n+M+p-1th image frame, you can refer to the nth Image frames to any one of the n+M+p-2th image frames.

Exemplarily, step S130 performs intra-frame coding on a partial area of the error correction refresh frame, and after sending the coded error correction refresh frame to the receiving end, if error feedback information sent by the receiving end is received, It means that there is an error in the error correction refresh frame currently received and decoded by the receiving end; the sending end returns to step S120 and step S130 to continue execution, for example, the error correction refresh frame is determined according to the error correction refresh frame, that is, the current image frame, and the error correction is performed again Refresh frame determination and partial area intra-frame coding.

In some embodiments, as shown in FIG. 10, performing intra-frame coding on a partial area of the first error correction refresh frame in step S131 includes:

S1311: Perform intra-frame coding on the header area of the first error correction refresh frame.

As shown in Figure 2, the sending end determines the nth image frame to the n+M-1th image frame as M error correction refresh frames according to the error feedback information sent by the receiving end, where the nth image frame is the first An error correction refresh frame. The header area of the first error correction refresh frame is used as the area for intra-frame encoding, and other areas can be inter-frame encoding.

Exemplarily, for a certain image frame, the image of the partial area acquired first is the image of the head area, for example, several rows of pixels in front of an image frame. For example, each image frame acquired by the sending end has a resolution of 1920x1080, in the horizontal direction, that is, there are 1920 pixels in each row, and in the vertical direction, that is, there are 1080 pixels in each column; then the first 200 rows, a total of The area of 200×1920 pixels is the head area of the image frame.

In this embodiment, performing intra-frame coding on a partial area of the error correction refresh frame in step S133 includes:

S1331: Move back the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded to obtain the area where the error correction refresh frame is intra-encoded, and perform intra-frame encoding on the error correction refresh frame The area is intra-coded.

Exemplarily, as shown in FIG. 2, the region where the n-th image frame is intra-coded is moved away from the head region, that is, moved backward, to obtain the region where the n+1-th image frame is intra-coded; The region where the n+1th image frame is intra-coded is moved to a direction away from the head region, that is, moved backward, and the region where the n+2th image frame is intra-coded is obtained.

In Figure 2, in the same error correction sequence, the areas where the error correction refresh frames are intra-coded do not overlap; for example, the area where the nth image frame is intra-coded is the pixel area from row 0 to row 200. The area where n+1 image frames are intra-coded is the pixel area from line 201 to line 400, and the area where n+2 image frames are intra-coded is the pixel area from line 401 to line 800; each The area of the error correction refresh frame for intra-frame coding may be equal or unequal.

In another feasible implementation manner, the intra-encoded area of each error correction refresh frame in the same error correction sequence may be partially overlapped; for example, the area where the nth image frame is intra-encoded is the 0th to 300th line. The pixel area of the row, the area where the n+1 image frame is intra-coded is the pixel area from the 201st line to the 500th line, and the area where the n+2 image frame is intra-coded is the 401th to the 700th line The pixel area of the row; the area of each error correction refresh frame for intra-frame coding can be equal or unequal.

In this embodiment, the union of regions where all preset number of error correction refresh frames in the same error correction sequence are intra-coded is equal to the range of any error correction refresh frame. For example, all the error correction refresh frames in the same error correction sequence need to perform intra-frame coding for each pixel range from the 1st line to the 1080th line. As shown in FIG. 2, the union of the regions where the intra-frame coding is performed from the nth image frame to the n+2th image frame is equal to the range of a complete image frame. Therefore, the receiving end can refresh the reference data completely by decoding the preset number of error correction refresh frames, so as to improve the accuracy of the receiving end to decode subsequent image frames and prevent the received video from being blurred. .

In some embodiments, as shown in FIG. 11, step S121 determines a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame, including:

S1211, according to the number of stripes in the current image frame, determine the same number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame.

A slice contains part or all of the data of a frame of image. In other words, a frame of video image can be encoded into one or several slices. A strip contains at least one macroblock (Macroblock) and at most contains the data of the entire frame of image. In different coding implementations, the number of stripes formed in the same frame of image may not be the same.

In this embodiment, each image frame includes at least two slices; the encoding and decoding between different slices in the same image frame are independent; the data referred to in the decoding process of slices, such as predictive coding, cannot exceed the The border of the strip can prevent the spread of errors.

Exemplarily, if the number of strips in the image frame is 3, then 3 error correction refresh frames are determined in the current image frame and/or the image frames after the current image frame; as shown in FIG. 2, the first image frame is determined The n image frames to the n+2th image frame are error correction refresh frames.

Exemplarily, the step S1311 performing intra-frame encoding on the header area of the first error correction refresh frame includes:

S1311'. Perform intra-frame coding on the first slice of the first error correction refresh frame.

Exemplarily, intra-frame coding is performed on the first slice of the first error correction refresh frame, and inter-frame coding is performed on other slices. The slice is intra-coded to obtain I slice, namely I slice; the slice is inter-coded, and P slice is obtained, namely P slice.

Exemplarily, as shown in FIG. 2, intra-frame coding is performed on the first slice of the n-th image frame.

Exemplarily, only type I macroblocks are included in the I slice, that is, intra-frame prediction macroblocks, and no P macroblocks, that is, inter-frame prediction macroblocks are not included; in the P slice, except for the corresponding P macroblocks , Can also contain I macroblocks.

After the slice is coded intra-frame or inter-frame, an I slice or P slice is obtained; the slice type information slice_type included in the I slice is the code corresponding to the I slice, and the I slice also includes the first slice contained in the slice. Information about the position of a macroblock in the entire frame, etc.; the slice type information slice_type included in the P slice is the code corresponding to the P slice, and the P slice also includes the first macroblock contained in the slice in the whole frame Location information, etc.

After receiving the encoded slice from the sending end, the receiving end judges whether to perform intra-frame decoding or inter-frame decoding according to the slice type information slice_type; if the slice type information slice_type is the code corresponding to the I slice, the intra-frame decoding is performed, If the slice type information slice_type is the code corresponding to the P slice, then inter-frame decoding is performed; after decoding, the decoded data is spliced into the image frame according to the position information of the first macroblock contained in each slice in the whole frame data.

Exemplarily, in step S1331, the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded is moved back to obtain the area where the error correction refresh frame is intra-coded, and the error correction is performed Refresh the frame for intra-frame coding in the area where intra-frame coding is performed, including:

S1331', adding one to the stripe sequence number of the previous error correction refresh frame of the error correction refresh frame subjected to intra-frame coding to obtain the strip sequence number of the error correction refresh frame subjected to intra-frame coding, and refresh the error correction The slice in the frame corresponding to the slice number is intra-coded.

Exemplarily, as shown in FIG. 2, the slice number of the n-th image frame subjected to intra-coding is increased by one, and the slice number of the n+1-th image frame subjected to intra-coding is obtained as two; Perform intra-frame coding on the second slice of +1 image frame, and perform inter-frame coding on the first and third slices of the n+1th image frame; then obtain the n+2th image frame , Add one to the strip number of the n+1th image frame for intra-encoding, and get the strip number of the n+2th image frame for intra-encoding as three; then add one to the strip number of the n+2th image frame. Three slices are subjected to intra-frame coding, and the other two slices of the n+2th image frame are subjected to inter-frame coding.

Each error correction refresh frame in the same error correction sequence performs intra-coding on a stripe respectively, and the union of the regions where all the preset number of error correction refresh frames are intra-coded is equal to the range of any error correction refresh frame. After the receiving end decodes an error correction refresh frame, it updates the reference information of a strip in the image frame; after decoding all the preset number of error correction refresh frames in the same error correction sequence, the reference information of each strip in the image frame is all Get updated; combining the division of the area for intra-frame coding in the error correction refresh frame into the coding unit, that is, the strip, which can reduce the amount of calculation and improve the accuracy.

In other embodiments, the error feedback information received by the transmitting end from the receiving end includes area index information.

Exemplarily, if an error occurs when the receiving end receives and decodes data of a certain area in the current image frame, it generates error feedback information according to the area index information of the area and sends it to the sending end.

Exemplarily, when the sending end sends the encoded current image frame to the receiving end, the error feedback information sent by the receiving end is received, and the error feedback information includes the area index corresponding to the pixel area from line 201 to line 400 of the image frame information.

As shown in FIG. 12, performing intra-frame coding on a partial area of the first error correction refresh frame in step S131 includes:

S1312. Determine the region where the first error correction refresh frame is intra-coded according to the region index information in the acquired error feedback information, and perform intra-frame coding on the region where the first error correction refresh frame is intra-coded .

Exemplarily, it is determined according to the area index information in the error feedback information that the area where the first error correction refresh frame is intra-coded is the pixel area from the 201st line to the 400th line of the first error correction refresh frame.

Exemplarily, intra-frame coding is performed on the region where the first error correction refresh frame is intra-coded, and other regions are inter-frame coding. For example, the pixel area from line 201 to line 400 of the nth image frame is intra-encoded, and other areas are inter-encoded; then the encoded nth image frame is sent to the receiving end.

The receiving end decodes the n-th image frame, especially after intra-frame decoding is performed on the area where the n-th image frame is intra-encoded, the reference data of the area can be updated; therefore, the reference data of the receiving and decoding error area can be updated more quickly The data is conducive to the rapid recovery of the error state and reduces the probability of the phenomenon of blurring.

Exemplarily, as shown in FIG. 13, according to the region index information in the error feedback information, it is determined that the region where the n-th image frame is intra-coded is the middle region, and then the middle region of the n-th image frame is intra-coded. Perform inter-frame coding on the head area and the tail area of the nth image frame.

S1332: Move back the area where the previous error correction refresh frame of the error correction refresh frame is intra-coded to obtain the area where the error correction refresh frame is intra-coded, and perform intra-coding on the error correction refresh frame The area is intra-coded.

Exemplarily, the region where the nth image frame is intra-coded is moved backward, and the region where the n+1th image frame is intra-coded is the tail; then the tail region of the n+1th image frame is framed Intra coding, inter-coding the header area and middle area of the n+1th image frame.

In some instances, the area where the first error correction refresh frame is intra-encoded is not at the head. Therefore, when the area where an error correction refresh frame is intra-coded is moved to the end, the head area of the image frame may still be No intra-frame coding is performed, and a subsequent error correction refresh frame is required to perform intra-frame coding on this part of the area.

Exemplarily, as shown in FIG. 14, the region where the previous error correction refresh frame of the error correction refresh frame is intra-coded in step S1332 is moved backward to obtain the region where the error correction refresh frame is intra-coded ,include:

S320: Determine whether the intra-frame coding area of the previous error correction refresh frame of the error correction refresh frame is at the end of the previous error correction refresh frame.

If the region where the previous error correction refresh frame is intra-coded is not at the end of the previous error correction refresh frame, execute:

S321: Move back the region where the previous error correction refresh frame is intra-coded to obtain the region where the error correction refresh frame is intra-coded.

Exemplarily, as shown in FIG. 13, the region where the n-th image frame is intra-encoded is the middle region. If it is not at the end of the n-th image frame, the region where the n-th image frame is intra-encoded is moved backward. The region where the n+1th image frame is intra-encoded is the tail.

If the region where the previous error correction refresh frame is intra-encoded is at the end of the previous error correction refresh frame, execute:

S322. Determine the header area of the error correction refresh frame as an area where the error correction refresh frame performs intra-frame coding.

Exemplarily, as shown in FIG. 13, the area where the n+1th image frame is intra-coded is the tail, and the area where the n+2th image frame is intra-coded is the head; The header area of the two image frames is intra-encoded, and the middle area and the tail area of the n+2th image frame are inter-encoded.

In step S1332, the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded is moved back to obtain the area where the error correction refresh frame is intra-encoded, and a partial area frame is performed on the error correction refresh frame After the internal coding, if the error correction refresh frame is not the last one of the preset number of error correction refresh frames corresponding to the error feedback information, return to step S132 to obtain the next error correction refresh frame; if the error correction refresh frame Is the last one of the preset number of error correction refresh frames corresponding to the error feedback information, the sending end may return to step S110 to obtain, for example, the image frame newly shot by the camera, that is, the current image frame, and encode the current image frame, Send the encoded current image frame to the receiving end.

After each error correction refresh frame in the error correction sequence corresponding to the error feedback information, the intra-frame coding area is moved back, and the intra-frame coding area reaches the tail and returns to the head for intra-frame coding; realizes all errors in the error correction sequence After intra-encoding is performed on some areas of the wrong refresh frame, all areas in the image frame are intra-encoded; after sending to the receiving end, the receiving end can completely refresh the reference data corresponding to the entire image frame.

In some embodiments, as shown in FIG. 15, step S121 determining a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame includes:

S1212: Determine the same number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame according to the number of stripes in the current image frame.

Exemplarily, if the number of strips in the image frame is 3, then 3 error correction refresh frames are determined in the current image frame and/or the image frames after the current image frame; as shown in FIG. 13, the first The n image frames to the n+2th image frame are error correction refresh frames.

Exemplarily, determining the region where the first error correction refresh frame performs intra-frame coding according to the region index information in the acquired error feedback information in step S1312 includes:

S1312'. Determine that the strip corresponding to the strip number in the error feedback information in the first error correction refresh frame is an area for intra-frame coding.

Exemplarily, the sending end encodes each strip in the image frame and sends it to the receiving end. If an error occurs when the receiving end receives and decodes a certain strip, the error feedback information is generated according to the strip sequence number of the strip, and the error is fed back The information is sent to the sender.

For example, if an error occurs when the receiving end receives and decodes the second slice of the current image frame, the error feedback information sent to the sending end includes the strip sequence number of the second slice.

The sending end determines, according to the strip sequence number in the error feedback information, that the region where the first error correction refresh frame is intra-coded is the second strip.

Exemplarily, intra-frame coding is performed on the second slice of the first error correction refresh frame, and the first slice and the third slice are subjected to inter-frame coding.

Exemplarily, shifting the region where the previous error correction refresh frame is intra-coded in step S321 to obtain the region where the error correction refresh frame is intra-coded includes:

S321', adding one to the stripe sequence number of the previous error correction refresh frame that is intra-coded to obtain the stripe number of the error correction refresh frame that is intra-coded.

For example, add one to the stripe number of the n-th image frame for intra-coding, and get the stripe number of the n+1-th image frame for intra-coding to be three; then add one to the third image frame of the n+1th image frame. Intra-coding is performed on one slice, and inter-coding is performed on the first slice and the second slice of the n+1th image frame.

Exemplarily, the determining the header area of the error correction refresh frame as the area where the error correction refresh frame performs intra-frame coding in step S322 includes:

S322'. Determine the first slice of the error correction refresh frame as the slice of the error correction refresh frame for intra-frame coding.

For example, since the strip number of the n+1th image frame for intra-encoding is the strip number corresponding to the end of the image frame, that is, three, then the n+2th image frame is obtained, and the n+1th image frame The sequence number of the slice for intra-coding is determined to be one, then the first slice of the n+2th image frame is intra-coded, and the other two slices of the n+2th image frame are inter-coded .

Each error correction refresh frame in the same error correction sequence performs intra-coding on a stripe respectively, and the union of the regions where all the preset number of error correction refresh frames are intra-coded is equal to the range of any error correction refresh frame. The first error correction refresh frame first performs intra-frame coding on the receiving and decoding error areas. After decoding, the receiving end can update the error reference information first, which can recover transmission errors more quickly.

In some embodiments, as shown in FIG. 16 and FIG. 17, encoding the current image frame in step S110 includes at least step S111 to step S112.

S111: Determine whether the current image frame is an interval refresh frame according to an interval refresh condition.

In this embodiment, when encoding the acquired current image frame in step S110, in addition to inter-encoding some image frames, an inter-encoded frame, namely P frame, is obtained; some image frames can also be framed at intervals. Intra-coding or intra-frame coding of a part of the area so that the receiving end can update the reference data in time.

Exemplarily, when the error feedback information sent by the receiving end is not received, the acquired image frame is encoded according to the refresh cycle.

Specifically, the refresh period includes adjacent refresh ranges and prediction ranges.

Specifically, the image frames within the prediction range in the same refresh period are all located behind all the image frames within the refresh range.

In a refresh cycle, the sender first performs intra-frame encoding on the image frame within the refresh range, and the receiver can update the reference data of the image frame after decoding; then the sender refers to the image frame or prediction range within the refresh range within the refresh period The previous image frame within the frame performs inter-frame coding on the newly acquired image frame; the receiving end can decode the inter-coded image according to the reference data.

Exemplarily, step S111 determines whether the current image frame is an interval refresh frame according to an interval refresh condition, including: if the number of the current image frame is within the refresh range of any refresh period, determining that the current image frame is an interval refresh Frame; if the number of the current image frame is within the prediction range of any refresh period, it is determined that the current image frame is not an interval refresh frame.

Exemplarily, when the error feedback information sent by the receiving end is not received, it is determined whether the current image frame is an interval refresh frame according to the rules of the group of pictures (GOP) division.

An image sequence, that is, an image group, is a group of consecutive pictures in an encoded video stream. Each coded video stream consists of a continuous sequence of images.

Exemplarily, the data change between each image frame in the same image sequence is small; for example, the previous one of two adjacent image frames with large data changes can be used as the last frame in the previous image sequence, and The latter of two adjacent image frames with large data changes is taken as the first frame in the subsequent image sequence.

Exemplarily, the image sequence includes adjacent refresh subsequences and predictive subsequences, and the refresh subsequence includes at least one image frame at the beginning of the image sequence.

Specifically, the predictor subsequence in the same image sequence is located after the refresh subsequence.

When encoding image frames in an image sequence, the sender first performs intra-frame encoding on the image frames in the refresh subsequence, and the receiver can update the reference data of the image frame after decoding; then the sender refers to the refresh subsequence The newly acquired image frame is inter-encoded on the image frame or the previous image frame in the predicted sub-sequence; the receiving end can decode the inter-encoded image according to the reference data.

Exemplarily, step S111 judging whether the current image frame is an interval refresh frame according to the interval refresh condition includes: if the number of the current image frame is in the refresh subsequence of any image sequence, determining that the current image frame is Interval refresh frame; if the number of the current image frame is in the predicted subsequence of any image sequence, it is determined that the current image frame is not an interval refresh frame.

S112: If the current image frame is not an interval refresh frame, perform inter-frame encoding on the current image frame.

Exemplarily, as shown in FIG. 18, every 8 image frames is a refresh period, wherein the first image frame in each refresh period is an interval refresh frame, and the second to eighth image frames are not interval refresh frames ; Then respectively perform inter-frame encoding on the acquired second to eighth image frames to obtain respective corresponding inter-frame encoded frames, namely P frames. For example, refer to the first image frame to perform inter-coding on the second to eighth image frames; or refer to any one of the first to seventh image frames to perform inter-frame coding on the eighth image frame.

Exemplarily, as shown in FIG. 19, every 8 image frames is a refresh period, wherein the first to third image frames in each refresh period are interval refresh frames, and the fourth to eighth image frames It is not an interval refresh frame; then the acquired 4th to 8th image frames are inter-encoded respectively to obtain respective corresponding inter-encoded frames, namely P frames. For example, refer to the first image frame to perform inter-frame coding on the fourth to eighth image frames; or refer to any one of the first to seventh image frames to perform inter-frame coding on the eighth image frame.

In some embodiments, as shown in FIG. 16, after step S111 determines whether the current image frame is an interval refresh frame according to the interval refresh condition, the method includes:

S113: If the current image frame is an interval refresh frame, perform intra-frame coding on the current image frame.

Exemplarily, as shown in FIG. 18, the first image frame in each refresh cycle is an interval refresh frame; then the entire frame of the image frame is intra-coded to obtain an intra-coded frame, that is, an I frame.

By intra-coding a refresh cycle or interval refresh frame in an image sequence, after receiving and decoding each interval refresh frame, the receiving end can update the reference data completely to improve the refresh cycle or each non-refresh frame in the image sequence. Refresh the accuracy of frame decoding at intervals to prevent blurring.

In some embodiments, as shown in FIG. 17, step S111 after determining whether the current image frame is an interval refresh frame according to the interval refresh condition includes:

S114: If the current image frame is an interval refresh frame, perform intra-frame coding on a partial area of the current image frame.

Exemplarily, as shown in FIG. 19, every 8 image frames is a refresh period, and the first to third image frames in each refresh period are interval refresh frames; part of each interval refresh frame is performed Intra coding.

Exemplarily, the region where each interval refresh frame in the same refresh period or image sequence undergoes intra-coding is different, and the union of the regions where all interval refresh frames in the same refresh period or image sequence undergo intra-frame encoding is equal to the image of each image frame range.

Exemplarily, in the same refresh period or image sequence, regions where intra-frame coding is performed on refresh frames at different intervals may partially overlap, or may not overlap with each other.

By performing intra-frame coding on a partial area of each interval refresh frame in a refresh period or an image sequence, after receiving and decoding each interval refresh frame, the receiving end can update the reference data completely to improve the refresh period or image sequence The accuracy of the decoding of the non-interval refresh frames in the middle, to prevent the screen blur.

Exemplarily, in step S114, if the current image frame is an interval refresh frame, performing intra-frame encoding on a partial area of the current image frame includes: if the current image frame is an interval refresh frame, performing intra-frame encoding on the current image frame Part of the area is intra-coded, and the remaining area is inter-coded.

Performing intra-frame coding on part of the interval refresh frame can not only reduce the resource occupation of network resources, avoid the intra-frame coding of the complete image frame, which will cause the burst rate and increase the delay and jitter, and it can be received at intervals. The terminal provides the reference data needed for decoding.

In some embodiments, as shown in FIG. 20, step S110 acquires a current image frame, encodes the current image frame, and before sending the encoded current image frame to the receiving end, the method includes:

S101. Perform intra-frame encoding on an image frame, and send the encoded image frame to the receiving end.

Exemplarily, as shown in FIG. 21, the image frame a is the first image frame sent by the sending end to the receiving end at a certain moment. The transmitting end performs intra-frame encoding on the image frame to obtain an I frame, and sends the encoded image frame to the receiving end; so that the receiving end can quickly obtain complete reference data after decoding.

Exemplarily, the first image frame collected by the sending end after the camera is turned on for a certain time is the initial image frame, and the sending end performs intra-frame coding on the image frame.

Exemplarily, step S101 performs intra-frame encoding on an image frame, and after sending the encoded image frame to the receiving end, the method further includes:

S102: Perform inter-frame encoding on several image frames following the image frame according to the intra-encoded image frame, and send the encoded image frame to the receiving end.

After the image frame a is intra-encoded and sent to the receiving end, the receiving end can obtain the complete reference data; the sending end can refer to the image frame a to perform inter-frame encoding on the obtained image frame b, and obtain the P frame and send it to the receiving end , The receiving end decodes the image frame b according to the reference data. The sending end may refer to the image frame a or the image frame b to perform inter-frame encoding on the acquired image frame c, and obtain the P frame to send to the receiving end, and the receiving end decodes the image frame c according to the reference data.

Exemplarily, the number of image frames to be inter-encoded in step S102 is equal to the number of image frames in one refresh cycle minus one. It is convenient for the sender to plan the interval refresh frame and the non-interval refresh frame in the refresh cycle.

Exemplarily, when part of the error correction refresh frame or interval refresh frame is intra-encoded, when the remaining areas are inter-encoded, the inter-encoded area may refer to the image frame before the error correction refresh frame or the interval refresh frame. Corresponding area.

As shown in Figure 13, when performing intra-frame coding on the first slice of the n+2th image frame and performing inter-frame coding on the other two slices of the n+2th image frame, you can refer to the n+th The second slice and the third slice in one image frame are inter-coding, but the second slice and the third slice in the n-th image frame can be referenced for inter-coding.

Through the above method, the embodiment of the present application performs intra-frame coding on part of the error correction refresh frame when an image frame is transmitted incorrectly, which can reduce the resource occupation of network resources and avoid the intra-frame coding of the complete image frame. The burst of code rate causes delay and jitter to increase, and can provide the receiving end with reference data required for decoding in time, so that transmission errors can be recovered quickly.

Please refer to FIG. 22. FIG. 22 is a schematic flowchart of a video receiving method for the receiving end according to an embodiment of the present application. The video receiving method can be applied to a terminal device or a server for receiving video from the receiving end, etc. Process; the terminal device can be a mobile phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, a wearable device, a projector or a display, etc. The server can be an independent server or a server cluster, such as providing live video services Server. Further, the terminal device may be a virtual reality head-mounted display device used to obtain video data from a drone, such as VR glasses, or a ground station used to obtain video data from a drone.

As shown in FIG. 22, the video sending method of this embodiment includes step S210 to step S230.

S210. Receive an encoded current image frame from the sending end, and decode the current image frame.

Exemplarily, the sending end acquires the current image frame, encodes the current image frame, and sends the encoded current image frame to the receiving end. For example, the current image frame is an image frame acquired in real time by a camera set on the sending end, which represents an image frame acquired and to be sent by the sending end at the current moment.

Exemplarily, the receiving end receives the encoded image frames from the sending end in real time, decodes the received image frames; and then stores and/or plays the decoded video data.

S220. If an error occurs in receiving the current image frame or decoding the current image frame, send error feedback information to the sending end, and receive the intra-frame encoding of the partial area sent by the sending end in response to the error feedback information Error correction refresh frame.

In this embodiment, if the receiving end receives an error in an image frame or decodes an image frame, it sends error feedback information to the transmitting end.

Exemplarily, the error feedback information includes feedback information of the receiving end when a packet loss event occurs, or includes feedback information of the receiving end when an integrity check error occurs.

Different from performing intraframe (Intraframe) coding on the entire image frame in the prior art, in this embodiment, the sending end performs intraframe coding on a partial area of the error correction refresh frame, and sends the coded error correction refresh frame to the The receiving end sends; the receiving end receives the error correction refresh frame that the sender sends in response to the part of the error feedback information to perform intra-frame coding.

As shown in Fig. 2, the sending end performs intra-frame coding on a partial area from the nth image frame to the n+2th image frame.

S230. Decode the error correction refresh frame, and refresh the reference data used for decoding according to the decoding result.

When the receiving end decodes the area of the error correction refresh frame for intra-frame coding, it can decode this part of the data without referring to other image frames. The decoded data can be used as the reference data required when decoding other image frames, realizing interoperability. Update of reference data for decoding.

In the video receiving method provided in this embodiment, when an image frame transmission error occurs, the sending end performs intra-frame coding error correction refresh frames for some areas; it can reduce the resource occupation of network resources and avoid receiving the entire frame for intra-frame The encoded image frame causes a burst of code rate and causes delay and jitter to increase, and the receiving end can update the reference data required for decoding in time, so that transmission errors can be recovered quickly.

Exemplarily, as shown in FIG. 23 and FIG. 24, the decoding of the current image frame in step S210 includes step S2101 and step S211.

S2101: Parse the encoding format corresponding to the current image frame.

Exemplarily, when the sending end sends the encoded image frame to the receiving end, the sent data includes the encoding format corresponding to the image frame; the receiving end can obtain the encoding format corresponding to the current image frame by analyzing the received data.

S211: If the encoding format corresponding to the current image frame is inter-frame encoding, decode the current image frame according to the reference data used for decoding.

In some embodiments, the transmitting end performs interframe encoding on the current image frame to obtain a predictive-coded picture (P frame); the receiving end can perform interframe encoding on the image frame based on the reference data. decoding.

In some embodiments, as shown in FIG. 23, the decoding of the current image frame in step S210 includes step S2121 to step S2122.

S2121, if the encoding format corresponding to the current image frame is intra-frame encoding, perform intra-frame decoding on the current image frame.

Exemplarily, as shown in FIG. 18, the first image frame in each refresh cycle is an interval refresh frame; the sending end performs intra-frame encoding on the entire image frame to obtain an intra-frame coded frame, that is, an I frame. After receiving the image frame, the receiving end performs intra-frame decoding on the image frame.

S2122, refresh the reference data according to the result of intra-frame decoding.

Exemplarily, the receiving end receives and decodes the interval refresh period or the interval refresh frame in the image sequence, such as an intra-frame encoding frame, the reference data can be completely updated once to improve the refresh period or each non-interval refresh in the image sequence The accuracy of frame decoding prevents screen flickering.

Exemplarily, as shown in FIG. 23, after the step S2122 refreshes the reference data according to the result of intra-frame decoding, the method includes:

S2123: Send correct feedback information to the sending end.

For example, as shown in FIG. 18, after receiving and decoding the interval refresh frame in the refresh cycle, the receiving end correctly updates the reference data, and then sends correct feedback information to the transmitting end. After receiving the correct feedback information, the sending end performs inter-frame coding on the second to eighth image frames in the acquired refresh cycle to obtain respective inter-coded frames, that is, P frames and send them to the receiving end.

In other embodiments, as shown in FIG. 24, the decoding of the current image frame in step S210 includes step S2131 to step S2132.

S2131. If the encoding format corresponding to the current image frame is intra-encoding of a partial area, perform intra-frame decoding on the intra-encoded area of the current image frame to obtain refresh segment data corresponding to the current image frame.

Exemplarily, as shown in FIG. 19, every 8 image frames is a refresh period, and the first to third image frames in each refresh period are interval refresh frames; the sending end refreshes the part of each interval The region is intra-coded and sent to the receiving end. After receiving the interval refresh frame coded in the partial region, the receiving end performs intra-frame decoding on the intra-coded area of the interval refresh frame.

S2132: Update part of the data in the reference data corresponding to the region encoded in the frame to the refresh segment data.

Exemplarily, when the receiving end decodes the intra-coded region of the current image frame, it can decode the part of the data without referring to other image frames, and the decoded data can be used as reference data for decoding other image frames , To realize the periodic update of the reference data used for decoding.

Exemplarily, as shown in FIG. 24, the decoding of the current image frame in step S210 further includes:

S2133: If the encoding format corresponding to the current image frame is intra-frame encoding of a partial area, perform inter-frame decoding on the remaining area of the current image frame according to the reference data.

Exemplarily, the sender performs intra-frame coding on some areas of the interval refresh frame, and performs inter-frame coding on the remaining areas; this can reduce the resource occupation of network resources and avoid the intra-frame coding of the complete image frame, which may cause a burst of bit rate. It causes delay and jitter to increase, and can provide the receiver with the reference data needed for decoding at intervals.

Exemplarily, as shown in FIG. 24, after the step S2132 updates part of the data in the reference data corresponding to the intra-coded region to the refresh segment data, the step S2132 includes:

S2134: If all data in the reference data are updated once, send correct feedback information to the sending end.

Exemplarily, the union of regions where all interval refresh frames in the same refresh period or image sequence undergo intra-frame coding is equal to the image range of each image frame; as shown in Figure 19, the receiving end receives and decodes three After refreshing the frame at intervals, the reference data can be completely updated once, and then correct feedback information is sent to the sending end.

After receiving the correct feedback information, the sending end can perform inter-frame coding on the 4th to 8th image frames in the acquired refresh period to obtain respective inter-frame coded frames, that is, P frames and send them to the receiving end.

Exemplarily, when the receiving end receives and decodes an image frame whose encoding format is inter-frame encoding, intra-frame encoding, or intra-encoded in a partial area, if there is an error in receiving or decoding, it will send the corresponding image frame to the transmitting end. Error feedback information. When the sending end receives the error feedback information corresponding to any image frame, it determines the error correction refresh frame according to the image frame, and performs intra-frame encoding on a part of the error correction refresh frame, and the encoded error correction refresh frame Send to the receiving end. Therefore, the sending end determines the error correction refresh frame corresponding to the error feedback information in response to the error feedback information of the receiving end, performs intra-frame coding on a part of the error correction refresh frame, and sends the coded error correction refresh frame to the receiver The receiving end receives and decodes the error correction refresh frame, and refreshes the reference data used for decoding according to the decoding result.

In some embodiments, the receiving end receives the encoded current image frame from the sending end, and decoding the current image frame is specifically: the receiving end receives from the sending end the image area data encoded for each area of the current image frame, Decoding each image area data of the current image frame.

Exemplarily, when the sending end encodes an image frame, it divides the image frame into multiple areas and encodes the image area data respectively, and sends each image area data to the receiving end; the receiving end receives and decodes each image area data one by one .

For example, the sending end divides the image frame into multiple slices, and the encoding and decoding of different slices in the same image frame are independent. The sending end sends the image area data obtained by encoding each strip to the receiving end; the receiving end receives and decodes each image area data one by one.

Exemplarily, the image area data includes encoding format information, and the image area data can be decoded according to a decoding manner corresponding to the encoding format information.

For example, the transmitting end performs intra-frame coding and inter-frame coding on the slices to obtain image area data whose coding format information is intra-frame coding and inter-frame coding, namely, I-strip and P-strip.

The coding format information included in the I slice, such as slice type information slice_type is the code corresponding to the I slice; the coding format information included in the P slice, such as the slice type information slice_type is the code corresponding to the P slice; exemplary, The I slice also includes the information (first_mb_in_slice) of the position of the first macroblock contained in the slice in the whole frame, etc., and the P slice also includes the information of the position of the first macroblock contained in the slice in the whole frame ( first_mb_in_slice) and so on.

Exemplarily, the encoding format is the current image frame of intra-encoded, and the encoding format information of each image area data after encoding is intra-encoded; the encoding format is the current image frame of intra-encoded partial area, corresponding to the frame The coding format information of the image area data in the inner coding area is intra-frame coding, and the coding format information of the image area data corresponding to other areas is inter-frame coding.

For example, after receiving the image area data, the receiving end judges whether the image area data is an I stripe or a P stripe according to the encoding format information in the image area data; if it is an I stripe, it performs intra-frame decoding on the image area data; if so, P strips, then perform inter-frame decoding on the image area data.

In some embodiments, in step S220, if there is an error in receiving the current image frame or if there is an error in decoding the current image frame, sending error feedback information to the sending end includes: if the current image frame is lost when receiving the encoded current image frame. Packet events, send error feedback information to the sender.

Exemplarily, the receiving end receives the encoded image frame from the sending end, performs protocol header parsing and error-tolerant processing, and decodes it if it is confirmed to be correct; if the sequence number of the encoded frame in the frame header protocol is not continuous, it means there is packet loss, that is Packet loss event.

For example, the sender has a global coded frame counter (frame_idx), which records the frame number of the coded frame by frame. The frame sequence number will be sent along with the protocol header. If the frame numbers of adjacent frames received by the receiving end are not continuous, indicating that there is frame loss during the transmission process, error feedback information is sent to the sending end.

In some embodiments, if the encoded image area data is not correctly received from the sending end, error feedback information is generated based on the image area data, and the error feedback information is sent to the sending end.

Exemplarily, the receiving end receives the encoded image area data of each area of the image frame from the sending end one by one. If the area corresponding to the image area data received in a certain time is not continuous with the area corresponding to the image area data received in the previous time, it means the middle If the data in the image area is lost and not received, an error feedback message is sent to the sender.

Exemplarily, the receiving end receives the image area data encoded by each stripe of the image frame from the sending end one by one; the sending end strictly sends the image area data of a single strip as a data service packet, and the receiving end can be based on the physical layer business The packet number index corresponds to the stripe index of the image area data; if the stripe index of the image area data received by the receiving end is not continuous with the stripe index of the image area data received the previous time, it means that there is an image corresponding to the stripe If the area data is lost and not received, an error feedback message is sent to the sender. Exemplarily, the transmission protocol header may not include the slice index corresponding to the current transmission code stream.

Exemplarily, when the receiving end receives from the sending end, the image area data encoded in each area of the current image frame at the sending end includes a data packet sequence number.

In some embodiments, if the packet sequence number of the image area data is not continuous with the packet sequence number of the previous image area data, according to the packet sequence number of the previous image area data and the first image corresponding to the current image frame The data packet sequence number of the area data calculates the area index information corresponding to the image area data; and generates error feedback information including the area index information.

The area index information may indicate the position of the area in the image frame corresponding to the error image area data. For example, if the packet sequence number of the image area data received by the receiving end at a certain time is 17, but the packet sequence number of the image area data received before is 14, it means that the image area data with packet numbers 15 and 16 has not been received; if The data packet sequence number of the first image area data corresponding to the current image frame is 12, then the packet sequence number 14 of the previous image area data can be subtracted from the data packet sequence number 12 of the first image area data corresponding to the current image frame, and then Add one to obtain the area index information corresponding to the image area data as 3; or subtract the packet sequence number of the first image area data corresponding to the current image frame from 15 from the packet sequence number 15 of the first unreceived image area data , The area index information corresponding to the image area data is 3; it means that the first unreceived image area data is the image area data corresponding to the strip after the third strip in the current image frame, that is, the fourth Image area data corresponding to each band.

Exemplarily, each image area data sent by the sending end to the receiving end includes a slice index corresponding to the coded slice. For example, each data packet sent from the sender to the receiver includes a transmission protocol header with a strip index.

For example, if the strip index of the image area data is not continuous with the strip index of the previous image area data, according to the strip index of the previous image area data and the first image area corresponding to the current image frame The strip index of the data calculates the area index information corresponding to the image area data; and generates error feedback information including the area index information.

For example, if the stripe index of the image area data received by the receiving end at a certain time is 6, but the stripe index of the previous image area data is 3, it means that the image area with the stripe index of 4 and 5 has not been received. Data; the data stripe index of the first image area data corresponding to the current image frame is 1, then the stripe index 3 of the previous image area data can be subtracted from the data stripe of the first image area data corresponding to the current image frame With index 1, then add one to get the area index information corresponding to the image area data as 3; or subtract the first image area corresponding to the current image frame from the strip index 4 of the first unreceived image area data The data stripe index of the data is 1, and the area index information corresponding to the image area data is 3; it means that the first unreceived image area data corresponds to the stripe after the third stripe in the current image frame Image area data, that is, the image area data corresponding to the fourth band.

Exemplarily, the area index information corresponding to the image area data is calculated, and error feedback information including the area index information is generated; the error feedback information received by the sending end from the receiving end includes area index information. Therefore, the sending end can determine the region where the first error correction refresh frame is intra-coded according to the region index information in the acquired error feedback information, and frame the region where the first error correction refresh frame is intra-coded. Intra coding; for example, intra coding is performed on the fourth slice in the first error correction refresh frame.

Exemplarily, the receiving end decodes the first error correction refresh frame, especially after intra-frame decoding is performed on the region where the first error correction refresh frame is intra-coded, the reference data of the region can be updated; therefore, it can be faster Update the reference data of the receiving and decoding error area, which is conducive to the rapid recovery of the error state and reduces the probability of the screen blur phenomenon.

Exemplarily, as shown in FIG. 13, the sender determines the region where the n-th image frame is intra-encoded as the middle region according to the region index information in the error feedback information, and then performs intra-frame coding on the middle region of the n-th image frame. Encoding, inter-coding the header area and tail area of the nth image frame. After receiving the image frame, the receiving end performs intra-frame decoding on the middle region of the nth image frame, and updates the data corresponding to the middle region in the reference data.

In other embodiments, the image area data of each area of the current image frame at the transmitting end received by the receiving end from the transmitting end includes area index information.

Exemplarily, the certain image area data received by the receiving end from the sending end includes area index information, such as stripe index 3.

If the encoded image area data is not correctly received from the sending end, error feedback information including the area index information is generated, and the error feedback information is sent to the sending end.

Exemplarily, if the receiving end makes an error when receiving the image area data corresponding to the area index information of strip index 3, for example, although the area index information is obtained through protocol header analysis, the image data is incomplete or the bit error rate is too high, then Generate error feedback information including region index information of stripe index 3, and send the error feedback information to the sending end. The sending end may perform intra-frame coding on the third slice in the first error correction refresh frame according to the area index information in the obtained error feedback information.

In some embodiments, in step S220, if there is an error in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end includes: if the received current image frame fails the integrity Checking, sending error feedback information to the sending end, where the error feedback information includes area index information corresponding to the header area of the current image frame.

Exemplarily, after receiving the current image frame, if the length of the encoded frame in the parsed end-of-frame protocol and the checksum (checksum) check are not passed, it is determined that the integrity check is not passed; to the sending end Send error feedback information.

Exemplarily, if the received current image frame fails the integrity check, it indicates that the current image frame received an error, and the error feedback information that can be sent to the sender includes the area index information corresponding to the header area of the current image frame, such as a stripe Index 1: The sender performs intra-frame coding on the first slice in the first error correction refresh frame after the error feedback information of the slice index is 1.

Exemplarily, the stripe index corresponding to the header area of the current image frame is 0, and the sending end performs intra-frame on the first stripe in the first error correction refresh frame according to the error feedback information with the stripe index of 0 coding.

In some embodiments, in step S220, if there is an error in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end includes: if decoding the current image frame fails, sending error feedback to the The sending end sends error feedback information.

Exemplarily, when decoding the current image frame or the image area data of the current image frame, a code word that does not exist in the code table is encountered, or the decoded code is determined according to the value range of the parameter specified by the syntax (syntax) If the value is unreasonable, it is determined that the decoding of the current image frame fails, and error feedback information is sent to the sending end.

For example, if the decoding of a certain image area data of the current image frame fails, an error feedback message containing the index information of the image area data area is sent to the sending end.

In some embodiments, the receiving of the error correction refresh frame for intra-frame coding of the partial area sent by the sending end in response to the error feedback information in step S220 includes: receiving a preset number of codes from the sending end After the error correction refresh frame, the error correction refresh frame is obtained by the transmitting end performing intra-frame encoding on a partial area of the error correction refresh frame determined according to the error feedback information.

In this embodiment, if the sending end receives the error feedback information sent by the receiving end, a preset one is determined in the image frame corresponding to the error feedback information, that is, the current image frame and/or the image frames subsequent to the current image frame. Number of error correction refresh frames.

For example, after receiving the error feedback information sent by the receiving end, the sending end uses a preset number of image frames starting from the image frame corresponding to the error feedback information as the error correction refresh frame.

For example, as shown in FIG. 2, the sending end determines the M image frames starting from the nth image frame, that is, the nth image frame to the n+M-1th image frame as M error correction refresh frames. To respond more quickly to the error feedback information from the receiving end, send an error correction refresh frame to the receiving end.

The steps for the sending end to perform intra-frame encoding on the partial area of the error correction refresh frame determined according to the error feedback information have been described in the video sending method applied to the sending end, and will not be repeated here.

Exemplarily, the region where the receiving end receives the first error correction refresh frame from the sending end for intra-frame coding may be the header area, such as the first stripe.

Exemplarily, the region where the receiving end receives the first error correction refresh frame from the sending end for intra-frame coding may also be an area determined according to the area index information in the error feedback information, such as the middle area or the fourth stripe.

In some embodiments, as shown in FIG. 25, in step S230, the receiving end decodes the error correction refresh frame and refreshes the reference data for decoding according to the decoding result, including steps S231 to S232.

S231. Perform intra-frame decoding on the region coded in the error correction refresh frame to obtain refresh segment data corresponding to the error correction refresh frame.

After receiving the error correction refresh frame from the sending end, the receiving end performs intra-frame decoding on the intra-encoded area to obtain the refresh segment data corresponding to the area.

Exemplarily, when the receiving end receives the error correction refresh frame in which the partial area sent by the transmitting end in response to the error feedback information performs intra-frame coding, the coded image area data is received from the transmitting end, and the image area The data includes data encoded by the sending end on a partial area of the error correction refresh frame, and encoding format information.

In this embodiment, the encoding format information of the image area data obtained by intra-encoding some areas in the error correction refresh frame is intra-frame encoding, and the encoding format information of several image area data obtained after inter-encoding the remaining areas is Inter-frame coding.

Exemplarily, if the encoding format information in the image area data received at a certain time by the receiving end is intra-frame encoding, then intra-frame decoding is performed on the partial area-encoded data in the image area data to obtain the image area data The corresponding refresh fragment data.

In some embodiments, the image area data further includes area index information of the partial area. Exemplarily, the receiving end performs intra-frame decoding on the encoded data of the partial region to obtain refresh segment data corresponding to the region index information.

Exemplarily, the sending end performs intra-frame encoding on the third stripe of a certain error correction refresh frame, and performs inter-frame encoding on the first and second stripe to obtain three image area data, of which the third stripe The encoding format information of the corresponding image area data is intra-frame encoding; after receiving the image area data corresponding to the third stripe, the receiving end performs intra-frame decoding on the image area data to obtain the refresh corresponding to the third stripe Fragment data.

Exemplarily, intra-frame decoding is performed on the area coded in the error correction refresh frame, and the remaining areas are inter-decoded according to the reference data used for decoding. For example, after receiving the image area data corresponding to the first and second slices, the receiving end performs inter-frame decoding according to the data corresponding to the first and second slices in the reference data.

S232. Refresh the reference data according to the refresh segment data corresponding to the error correction refresh frame.

Since only part of the area of each refresh frame is intra-encoded, the receiving end decodes the part of the intra-encoded area to obtain part of the reference data corresponding to the corresponding area, that is, refresh segment data, and then update the reference data according to the refresh segment data. Part of the data corresponding to this area.

Exemplarily, part of the data corresponding to the partial area in the reference data is updated to the refresh segment data.

In some embodiments, the image area data further includes area index information of the partial area. Exemplarily, the receiving end updates part of the data corresponding to the area index information in the reference data to the refresh segment data.

Exemplarily, after receiving the image area data corresponding to the third stripe, the receiving end performs intra-frame decoding on the image area data to obtain the refresh segment data corresponding to the third stripe; the reference data corresponds to the third stripe. Part of the data of each stripe is updated to the refresh segment data.

In some embodiments, after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, correct feedback information is sent to the sending end.

Exemplarily, the union of the regions where the preset number of error correction refresh frames corresponding to the same error feedback information is intra-encoded is equal to the image range of each image frame; as shown in Figure 2, the receiving end receives and decodes three errors After the wrong refresh frame, the reference data can be completely updated, and then correct feedback information is sent to the sending end. After receiving the correct feedback information, the sending end can continue to encode subsequent image frames.

Exemplarily, the sending end determines an error correction sequence according to the error feedback information sent by the receiving end, and the error correction sequence includes a preset number of error correction refresh frames and several non-error correction refresh frames, such as predicted frames . After refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, the receiving end sends correct feedback information to the sending end; after receiving the correct feedback information, the sending end corrects the error correction sequence The predicted frame of is subjected to inter-frame encoding, and the encoded predicted frame is sent to the receiving end.

Exemplarily, if there is an error in the error correction refresh frame currently received and decoded by the receiving end, the error feedback information is sent to the transmitting end; the transmitting end determines the preset number of error correction refresh frames according to the error correction refresh frame, and repeats One-time error correction refresh frame determination and partial area intra-frame coding.

In some embodiments, the receiving end refreshes the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, then returns to step S210 to continue execution; that is, continues to receive the encoded current image frame from the sender , The step of decoding the current image frame continues to be executed until receiving the current image frame error or decoding the current image frame error.

Exemplarily, after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, the receiving end sends correct feedback information to the sending end; after the sending end receives the correct feedback information, Obtain, for example, the image frame newly shot by the camera, that is, the current image frame, encode the current image frame, and send the encoded current image frame to the receiving end; the receiving end receives the encoded current image frame from the sending end, and decodes the current image frame. Describe the current image frame.

Please refer to FIG. 26. FIG. 26 is a schematic flowchart of a video transmission method provided by an embodiment of the present application, which is used for the aforementioned sending end to transmit video to the aforementioned receiving end.

As shown in FIG. 26, the video transmission method of this embodiment includes steps S510 to S540.

S510: The sending end encodes the acquired current image frame, and sends the encoded current image frame to the receiving end.

In this embodiment, the sending end sends image frames to the receiving end frame by frame, as shown in Figure 2; exemplary, the camera on the sending end acquires the image frames in real time, encodes the acquired image frames frame by frame, and sends them to the receiving end. .

Exemplarily, the sending end determines whether the current image frame is an interval refresh frame according to an interval refresh condition; if the current image frame is not an interval refresh frame, the sending end performs inter-frame encoding on the current image frame.

Exemplarily, if the number of the current image frame is within the refresh range of any refresh period, the sending end determines that the current image frame is an interval refresh frame, and the refresh period includes adjacent refresh ranges and prediction ranges; If the number of the current image frame is within the prediction range of any refresh period, the sending end determines that the current image frame is not an interval refresh frame.

Exemplarily, if the number of the current image frame is in the refresh subsequence of any image sequence, the sending end determines that the current image frame is an interval refresh frame, and the image sequence includes adjacent refresh subsequences and A predictive subsequence, where the refresh subsequence includes at least one image frame at the beginning of the image sequence; if the number of the current image frame is in the predictor subsequence of any image sequence, the sending end determines the current image The frame is not an interval refresh frame.

In some embodiments, if the current image frame is an interval refresh frame, the sending end performs intra-frame encoding on the current image frame.

Exemplarily, as shown in FIG. 18, every 8 image frames is a refresh period, wherein the first image frame in each refresh period is an interval refresh frame, and the second to eighth image frames are not interval refresh frames ; Then respectively perform inter-frame encoding on the acquired second to eighth image frames to obtain respective corresponding inter-frame encoded frames, namely P frames. By intra-coding a refresh cycle or interval refresh frame in an image sequence, after receiving and decoding each interval refresh frame, the receiving end can update the reference data completely to improve the refresh cycle or each non-refresh frame in the image sequence. Refresh the accuracy of frame decoding at intervals to prevent blurring.

In some other implementation manners, if the current image frame is an interval refresh frame, the sending end performs intra-frame coding on a partial area of the current image frame.

Exemplarily, if the current image frame is an interval refresh frame, the sending end performs intra-frame coding on a partial area of the current image frame, and performs inter-frame coding on the remaining areas.

Exemplarily, as shown in FIG. 19, every 8 image frames is a refresh period, wherein the first to third image frames in each refresh period are interval refresh frames, and the fourth to eighth image frames It is not an interval refresh frame; then the acquired 4th to 8th image frames are inter-encoded respectively to obtain respective corresponding inter-encoded frames, namely P frames.

In some embodiments, in step S510, the sending end encodes the acquired current image frame, and before sending the encoded current image frame to the receiving end, the sending end performs intra-frame encoding on an image frame, and the encoded The image frame is sent to the receiving end; the sending end performs inter-frame encoding on several image frames following the image frame according to the intra-encoded image frame, and sends the encoded image frame to the receiving end.

Exemplarily, as shown in FIG. 21, the image frame a is the first image frame sent by the sending end to the receiving end at a certain moment. The transmitting end performs intra-frame encoding on the image frame to obtain an I frame, and sends the encoded image frame to the receiving end; so that the receiving end can quickly obtain complete reference data after decoding. After the image frame a is intra-encoded and sent to the receiving end, the receiving end can obtain the complete reference data; the sending end can refer to the image frame a to perform inter-frame encoding on the obtained image frame b, and obtain the P frame and send it to the receiving end , The receiving end decodes the image frame b according to the reference data. The sending end may refer to the image frame a or the image frame b to perform inter-frame encoding on the acquired image frame c, and obtain the P frame to send to the receiving end, and the receiving end decodes the image frame c according to the reference data.

S520: The receiving end receives and decodes the current image frame, and if there is an error in receiving the current image frame or decoding the current image frame, it sends error feedback information to the sending end.

The receiving end receives and decodes the current image frame sent by the sending end. Exemplarily, if the encoding format corresponding to the current image frame is inter-frame encoding, the receiving end decodes the current image frame according to the reference data used for decoding.

In some implementations, if the encoding format corresponding to the current image frame is intra-frame encoding, the receiving end performs intra-frame decoding on the current image frame; the receiving end refreshes the reference according to the result of the intra-frame decoding data.

Exemplarily, the receiving end receives and decodes the refresh period or interval refresh frame in the image sequence, such as an intra-frame encoding frame, can update the reference data completely to improve the refresh period or each non-interval refresh frame in the image sequence The accuracy of decoding prevents screen flickering.

Exemplarily, after refreshing the reference data according to the result of intra-frame decoding, the receiving end sends correct feedback information to the sending end.

In other implementation manners, if the encoding format corresponding to the current image frame is intra-encoding of a partial area, the receiving end performs intra-frame decoding on the intra-encoded area of the current image frame to obtain the current image Refresh segment data corresponding to the frame; the receiving end updates part of the data in the reference data corresponding to the region encoded in the frame to the refresh segment data.

Exemplarily, if the encoding format corresponding to the current image frame is intra-frame encoding of a partial area, the receiving end performs inter-frame decoding on the remaining areas of the current image frame according to the reference data.

Exemplarily, if all data in the reference data are updated once, the receiving end sends correct feedback information to the sending end.

As shown in FIG. 19, after receiving and decoding three interval refresh frames in one refresh cycle, the receiving end can update the reference data completely, and then send correct feedback information to the sending end.

The sender performs intra-frame coding on some areas of the interval refresh frame, and inter-frame coding for the remaining areas; it can reduce the resource occupation of network resources and avoid the complete image frame intra-frame coding that causes a burst of code rate and causes delay and jitter Increase, and can provide the receiving end with reference data needed for decoding at intervals.

In some embodiments, if the receiving end determines that a packet loss event or an integrity check error occurs when receiving an image frame, it sends error feedback information to the transmitting end; the error feedback information includes the receiving end's response to the packet loss The feedback information of the event or the feedback information of the receiving end in response to the integrity check error.

Exemplarily, the receiving end sends success feedback information and error feedback information to the sending end through an uplink channel between the receiving end and the sending end.

In some embodiments, if a packet loss event occurs when the receiving end receives the encoded current image frame, error feedback information is sent to the sending end.

Exemplarily, the receiving end receives from the sending end the image area data that has been encoded for each area of the current image frame, and decodes each image area data of the current image frame.

Specifically, the image area data includes encoding format information. The receiving end decodes the image region data according to a decoding manner corresponding to the encoding format information.

Exemplarily, if the receiving end does not correctly receive the encoded image area data from the transmitting end, error feedback information is generated according to the image area data, and the error feedback information is sent to the transmitting end.

Exemplarily, the image area data includes a data packet sequence number.

Specifically, if the packet sequence number of the image area data is not continuous with the packet sequence number of the previous image area data, the receiving end uses the packet sequence number of the previous image area data and the first one corresponding to the current image frame. The data packet sequence number of the image area data calculates the area index information corresponding to the image area data; the receiving end generates error feedback information including the area index information.

Exemplarily, the image area data includes area index information.

Specifically, if the receiving end does not correctly receive the encoded image area data from the transmitting end, error feedback information including the area index information is generated.

In other implementations, if the current image frame received by the receiving end fails the integrity check, the receiving end sends error feedback information to the sending end, and the error feedback information includes the current image frame Area index information corresponding to the header area.

In another embodiment, if the receiving end fails to decode the current image frame, the receiving end sends error feedback information to the sending end.

For example, if a certain image area data of the current image frame fails to be decoded, error feedback information including the index information of the image area data area is sent to the sending end.

S530. The sending end determines an error correction refresh frame according to the error feedback information, performs intra-frame coding on a part of the error correction refresh frame, and sends the coded error correction refresh frame to the receiving end.

Exemplarily, the sending end performs intra-frame coding on a partial area of the error correction refresh frame, and after sending the coded error correction refresh frame to the receiving end, it returns to the current image frame obtained in step S510 Encoding is performed, and the step of sending the encoded current image frame to the receiving end is continued until the error feedback information sent by the receiving end is received.

In some embodiments, the sending end determines a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame, and performs part of each of the error correction refresh frames The region is intra-coded. For example, the sending end uses a preset number of image frames starting from the current image frame as error correction refresh frames.

As shown in FIG. 2, the M image frames starting from the nth image frame, that is, the nth image frame to the n+M-1th image frame, are determined as M error correction refresh frames. To respond more quickly to the error feedback information from the receiving end, send an error correction refresh frame to the receiving end.

Exemplarily, the sending end performs intra-frame coding on a part of the error correction refresh frame, and performs inter-frame coding on the remaining areas.

Exemplarily, the union of the regions where the preset number of error correction refresh frames is intra-coded is equal to the range of any error correction refresh frame.

In some implementation manners, the sending end uses the current image frame as the first error correction refresh frame, performs intra-frame encoding on a partial area of the first error correction refresh frame, and encodes the first The error correction refresh frame is sent to the receiving end; then the sending end obtains the next error correction refresh frame; then the sending end performs intra-frame encoding on a part of the error correction refresh frame, and the encoded The error correction refresh frame is sent to the receiving end; then it returns to the sending end to obtain the next error correction refresh frame, and the sending end performs intra-frame coding on a part of the error correction refresh frame, and the encoded all The step of sending the error correction refresh frame to the receiving end is continued until the last error correction refresh frame is encoded and sent to the receiving end.

In some embodiments, the sending end performs intra-frame coding on the header area of the first error correction refresh frame; the sending end performs intra-frame coding on the previous error correction refresh frame of the error correction refresh frame The coded area is moved backward to obtain the area where the error correction refresh frame is intra-coded, and the area where the error correction refresh frame is intra-coded is intra-coded.

Exemplarily, the sending end determines the same number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame according to the number of stripes in the current image frame; The sending end performs intra-frame coding on the first slice of the first error correction refresh frame; the sending end adds the sequence number of the intra-coded slice of the previous error correction refresh frame of the error correction refresh frame 1. Obtain the stripe sequence number for intra-coding of the error correction refresh frame, and perform intra-frame coding on the stripe corresponding to the stripe sequence number in the error correction refresh frame.

After the preset number of error correction refresh frames are intra-encoded in some areas, it is equivalent to that all areas of an entire image frame are intra-encoded; therefore, the receiving end decodes the preset number of error corrections Refresh the frame, you can completely refresh the reference data to improve the accuracy of the receiver's decoding of subsequent image frames, and prevent the received video from blurring.

In some other embodiments, the error feedback information includes area index information.

The sending end determines the intra-encoded area of the first error correction refresh frame according to the area index information in the obtained error feedback information, and performs intra-frame encoding on the area of the first error correction refresh frame Intra coding; the sending end shifts the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded to obtain the area where the error correction refresh frame is intra-coded, and refresh the error correction The area where the frame is intra-encoded is intra-encoded.

Specifically, if the region where the previous error correction refresh frame is intra-coded is not at the end of the previous error correction refresh frame, the transmitting end will be after the region where the previous error correction refresh frame is intra-coded To obtain the intra-frame coding area of the error correction refresh frame; if the intra-frame coding area of the previous error correction refresh frame is at the end of the previous error correction refresh frame, the sending end will The header area of the error correction refresh frame is determined as the area where the error correction refresh frame performs intra-frame coding.

Exemplarily, the sending end determines the same number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame according to the number of stripes in the current image frame; The sending end determines that the strip corresponding to the strip number in the error feedback information in the first error correction refresh frame is the area for intra-coding; if the previous error correction refresh frame is the area for intra-coding Not at the end of the previous error correction refresh frame, the sending end adds one to the stripe sequence number of the previous error correction refresh frame that is intra-coded to obtain the stripe of the error correction refresh frame that is intra-coded Serial number; if the area where the previous error correction refresh frame is intra-encoded is at the end of the previous error correction refresh frame, the sending end determines the first stripe of the error correction refresh frame as the Error correction refreshes the frame for intra-encoded stripes.

For example, the pixel area from line 201 to line 400 of the n-th image frame is intra-encoded, and other areas are inter-encoded; then the encoded n-th image frame is sent to the receiving end.

S540. The receiving end decodes the error correction refresh frame, and refreshes reference data used for decoding according to the decoding result.

In some embodiments, the receiving end receives a preset number of coded error correction refresh frames from the sending end.

Exemplarily, the receiving end performs intra-frame decoding on the region coded in the error correction refresh frame, and performs inter-frame decoding on the remaining regions according to the reference data used for decoding.

Exemplarily, the receiving end performs intra-frame decoding on the area coded in the error correction refresh frame to obtain refresh segment data corresponding to the error correction refresh frame; the receiving end corresponds to the error correction refresh frame The refresh segment data refreshes the reference data.

Exemplarily, when the receiving end decodes the area of the error correction refresh frame for intra-frame encoding, it can decode this part of the data without referring to other image frames, and the decoded data can be used as the reference data required for decoding other image frames , To achieve the update of the reference data used for decoding.

In some embodiments, when the receiving end receives a preset number of coded error correction refresh frames from the sending end, the receiving end receives the coded image area data from the sending end, and the image The area data includes data after the sending end encodes a partial area of the error correction refresh frame, and encoding format information.

Exemplarily, if the encoding format information in the image area data is intra-frame encoding, the receiving end performs intra-frame decoding on the partial area-encoded data to obtain refresh segment data corresponding to the image area data; The receiving end updates partial data corresponding to the partial area in the reference data to the refresh fragment data.

Exemplarily, the image area data further includes area index information of the partial area.

Specifically, the receiving end performs intra-frame decoding on the partial area coded data to obtain refresh segment data corresponding to the area index information; then the receiving end converts the reference data corresponding to the area index Part of the data of the information is updated to the refresh segment data.

In some embodiments, after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, the receiving end sends correct feedback information to the sending end.

In some embodiments, if the sending end receives the error feedback information sent by the receiving end, the sending end determines an error correction sequence according to the current image frame, and the error correction sequence includes a preset number of errors. The error refresh frame and the predicted frame after the preset number of error correction refresh frames.

Exemplarily, after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, the receiving end sends correct feedback information to the sending end.

The sending end performs intra-frame encoding on a partial area of the error correction refresh frame, and after sending the encoded error correction refresh frame to the receiving end, if the receiving end decodes the error correction refresh The correct feedback information sent at the time of the frame, the prediction frame in the error correction sequence is inter-frame coded, and the coded prediction frame is sent to the receiving end.

In some implementation manners, the sending end performs inter-frame encoding on the predicted frame in the error correction sequence, and after sending the encoded predicted frame to the receiving end, returns the current image frame obtained by the sending end to Perform encoding, send the encoded current image frame to the receiving end, and the steps of receiving and decoding the current image frame by the receiving end continue until the receiving end receives the current image frame in error or decodes the current image If a frame error occurs, error feedback information is sent to the sending end.

Exemplarily, each refresh period in the video transmission method includes N image frames, where the interval refresh frame includes consecutive M frames, and each image frame is divided into M strips for encoding, where N is greater than M. The receiving end sends feedback information to the sending end through the uplink channel. For example, the feedback information includes error status, the error status is a first value, such as 1, it means that the feedback information is wrong feedback information, and the error status is a second value, such as 0, it means that the feedback information is correct feedback information; wrong feedback The information may also include the slice_idx of the slice_idx of the slice with errors received and decoded.

Figure 27 shows a schematic diagram of image frame transmission.

In this example, the processing logic of the sender is as follows:

1. Perform intra-frame coding on the first frame, and send the resulting I frame to the receiving end;

2. Perform inter-frame encoding on the second frame to the Nth frame, and send the obtained P frame to the receiving end;

3. Perform periodic refresh, perform intra-frame coding of a part of the area from the first frame to the M-th frame in the refresh cycle, and send the obtained interval refresh frame to the receiving end. These M frames are combined with the intra-coded area The set is equal to the range of one frame, that is, the refresh area is superimposed into a whole frame;

4. Perform inter-frame encoding on the M+1th frame to the Nth frame in the refresh cycle, and send the obtained P frame to the receiving end;

5. Return to 3 and 4 to continue execution;

6. If the sending end receives the first value in the feedback information sent by the receiving end at any time, it immediately determines M error correction refresh frames according to the frame corresponding to the feedback information, and performs partial area detection on each error correction refresh frame. Intra-frame coding, the obtained error correction refresh frame is sent to the receiving end. The union of the regions where these M frames are intra-coded is equal to the range of one frame, that is, the refresh regions are superimposed into a whole frame; among them, according to the feedback The slice index slice_idx of the information determines the region where the first error correction refresh frame performs intra-frame coding;

7. After encoding and sending M error correction refresh frames, perform inter-frame encoding on the subsequent N-M frames, and send the obtained P frames to the receiving end;

8. Return to 3 and 4 to continue execution; among them, if the error status in the feedback information sent by the receiving end is still the first value, the slice_idx of the feedback information is used to determine the first frame in the refresh cycle for intra-frame encoding Area; if the error status in the feedback information sent by the receiving end is the second value, it is determined that the area where the first frame in the refresh cycle is intra-encoded is the first stripe;

In this example, the processing logic of the receiving end is as follows:

1. The error status in the initial feedback information is the second value, and the slice_idx of the slice index is 0;

2. Receive the encoded frame from the sender, perform protocol header analysis and fault tolerance processing, and then decode it;

3. When an I frame, or a complete set of M interval refresh frames, or a complete set of M error correction refresh frames is received, and after correctly received and decoded, the error status is set to the second value, and the bar Set slice_idx with index to 0;

4. Return to 2, 3 to continue execution;

Among them, when the protocol header is analyzed, if the physical layer packet sequence number or the encoded frame sequence number in the frame header protocol is not continuous, it means that there is packet loss, then the difference between the first packet sequence number and the first packet sequence number of the frame is recorded as slice_idx, at the same time the error status is set to the first value, and the feedback information is sent to the sender. If the coded frame length and checksum in the end-of-frame protocol fail, the error status is also set to the first value, slice_idx is recorded as 0, and the feedback information is sent to the sender.

In the above manner, in the video transmission method provided by the embodiments of the present application, when a transmission error occurs in a certain image frame, the transmitting end performs intra-frame coding on part of the error correction refresh frame, and the receiving end can update the reference data required for decoding in time. This enables faster recovery of transmission errors; and can reduce the resource occupation of network resources, avoiding the burst of code rate caused by the intra-frame encoding of the complete image frame and the increase of delay and jitter.

Exemplarily, the interval refresh frame and the error correction refresh frame are divided according to the stripe of the image frame, the number of each group of refresh frames is the same as the number of strips of the image frame, and each strip corresponds to a frame within a refresh frame Coding area.

Exemplarily, the receiving end sends feedback information to the sending order through the uplink channel, and the sending end can adjust the refresh period and the refresh mode according to the feedback information; for example, after receiving the error feedback information, break the original refresh period and encode M error corrections in time Refresh the frame so that the receiver can update the reference data.

Exemplarily, the transmission protocol guarantees that the transmitted code stream data service contains strip index information, and the receiving end can correspond to the two, so as to determine the strip index of the error location. There are, but not limited to, the following two implementation methods: one is to add a transmission protocol header with a stripe index to each sent data packet, and the other is that the transmission protocol header does not contain the stripe index corresponding to the current transmission code stream. The end sends a data service packet strictly according to a single strip, and the receiving end can obtain the strip index corresponding to the service packet number index of the physical layer.

Exemplarily, when the sender receives the error feedback information, it readjusts the refresh cycle and refresh format, which is not limited to the following implementation: if an error occurs, the refresh area corresponding to the strip index of the error feedback information starts to perform intra-frame encoding and Send, complete a full frame area refresh after M frames; after refresh complete, start a new refresh cycle after several P frames.

Exemplarily, one I frame, or a complete set of M interval refresh frames, or a complete set of M error correction refresh frames will be received, and the correct reception and decoding will be used as a judgment basis for error recovery.

Exemplarily, the video sending method provided in the embodiments of the present application can still perform decoding and display when the upstream error packet is relatively high or the upstream feedback is not timely, and the sending end will still maintain periodic refresh; when an error occurs , The sender will break the original refresh cycle, send out the error correction refresh frame as soon as possible, and the refresh area is the area corresponding to the stripe index of the error, ensuring the fastest and maximum error recovery; it can take into account the delay of image transmission and the delay Error recovery speed is the two most important image transmission indicators. The video sending method is particularly suitable for application scenarios that have extremely high requirements for image transmission delay and error recovery speed, such as traversing machines.

Please refer to FIG. 28. FIG. 28 is a schematic block diagram of a video sending device 600 according to an embodiment of the present application. The video sending device 600 includes a processor 601 and a memory 602. The processor 601 and the memory 602 are connected by a bus 603, which is, for example, an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 601 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 602 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 601 is configured to run a computer program stored in the memory 602, and implement the aforementioned video sending method when the computer program is executed.

Exemplarily, the processor 601 is configured to run a computer program stored in the memory 602, and implement the following steps when executing the computer program:

Please refer to FIG. 29, which is a schematic block diagram of an aircraft 10 according to an embodiment of the present application.

As shown in FIG. 29, the aircraft 10 includes:

The image acquisition component 11 is used to acquire images;

The above-mentioned video sending device 600 is used to obtain the image collected by the image collecting component, and send it to the receiving end after encoding;

The flight component 12 is used for flight.

Please refer to FIG. 30, which is a schematic block diagram of a video receiving apparatus 700 according to an embodiment of the present application. The video receiving device 700 includes a processor 701 and a memory 702, and the processor 701 and the memory 502 are connected by a bus 703, which is, for example, an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 701 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 702 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 701 is configured to run a computer program stored in the memory 702, and implement the aforementioned video receiving method when the computer program is executed.

Exemplarily, the processor 701 is configured to run a computer program stored in the memory 702, and implement the following steps when executing the computer program:

Please refer to FIG. 31, which is a schematic block diagram of a video playback device 20 according to an embodiment of the present application.

As shown in Figure 31, video playback equipment includes:

The above-mentioned video receiving device 700 is used for receiving and decoding an image from a sending end;

The display component 21 is used to display the data decoded by the video receiving device.

In the video sending device, aircraft, video receiving device, and video playback device provided by the above embodiments of the application, the sending end performs intra-frame coding on part of the error correction refresh frame when an image frame transmission error occurs, and the receiving end can update in time The reference data required for decoding enables faster recovery of transmission errors; and can reduce the resource occupation of network resources, avoiding the burst of code rate caused by the intra-frame encoding of the complete image frame and the increase of delay and jitter.

The embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation Example provides the steps of the video sending method.

The computer-readable storage medium may be the internal storage unit of the video sending device described in any of the foregoing embodiments, such as the hard disk or memory of the video sending device. The computer-readable storage medium may also be an external storage device of the video sending device, for example, a plug-in hard disk equipped on the video sending device, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

The embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation Example provides the steps of the video receiving method.

The computer-readable storage medium may be the internal storage unit of the video receiving device described in any of the foregoing embodiments, such as the hard disk or memory of the video receiving device. The computer-readable storage medium may also be an external storage device of the video receiving device, such as a plug-in hard disk equipped on the video receiving device, a Smart Media Card (SMC), and Secure Digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A video sending method applied to the sending end, characterized in that it includes:

Acquiring a current image frame, encoding the current image frame, and sending the encoded current image frame to the receiving end;

If error feedback information sent by the receiving end is received, determine an error correction refresh frame according to the current image frame;

Perform intra-frame coding on a part of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.
The video sending method according to claim 1, wherein the encoding the current image frame comprises:

Judging whether the current image frame is an interval refreshing frame according to the interval refresh condition;

If the current image frame is not an interval refresh frame, perform inter-frame coding on the current image frame.
The video sending method according to claim 2, wherein after the judging whether the current image frame is an interval refresh frame according to an interval refresh condition, the method comprises:

If the current image frame is an interval refresh frame, perform intra-frame coding on the current image frame.
The video sending method according to claim 2, wherein after the judging whether the current image frame is an interval refresh frame according to an interval refresh condition, the method comprises:

If the current image frame is an interval refresh frame, perform intra-frame coding on a partial area of the current image frame.
The video sending method according to claim 4, wherein, if the current image frame is an interval refresh frame, performing intra-frame coding on a partial area of the current image frame comprises:

If the current image frame is an interval refresh frame, intra-frame coding is performed on a part of the current image frame, and inter-frame coding is performed on the remaining regions.
The video sending method according to any one of claims 2-5, wherein the judging whether the current image frame is an interval refresh frame according to an interval refresh condition comprises:

If the number of the current image frame is within the refresh range of any refresh period, determining that the current image frame is an interval refresh frame, and the refresh period includes adjacent refresh ranges and prediction ranges;

If the number of the current image frame is within the prediction range of any refresh period, it is determined that the current image frame is not an interval refresh frame.
The video sending method according to any one of claims 2-5, wherein the judging whether the current image frame is an interval refresh frame according to an interval refresh condition comprises:

If the number of the current image frame is in the refresh subsequence of any image sequence, it is determined that the current image frame is an interval refresh frame, and the image sequence includes adjacent refresh subsequences and prediction subsequences, and the refresh subsequence The sequence includes at least one image frame at the beginning of the image sequence;

If the number of the current image frame is in the predicted subsequence of any image sequence, it is determined that the current image frame is not an interval refresh frame.
The video sending method according to any one of claims 1 to 5, wherein a part of the error correction refresh frame is intra-encoded, and the encoded error correction refresh frame is sent to the receiving end After that, include:

Return to the obtaining of the current image frame, encode the current image frame, and continue the steps of sending the encoded current image frame to the receiving end until the error feedback information sent by the receiving end is received.
The video sending method according to any one of claims 1 to 5, wherein before the acquiring the current image frame, encoding the current image frame, and sending the encoded current image frame to the receiving end, include:

Intra-encoding an image frame, and sending the encoded image frame to the receiving end;

Perform inter-frame encoding on several image frames following the image frame according to the intra-encoded image frame, and send the encoded image frame to the receiving end.
The video sending method according to any one of claims 1-5, wherein the determining an error correction refresh frame according to the current image frame comprises:

Determine a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame.
The video sending method according to claim 10, wherein the determining a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame comprises:

A preset number of image frames starting from the current image frame are used as error correction refresh frames.
The video sending method according to claim 11, wherein the performing intra-frame encoding on a partial area of the error correction refresh frame and sending the encoded error correction refresh frame to the receiving end comprises:

Use the current image frame as the first error correction refresh frame, perform intra-frame encoding on a partial area of the first error correction refresh frame, and send the encoded first error correction refresh frame to the receiving end ；

Obtain the next error correction refresh frame;

Performing intra-frame coding on a part of the error correction refresh frame, and sending the coded error correction refresh frame to the receiving end;

Return to the obtaining of the next error correction refresh frame, and perform intra-frame coding on a part of the error correction refresh frame, and continue the step of sending the encoded error correction refresh frame to the receiving end until the end An error correction refresh frame is encoded and sent to the receiving end.
The video sending method according to claim 12, wherein the union of the regions where the preset number of error correction refresh frames are intra-coded is equal to the range of any error correction refresh frame.
The video sending method according to claim 13, wherein the performing intra-frame coding on a partial area of the error correction refresh frame comprises:

Intra-frame coding is performed on a part of the error correction refresh frame, and inter-frame coding is performed on the remaining regions.
The video sending method according to claim 14, wherein the performing intra-frame coding on a partial area of the first error correction refresh frame comprises:

Perform intra-frame coding on the header area of the first error correction refresh frame.
The video sending method according to claim 15, wherein said performing intra-frame coding on a partial area of said error correction refresh frame, and sending the coded error correction refresh frame to said receiving end comprises :

Move back the area where the previous error correction refresh frame of the error correction refresh frame is intra-coded to obtain the area where the error correction refresh frame is intra-coded, and the area where the error correction refresh frame is intra-coded Perform intra-frame coding.
The video sending method according to claim 16, wherein the determining a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame comprises:

According to the number of strips in the current image frame, the same number of error correction refresh frames are determined in the current image frame and/or image frames subsequent to the current image frame.
The video sending method according to claim 17, wherein the performing intra-frame coding on the header area of the first error correction refresh frame comprises:

Perform intra-frame coding on the first slice of the first error correction refresh frame.
The video transmission method according to claim 18, wherein the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded is shifted backward to obtain the error correction refresh frame for intra-frame The coded area, performing intra-frame coding on the area where the error correction refresh frame is intra-coded, includes:

Add one to the stripe sequence number of the previous error correction refresh frame that is intra-coded in the error correction refresh frame to obtain the stripe sequence number of the error correction refresh frame that is intra-coded. Perform intra-frame coding on the slice of the slice number.
The video sending method according to claim 14, wherein the error feedback information includes area index information;

The performing intra-frame coding on a partial area of the first error correction refresh frame includes:

Determine the region where the first error correction refresh frame is intra-coded according to the region index information in the acquired error feedback information, and perform intra-frame coding on the region where the first error correction refresh frame is intra-coded.
The video sending method according to claim 20, wherein the performing intra-frame coding on a partial area of the error correction refresh frame comprises:

Move back the area where the previous error correction refresh frame of the error correction refresh frame is intra-coded to obtain the area where the error correction refresh frame is intra-coded, and the area where the error correction refresh frame is intra-coded Perform intra-frame coding.
22. The video sending method according to claim 21, wherein the area where the previous error correction refresh frame of the error correction refresh frame is intra-encoded is moved backward to obtain the error correction refresh frame for intra-frame The coded area includes:

If the region where the previous error correction refresh frame is intra-coded is not at the end of the previous error correction refresh frame, the region where the previous error correction refresh frame is intra-coded is moved backward to obtain the error correction Refresh the area of the frame for intra-frame coding;

If the area where the previous error correction refresh frame is intra-coded is at the end of the previous error correction refresh frame, the head area of the error correction refresh frame is determined as the error correction refresh frame for intra-frame encoding Area.
The video sending method according to claim 22, wherein the determining a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame comprises:

According to the number of strips in the current image frame, the same number of error correction refresh frames are determined in the current image frame and/or image frames subsequent to the current image frame.
The video sending method according to claim 23, wherein the determining the area of the first error correction refresh frame for intra-coding according to the area index information in the obtained error feedback information comprises:

It is determined that the strip corresponding to the strip number in the error feedback information in the first error correction refresh frame is an area for intra-frame coding.
24. The video sending method according to claim 24, wherein said shifting the region where the previous error correction refresh frame is intra-coded to obtain the region where the error correction refresh frame is intra-coded comprises :

The stripe sequence number of the previous error correction refresh frame subjected to intra-frame coding is increased by one to obtain the stripe sequence number of the error correction refresh frame subjected to intra-frame coding.
The video sending method according to claim 25, wherein the determining the header area of the error correction refresh frame as the area where the error correction refresh frame is intra-encoded comprises:

The first slice of the error correction refresh frame is determined as the slice of the error correction refresh frame that is intra-coded.
The video sending method according to claim 11, wherein, if error feedback information sent by the receiving end is received, determining an error correction refresh frame according to the current image frame comprises:

If error feedback information sent by the receiving end is received, an error correction sequence is determined according to the current image frame, and the error correction sequence includes a preset number of error correction refresh frames and the preset number of error correction refreshes The predicted frame after the frame.
The video sending method according to claim 27, wherein the performing intra-frame coding on a partial area of the error correction refresh frame and sending the coded error correction refresh frame to the receiving end comprises:

If receiving the correct feedback information sent by the receiving end when the error correction refresh frame is decoded, perform inter-frame encoding on the predicted frame in the error correction sequence, and send the encoded predicted frame to the receiving end .
The video sending method according to claim 28, wherein the performing inter-frame coding on the predicted frame in the error correction sequence and sending the coded predicted frame to the receiving end comprises:

Return to the obtaining of the current image frame, encode the current image frame, and continue the steps of sending the encoded current image frame to the receiving end until the error feedback information sent by the receiving end is received.
The video sending method according to any one of claims 1-5, wherein the error feedback information includes feedback information of the receiving end in response to a packet loss event, or includes the receiving end in response to completeness Check the error feedback information.
The video sending method according to any one of claims 1 to 5, wherein the current image frame is an image frame obtained in real time by a camera provided on the sending end.
A video receiving method applied to the receiving end, characterized in that it includes:

Receiving the encoded current image frame from the sending end, and decoding the current image frame;

If there is an error in receiving the current image frame or the decoding of the current image frame, error feedback information is sent to the sending end, and the receiving end performs error correction of intra-frame coding on the partial area sent by the sending end in response to the error feedback information Refresh frame

The error correction refresh frame is decoded, and the reference data used for decoding is refreshed according to the decoding result.
The video receiving method according to claim 32, wherein said decoding said current image frame comprises:

If the encoding format corresponding to the current image frame is inter-frame encoding, the current image frame is decoded according to the reference data used for decoding.
The video receiving method according to claim 33, wherein said decoding said current image frame comprises:

If the encoding format corresponding to the current image frame is intra-frame encoding, perform intra-frame decoding on the current image frame;

The reference data is refreshed according to the result of intra-frame decoding.
The video receiving method according to claim 34, wherein after the refreshing the reference data according to the result of intra-frame decoding, the method comprises:

Send correct feedback information to the sending end.
The video receiving method according to claim 33, wherein said decoding said current image frame comprises:

If the encoding format corresponding to the current image frame is intra-encoding of a partial area, perform intra-frame decoding on the intra-encoded area of the current image frame to obtain refresh segment data corresponding to the current image frame;

Part of the data in the reference data corresponding to the region encoded in the frame is updated to the refresh segment data.
The video receiving method according to claim 36, wherein said decoding said current image frame further comprises:

If the encoding format corresponding to the current image frame is intra-frame encoding of a partial area, perform inter-frame decoding on the remaining areas of the current image frame according to the reference data.
The video receiving method according to claim 36, wherein, after updating part of the data in the reference data corresponding to the intra-coded region to the refresh segment data, the method comprises:

If all data in the reference data is updated once, correct feedback information is sent to the sending end.
The video receiving method according to any one of claims 32-38, wherein if an error occurs in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end, include:

If a packet loss event occurs when receiving the encoded current image frame, send error feedback information to the sending end.
The video receiving method according to claim 39, wherein the receiving the encoded current image frame from the sending end and decoding the current image frame comprises:

Receiving, from the sending end, image region data encoded for each region of the current image frame;

Decoding each image area data of the current image frame.
The video receiving method according to claim 40, wherein if a packet loss event occurs when receiving the encoded current image frame, sending error feedback information to the sending end comprises:

If the encoded image area data is not correctly received from the sending end, error feedback information is generated according to the image area data, and the error feedback information is sent to the sending end.
The video receiving method according to claim 41, wherein the image area data includes a data packet sequence number;

If the encoded image area data is not correctly received from the sending end, generating error feedback information according to the image area data includes:

If the packet sequence number of the image area data is not continuous with the packet sequence number of the previous image area data, according to the packet sequence number of the previous image area data and the data packet sequence number of the first image area data corresponding to the current image frame Calculating area index information corresponding to the image area data;

Generate error feedback information including the area index information.
The video receiving method according to claim 41, wherein the image area data includes area index information;

The generating error feedback information according to the image area data includes:

Generate error feedback information including the area index information.
The video receiving method according to claim 40, wherein the image area data includes encoding format information;

The decoding each image region data of the current image frame includes:

Decoding the image area data according to a decoding manner corresponding to the encoding format information.
The video receiving method according to any one of claims 32-38, wherein if an error occurs in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end, include:

If the received current image frame fails the integrity check, error feedback information is sent to the sending end, where the error feedback information includes area index information corresponding to the header area of the current image frame.
The video receiving method according to any one of claims 32-38, wherein if an error occurs in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end, include:

If the decoding of the current image frame fails, error feedback information is sent to the sending end.
The video receiving method according to any one of claims 32-38, wherein the receiving the error correction refresh frame of the intra-encoded part of the area sent by the sending end in response to the error feedback information comprises :

A preset number of coded error correction refresh frames are received from the sending end, where the error correction refresh frame is that the sending end performs intra-frame coding on a partial area of the error correction refresh frame determined according to the error feedback information owned.
The video receiving method according to claim 47, wherein the decoding the error correction refresh frame and refreshing the reference data used for decoding according to the decoding result comprises:

Performing intra-frame decoding on the region coded in the error correction refresh frame to obtain refresh segment data corresponding to the error correction refresh frame;

The reference data is refreshed according to the refresh segment data corresponding to the error correction refresh frame.
The video receiving method according to claim 48, wherein the performing intra-frame decoding on the intra-frame coded area of the error correction refresh frame comprises:

Perform intra-frame decoding on the area coded in the error correction refresh frame, and perform inter-frame decoding on the remaining areas according to the reference data used for decoding.
The video receiving method according to claim 49, wherein the receiving the error correction refresh frame for intra-encoding a partial area sent by the sending end in response to the error feedback information comprises:

The encoded image area data is received from the sending end, where the image area data includes data after the sending end encodes a part of the error correction refresh frame, and encoding format information.
The video receiving method according to claim 50, wherein said performing intra-frame decoding on the intra-coded region of said error correction refresh frame to obtain refresh segment data corresponding to said error correction refresh frame comprises:

If the encoding format information in the image area data is intra-frame encoding, intra-frame decoding is performed on the data after the partial area encoding to obtain refresh segment data corresponding to the image area data.
The video receiving method of claim 51, wherein the refreshing the reference data according to the refresh segment data corresponding to the error correction refresh frame comprises:

Part of the data corresponding to the partial area in the reference data is updated to the refresh segment data.
The video receiving method according to claim 52, wherein the image area data further includes area index information of the partial area;

The performing intra-frame decoding on the encoded data of the partial area to obtain refresh segment data corresponding to the partial area includes:

Perform intra-frame decoding on the encoded data of the partial region to obtain refresh segment data corresponding to the region index information.
The video receiving method according to claim 53, wherein the updating part of the data corresponding to the partial area in the reference data to the refresh segment data comprises:

Part of the data corresponding to the area index information in the reference data is updated to the refresh segment data.
The video receiving method according to claim 48, wherein after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, correct feedback information is sent to the sending end.
The video receiving method according to claim 48, characterized in that, after refreshing the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, the current coded data received from the sending end is returned For image frames, the step of decoding the current image frame continues to be executed until receiving the current image frame error or decoding the current image frame error.
A video transmission method, characterized by comprising:

The sending end encodes the acquired current image frame, and sends the encoded current image frame to the receiving end;

The receiving end receives and decodes the current image frame, and if there is an error in receiving the current image frame or decoding the current image frame, sending error feedback information to the sending end;

The sending end determines an error correction refresh frame according to the error feedback information, performs intra-frame coding on a part of the error correction refresh frame, and sends the coded error correction refresh frame to the receiving end;

The receiving end decodes the error correction refresh frame, and refreshes the reference data used for decoding according to the decoding result.
The video transmission method according to claim 57, wherein the sending end determines whether the current image frame is an interval refresh frame according to an interval refresh condition;

If the current image frame is not an interval refresh frame, the sending end performs inter-frame encoding on the current image frame;

If the encoding format corresponding to the current image frame is inter-frame encoding, the receiving end decodes the current image frame according to the reference data used for decoding.
The video transmission method according to claim 58, wherein if the current image frame is an interval refresh frame, the sending end performs intra-frame coding on the current image frame;

If the encoding format corresponding to the current image frame is intra-frame encoding, the receiving end performs intra-frame decoding on the current image frame;

The receiving end refreshes the reference data according to the result of intra-frame decoding.
The video transmission method according to claim 59, wherein the receiving end sends correct feedback information to the sending end after refreshing the reference data according to the result of intra-frame decoding.
The video transmission method according to claim 58, wherein if the current image frame is an interval refresh frame, the sending end performs intra-frame coding on a partial area of the current image frame;

If the encoding format corresponding to the current image frame is intra-encoding of a partial area, the receiving end performs intra-frame decoding on the intra-encoded area of the current image frame to obtain refresh segment data corresponding to the current image frame;

The receiving end updates part of the data in the reference data corresponding to the region encoded in the frame to the refresh segment data.
The video transmission method according to claim 61, wherein if the current image frame is an interval refresh frame, the sending end performs intra-frame coding on part of the current image frame, and inter-frame coding on the remaining areas ；

If the encoding format corresponding to the current image frame is intra-frame encoding of a partial area, the receiving end performs inter-frame decoding on the remaining area of the current image frame according to the reference data.
The video transmission method according to claim 62, wherein if all data in the reference data is updated once, the receiving end sends correct feedback information to the sending end.
The video transmission method according to any one of claims 58-63, wherein if the number of the current image frame is within the refresh range of any refresh period, the sending end determines that the current image frame is an interval A refresh frame, where the refresh period includes adjacent refresh ranges and prediction ranges;

If the number of the current image frame is within the prediction range of any refresh period, the sending end determines that the current image frame is not an interval refresh frame.
The video transmission method according to any one of claims 58-63, wherein if the number of the current image frame is in the refresh subsequence of any image sequence, the sending end determines that the current image frame Is an interval refresh frame, the image sequence includes adjacent refresh subsequences and prediction subsequences, and the refresh subsequence includes at least one image frame at the beginning of the image sequence;

If the number of the current image frame is in the predicted subsequence of any image sequence, the sending end determines that the current image frame is not an interval refresh frame.
The video transmission method according to any one of claims 57-63, wherein the sending end performs intra-frame coding on a part of the error correction refresh frame, and transmits the coded error correction refresh frame to the After the receiving end sends, the steps of encoding the acquired current image frame and sending the encoded current image frame to the receiving end are continued until the error feedback information sent by the receiving end is received.
The video transmission method according to any one of claims 57-63, wherein the sending end encodes the acquired current image frame, and before sending the encoded current image frame to the receiving end, the method comprises:

The sending end performs intra-frame encoding on an image frame, and sends the encoded image frame to the receiving end;

The sending end performs inter-frame encoding on several image frames following the image frame according to the intra-encoded image frame, and sends the encoded image frame to the receiving end.
The video transmission method according to any one of claims 57-63, wherein the error feedback information includes feedback information of the receiving end in response to a packet loss event, or includes the receiving end in response to integrity Check the error feedback information.
The video transmission method according to claim 68, wherein the sending end determines a preset number of error correction refresh frames in the current image frame and/or image frames subsequent to the current image frame, and Intra-frame coding is performed on a partial area of each of the error correction refresh frames;

The receiving end receives a preset number of coded error correction refresh frames from the sending end.
The video transmission method according to claim 69, wherein the sending end performs intra-frame coding on part of the error correction refresh frame, and performs inter-frame coding on the remaining areas;

The receiving end performs intra-frame decoding on the area coded in the error correction refresh frame, and performs inter-frame decoding on the remaining areas according to the reference data used for decoding.
The video transmission method according to claim 70, wherein the union of the regions where the preset number of error correction refresh frames are intra-coded is equal to the range of any error correction refresh frame.
The video transmission method according to claim 70, wherein the receiving end performs intra-frame decoding on the region coded in the error correction refresh frame to obtain refresh segment data corresponding to the error correction refresh frame;

The receiving end refreshes the reference data according to refresh segment data corresponding to the error correction refresh frame.
The video transmission method according to claim 69, wherein the sending end uses a preset number of image frames starting from the current image frame as error correction refresh frames.
The video transmission method according to claim 69, wherein the sending end determines an error correction refresh frame according to the error feedback information, performs intra-frame coding on a part of the error correction refresh frame, and encodes the coded Sending the error correction refresh frame to the receiving end includes:

The sending end uses the current image frame as the first error correction refresh frame, performs intra-frame encoding on a partial area of the first error correction refresh frame, and sends the encoded first error correction refresh frame to all The receiving end sends;

Acquiring the next error correction refresh frame by the sending end;

The sending end performs intra-frame coding on a partial area of the error correction refresh frame, and sends the coded error correction refresh frame to the receiving end;

Return to the sending end to obtain the next error correction refresh frame, and the sending end performs intra-frame encoding on a partial area of the error correction refresh frame, and sends the encoded error correction refresh frame to the receiving end The steps continue to be executed until the last error correction refresh frame is encoded and sent to the receiving end.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 74, wherein the sending end performs intra-frame coding on the header area of the first error correction refresh frame;

The sending end moves back the area where the previous error correction refresh frame of the error correction refresh frame is intra-coded to obtain the area where the error correction refresh frame is intra-coded, and frame the error correction refresh frame The intra-coded area is intra-coded.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 75, wherein the sending end determines the number of strips in the current image frame and/or the image frame after the current image frame according to the number of stripes in the current image frame. Determine the same number of error correction refresh frames;

The transmitting end performs intra-frame coding on the first slice of the first error correction refresh frame;

The sending end adds one to the strip sequence number of the previous error correction refresh frame of the error correction refresh frame that is intra-coded to obtain the strip sequence number of the error correction refresh frame that is intra-coded, and corrects the error The slice corresponding to the sequence number of the slice in the refresh frame is intra-coded.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 74, wherein the error feedback information includes area index information;

The sending end determines the intra-encoded area of the first error correction refresh frame according to the area index information in the obtained error feedback information, and performs intra-frame encoding on the area of the first error correction refresh frame Internal coding

The sending end moves back the area where the previous error correction refresh frame of the error correction refresh frame is intra-coded to obtain the area where the error correction refresh frame is intra-coded, and frame the error correction refresh frame The intra-coded area is intra-coded.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 77, wherein if the region where the previous error correction refresh frame is intra-coded is not at the end of the previous error correction refresh frame, the sending end will An area where an error correction refresh frame performs intra-frame coding is moved backward, and an area where the error correction refresh frame performs intra-frame coding is obtained;

If the area where the previous error correction refresh frame is intra-encoded is at the end of the previous error correction refresh frame, the sending end determines the head area of the error correction refresh frame as the error correction refresh frame The area for intra-coding.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 77, wherein the sending end uses the number of strips in the current image frame in the current image frame and/or the image frame after the current image frame. Determine the same number of error correction refresh frames;

Determining, by the sending end, the strip corresponding to the strip number in the error feedback information in the first error correction refresh frame as an area for intra-coding;

If the region where the previous error correction refresh frame is intra-encoded is not at the end of the previous error correction refresh frame, the sending end adds one to the strip sequence number of the previous error correction refresh frame that is intra-encoded , Obtain the strip sequence number of the error correction refresh frame for intra-encoding;

If the area where the previous error correction refresh frame is intra-encoded is at the end of the previous error correction refresh frame, the sending end determines the first stripe of the error correction refresh frame as the error correction Refresh the frame for intra-encoded slices.
The video transmission method according to any one of claims 57-63, wherein if a packet loss event occurs when the receiving end receives the encoded current image frame, error feedback information is sent to the sending end.
[Correct 11.06.2019 in accordance with Rule 91]
The video transmission method according to claim 80, wherein the receiving end receives the image area data of each area of the current image frame encoded from the transmitting end, and decodes the image area data of the current image frame.
[Correct 11.06.2019 in accordance with Rule 91]
The video transmission method according to claim 81, wherein if the receiving end does not correctly receive the encoded image area data from the transmitting end, error feedback information is generated based on the image area data and sent to the transmitting end The terminal sends the error feedback information.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 82, wherein the image area data includes a data packet sequence number;

If the packet sequence number of the image area data is not continuous with the packet sequence number of the previous image area data, the receiving end uses the packet sequence number of the previous image area data and the first image area data corresponding to the current image frame Calculate the area index information corresponding to the image area data according to the data packet sequence number;

The receiving end generates error feedback information including the area index information.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 82, wherein the image area data includes area index information;

If the receiving end does not correctly receive the encoded image area data from the transmitting end, generate error feedback information including the area index information.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 80, wherein the image area data includes encoding format information;

The receiving end decodes the image region data according to a decoding manner corresponding to the encoding format information.
The video transmission method according to any one of claims 57-63, wherein if the current image frame received by the receiving end fails the integrity check, the receiving end sends an error to the sending end Feedback information, where the error feedback information includes area index information corresponding to the header area of the current image frame.
The video transmission method according to any one of claims 57-63, wherein if the receiving end fails to decode the current image frame, the receiving end sends error feedback information to the sending end.
The video transmission method according to claim 69, wherein the receiving end receives a preset number of coded error correction refresh frames from the sending end, comprising:

The receiving end receives encoded image area data from the transmitting end, where the image area data includes data encoded by the transmitting end on a partial area of the error correction refresh frame, and encoding format information.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 88, wherein if the encoding format information in the image area data is intra-frame encoding, the receiving end performs intra-frame decoding on the partial area-encoded data to obtain Refresh segment data corresponding to the image area data;

The receiving end updates partial data corresponding to the partial area in the reference data to the refresh fragment data.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 89, wherein the image area data further includes area index information of the partial area;

The receiving end performs intra-frame decoding on the data after the partial region encoding to obtain refresh segment data corresponding to the region index information;

The receiving end updates part of the data corresponding to the area index information in the reference data to the refresh segment data.
[Correct 11.06.2019 in accordance with Rule 91]
The video transmission method according to claim 89, wherein the receiving end refreshes the reference data according to the refresh segment data corresponding to the preset number of error correction refresh frames, and sends the correct data to the sending end. Feedback.
[Correct 11.06.2019 in accordance with Rule 91]

The video transmission method according to claim 91, wherein if the sending end receives error feedback information sent by the receiving end, the sending end determines an error correction sequence according to the current image frame, and the error correction sequence is The error sequence includes a preset number of error correction refresh frames and prediction frames after the preset number of error correction refresh frames;

The sending end performs intra-frame encoding on a partial area of the error correction refresh frame, and after sending the encoded error correction refresh frame to the receiving end, if the receiving end decodes the error correction refresh The correct feedback information sent at the time of the frame, the prediction frame in the error correction sequence is inter-frame coded, and the coded prediction frame is sent to the receiving end.
[Correct 11.06.2019 in accordance with Rule 91]
The video transmission method according to claim 92, wherein the sending end performs inter-frame coding on the predicted frame in the error correction sequence, and after sending the coded predicted frame to the receiving end, it returns to the The sending end encodes the acquired current image frame, and sends the encoded current image frame to the receiving end. The steps of receiving and decoding the current image frame by the receiving end are continued until the receiving end receives the current image frame. If there is an error in the image frame or decoding the current image frame, error feedback information is sent to the sending end.
A video sending device, characterized in that the video sending device includes a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program, and when executing the computer program, implement the following steps:

Acquiring a current image frame, encoding the current image frame, and sending the encoded current image frame to the receiving end;

If error feedback information sent by the receiving end is received, determine an error correction refresh frame according to the current image frame;

Perform intra-frame coding on a part of the error correction refresh frame, and send the coded error correction refresh frame to the receiving end.
[Correct 11.06.2019 in accordance with Rule 91]

An aircraft, characterized in that it comprises:

Image acquisition component, used to acquire images;

The video sending device according to claim 94, which is used to obtain the image collected by the image collecting component, and send it to the receiving end after encoding;

Flight components, used for flight.
A video receiving device, characterized in that the video sending device includes a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Receiving the encoded current image frame from the sending end, and decoding the current image frame;

If there is an error in receiving the current image frame or the decoding of the current image frame, error feedback information is sent to the sending end, and the receiving end performs error correction of intra-frame coding on the partial area sent by the sending end in response to the error feedback information Refresh frame

The error correction refresh frame is decoded, and the reference data used for decoding is refreshed according to the decoding result.
[Correct 11.06.2019 in accordance with Rule 91]

A video playback device, characterized by comprising:

The video receiving device according to claim 96, configured to receive an image from a sending end and decode it;

The display component is used to display the data decoded by the video receiving device.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the method described in any one of claims 1-31. The video sending method described; or

Implement the video receiving method according to any one of claims 32-56.