WO2021102992A1

WO2021102992A1 - Data transmissio method, device and system as well as storage medium

Info

Publication number: WO2021102992A1
Application number: PCT/CN2019/122168
Authority: WO
Inventors: 周士贞; 苏文艺; 赵亮
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-11-29
Filing date: 2019-11-29
Publication date: 2021-06-03
Also published as: CN112771830A

Abstract

A data transmission method, device, system and a storage medium, wherein the method comprises: obtaining a target resolution (S101); obtaining first image data after a first encode is reconstructed (S102); obtaining first code stream data encoded by a second encoder, wherein the first code stream data corresponds to the target resolution (S103).

Description

Data transmission method, device, system and storage medium

Technical field

This application relates to the field of communication technology, and in particular to a data transmission method, device, system and storage medium.

Background technique

In a video coding sequence, there are mainly three types of coding frames: I frames, P frames, and B frames. The I frame does not refer to other image frames, so usually when the code stream state is switched, such as switching the resolution, an I frame is sent first to ensure that the referenced video frame is correct and valid when the subsequent video frame is decoded.

In the field of real-time video wireless transmission, the wireless channel is a complex, changeable and unstable communication condition. After the channel quality deteriorates, on the one hand, the bit stream data sent by the encoding end cannot be sent to the decoding end in a timely manner, and error packets may occur. On the other hand, a reduced-resolution transmission strategy can be adopted.

At present, after the channel quality deteriorates, it is impossible to guarantee the correctness and validity of the reference frames at the encoding end and the decoding end in time.

Summary of the invention

Based on this, this application provides a data transmission method, a data transmission device, a data transmission system, and a storage medium.

In the first aspect, this application provides a data transmission method, including:

Obtain the target resolution;

Acquiring the first image data reconstructed by the first encoder;

Obtain the first code stream data encoded by the second encoder, where the first code stream data corresponds to the target resolution.

In the second aspect, this application provides a data transmission method, including:

Obtain the target resolution;

Acquiring the third image data reconstructed by the first decoder;

Acquiring second code stream data encoded by the third encoder, where the second code stream data corresponds to the target resolution;

Obtain the fourth image data after the second decoder decodes the second code stream data.

In a third aspect, the present application provides a data transmission device, the device includes: a memory and a processor;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Obtain the target resolution;

Acquiring the first image data reconstructed by the first encoder;

In a fourth aspect, the present application provides a data transmission device, the device includes: a memory and a processor;

The memory is used to store a computer program;

Obtain the target resolution;

Acquiring the third image data reconstructed by the first decoder;

In a fifth aspect, the present application provides a data transmission system, which includes the data transmission device described in the third aspect and the data transmission device described in the fourth aspect.

In a sixth aspect, the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes the data described in the first aspect. Transmission method.

In a seventh aspect, the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes the data described in the second aspect above. Transmission method.

The embodiments of the present application provide a data transmission method, device, system and storage medium. The data transmission sending end (encoding end): obtains the target resolution; obtains the first image data reconstructed by the first encoder; obtains the second code The first code stream data corresponding to the target resolution after encoding by the encoder; since the encoding end first obtains the reconstructed first image data after obtaining the target resolution, it is equivalent to the image data decoded by the analog decoding end, the first image data Matching with the decoded image data at the decoding end, the first image data can be used as the basic image data of the reference image data of the subsequent image data. In this way, correct and effective reference image data can be provided for the subsequent image data in time, especially Provide correct and effective reference image data to provide support for encoding the image data after the subsequent resolution change in time. Data transmission receiving end (decoding end): Obtain the target resolution; Obtain the third image data reconstructed by the first decoder; Obtain the second code stream data corresponding to the target resolution encoded by the third encoder; Obtain the second decoding The fourth image data after the decoder decodes the second code stream data; since the decoder first obtains the reconstructed third image data after obtaining the target resolution, the third image data can be used as reference image data for subsequent image data In this way, it can provide correct and effective reference image data support for subsequent image data in a timely manner, in particular, it can provide correct and effective reference image data for decoding the subsequent image data after the resolution is changed in time. In the case of poor channel quality, there is no need to retransmit the complete, large-rate reference image data bit stream data at the encoding end, and the decoding end can also obtain correct and effective reference image data in time, thereby ensuring that the decoded Image quality provides support; since the decoding end simulates the encoding process of the encoding end corresponding to the target resolution and performs decoding, in this way, technical support can be provided to ensure the seamless connection between the decoding end and the encoding end. Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data are matched. In this way, the encoding end can only send the Provide technical support for error-corrected frames at the decoding end.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of an embodiment of a data transmission method according to the present application;

2 is a schematic flowchart of another embodiment of the data transmission method of the present application;

FIG. 3 is a schematic flowchart of another embodiment of the data transmission method of the present application;

4 is a schematic flowchart of another embodiment of the data transmission method of the present application;

FIG. 5 is a schematic flowchart of another embodiment of the data transmission method of the present application

Fig. 6 is a flow diagram of the sending end and the receiving end of the first application of the data transmission method of this application before the resolution is switched;

FIG. 7 is a flowchart of the sending end and the receiving end during the switching process of the switching resolution of the application of FIG. 6;

Fig. 8 is a flow chart of the sending end and the receiving end of the application of Fig. 6 after the resolution switching is completed;

FIG. 9 is a schematic structural diagram of an embodiment of a data transmission device of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

In a video coding sequence, there are mainly three types of coding frames: I frames, P frames, and B frames. The I frame does not refer to other image frames. Usually, when the code stream state is switched, an I frame is sent first to ensure that the reference video frame is correct and effective when the subsequent video frame is decoded. In the field of real-time video wireless transmission, the wireless channel is a complex, changeable and unstable communication condition. After the channel quality deteriorates, the correct and effective reference video frames of the encoder and decoder cannot be guaranteed in time.

The data transmission sending end (encoding end) of the embodiment of the present application: obtains the target resolution; obtains the first image data reconstructed by the first encoder; obtains the first code stream data corresponding to the target resolution encoded by the second encoder; Since the encoding end first obtains the reconstructed first image data after acquiring the target resolution, which is equivalent to the image data decoded by the analog decoding end. The first image data matches the decoded image data of the decoding end, and the first image data can be As the basic image data of the reference image data of the subsequent image data, in this way, it can provide correct and effective reference image data for the subsequent image data in time, especially the correct and effective image data for the subsequent image data after the resolution is changed. Provide support for the reference image data, such as resampling on the basis of the first image data. Data transmission receiving end (decoding end): Obtain the target resolution; Obtain the third image data reconstructed by the first decoder; Obtain the second code stream data corresponding to the target resolution encoded by the third encoder; Obtain the second decoding The fourth image data after the decoder decodes the second code stream data; since the decoder first obtains the reconstructed third image data after obtaining the target resolution, the third image data can be used as reference image data for subsequent image data In this way, it can provide correct and effective reference image data support for subsequent image data in a timely manner, in particular, it can provide correct and effective reference image data support for the subsequent decoding of image data after the resolution is changed in time, In the case of poor channel quality, the encoder does not need to retransmit the complete, large-rate reference data stream data, and the decoder can also obtain correct and effective reference image data in time, thereby ensuring the decoded image Quality support; since the decoding end simulates the encoding process of the encoding end corresponding to the target resolution and performs decoding, in this way, technical support can be provided to ensure the seamless connection between the decoding end and the encoding end. Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data match, and the decoding end simulates the encoding process of the encoding end and performs decoding. In this way, It can provide technical support for the encoding end to only generate error correction frames to the decoding end when the subsequent error correction is required.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Referring to FIG. 1, FIG. 1 is a schematic flowchart of an embodiment of a data transmission method according to the present application. It should be noted that the method in the embodiment of the present application is a data transmission method at the encoding end. The data transmission method at the encoding end includes: step S101, step S102, and step S103.

Step S101: Obtain a target resolution.

Step S102: Obtain the first image data reconstructed by the first encoder.

Step S103: Obtain the first code stream data encoded by the second encoder, where the first code stream data corresponds to the target resolution.

In this embodiment, the target resolution refers to the resolution of image data that currently needs to be encoded. In an application scenario, the target resolution matches the quality of the data transmission channel. The quality of the data transmission channel can be current or pre-calculated. Specifically, when the quality of the data transmission channel is good, the target resolution can be increased to transmit as large an image as possible, and the size of the decoding end image can be increased as much as possible on the basis of ensuring the image quality; when the quality of the data transmission channel is poor At this time, the target resolution can be reduced to achieve the encoding reduction of the output code stream, reduce the error packet loss as much as possible, and ensure that the image quality of the decoding end is basically acceptable. In a specific application, a list of correspondences between the quality level of the data transmission channel and the matched target resolution can be preset, and the matched target resolution can be determined according to the currently detected quality level of the data transmission channel. In particular, the quality of the data transmission channel can be detected by the encoding end, so that timely feedback on changes in channel quality can be obtained.

The first image data refers to the reconstructed image data obtained from the first encoder according to the coding principle. The first code stream data can be obtained after encoding the image data corresponding to the target resolution in the second encoder. The resolution of the first image data may be the same as the target resolution, for example, before the resolution switching; the resolution of the first image data may also be different from the target resolution, for example, during the resolution switching process and the switching It can be different after completion. Among them, the first encoder and the second encoder can be two independent hardware encoders, or multiplex the same hardware encoder, that is, use the same hardware encoder for time-sharing encoding: as the first The encoder performs encoding, and performs encoding as a second encoder in some other periods.

Under normal circumstances, after the encoding end obtains the target resolution, the first code stream data can be obtained after encoding the image data corresponding to the target resolution. However, after obtaining the target resolution in the embodiment of this application, before encoding, it also obtains the first image data reconstructed by the first encoder, which is equivalent to simulating the image data decoded by the decoder. The first image data can be used as the basic image data of the reference image data of the subsequent image data. In this way, it can be ensured that the reference image data of the encoding end is the same as the reference image data of the decoding end after the resolution is switched, so that it can be timely Provide correct and effective reference image data for subsequent image data, in particular, provide support for correct and effective reference image data for encoding of image data after subsequent resolution changes in time, so as to ensure correct decoding at the decoder.

Referring to Figure 2, in one embodiment, when switching resolutions, in order to facilitate the synchronization of the encoding end and the decoding end, maintain a switching process of switching resolutions before the formal switching (that is, the intermediate state of the switching), that is, through Obtain the image data corresponding to the target resolution in a simulated manner, so as to maintain a switching process of switching resolutions at the encoding end, that is, after obtaining the first image data reconstructed by the first encoder in step S102, the method may further include: S104.

Step S104: Obtain second image data after resampling the first image data, where the second image data is image data corresponding to the first code stream data.

In this embodiment, the resolution of the second image data is the target resolution, which is image data obtained by re-sampling the first image data (that is, obtained by simulation). Through the re-sampling module on the encoding end and the decoding end, it is possible to ensure that the images output by the two ends in the resolution switching match. In particular, the degree of resampling at the encoding end and the decoding end can be the same to ensure the consistency of image transmission during the resolution switching process. Then, step S103 is to encode the second image data by the second encoder to obtain the first code stream data.

Specifically, step S104 may have the following two situations:

The first type: if the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.

Downsampling (subsampled), also known as reduced image or downsampling (downsampled), has two main purposes: (1) Make the image fit the size of the display area; (2) Generate a thumbnail of the corresponding image, and the thumbnail is relative Compared with the original image, the redundancy can be removed, and the occupied bandwidth can be reduced when the data is transmitted after encoding. For example, when the channel quality is degraded, the redundancy can be removed by down-sampling, and the packet loss and error packet can be minimized when the data is transmitted after encoding. , As far as possible to ensure the image quality of the decoding end.

Down-sampling principle: For an image I with a size of M*N, perform s times downsampling to obtain a resolution image of (M/s)*(N/s) size. Of course, s should be M and N The common divisor works. If you consider the image in matrix form, the image in the s*s window of the original image is turned into a pixel, and the value of this pixel is the mean value of all pixels in the window.

The second type: if the target resolution is less than the resolution of the first image data, down-sampling the first image data to obtain the second image data of the target resolution.

Upsampling (upsampling), also called enlarged image or image interpolation (interpolating), its main purpose is to enlarge the image before sampling (sampled image), so that it can be displayed on a higher resolution display device, or get a large size Image.

Up-sampling principle: Image magnification almost always uses interpolation methods, that is, on the basis of the original image pixels, a suitable interpolation algorithm is used to insert new elements between pixels.

There are many ways to sample, whether you zoom in (downsampling) or zoom in (upsampling). Such as nearest neighbor interpolation, bilinear interpolation, mean interpolation, median interpolation and other methods.

Wherein, whether the first code stream data obtained by encoding in step S103 is sent depends on the current state of the encoding end:

(1) If the current state includes the switching intermediate state, discard the first code stream data.

Switching the intermediate state means that the resolution switching has not been officially completed during the switching process of the resolution. Therefore, the first bit stream data at this time does not need to be sent, and the first bit stream data is discarded, which can avoid the first bit stream data. Take up storage space. In an application, the switching intermediate state of the encoding end includes the state of maintaining encoding with different resolutions.

(2) If the current status includes the switching completion status, send the first code stream data.

The switching completion state means that the resolution switching is officially completed. At this time, the first bit stream data is sent to the decoding end. On the one hand, it can ensure that the resolution switching is completed, and on the other hand, it can ensure the synchronous switching with the decoding end. In an application, the switching completion state of the encoding end includes the state of maintaining the resolution encoding after switching.

In another embodiment, if the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data corresponds to the first bitstream data. Image data.

In this embodiment, the second image data is the image data corresponding to the target resolution, and is the image data corresponding to the first bitstream data; wherein, the second image data may also be the image data after the first image data is resampled. In this way, it can ensure that the images and code streams output during the resolution switching process at both ends are consistent. After the switch is completed, the resolution of the input image data is the target resolution. When the input image data needs to be encoded into other video frames (for example: P frames) other than the I frame, the second image data can be used as the new input image The reference image data of the data encodes the new input image data.

Wherein, the new input image data includes the image data of the target resolution obtained by re-sampling or the image data of the target resolution directly output by the camera device. That is to say, the image source input by the encoding end can have two schemes: one is that the output image of the camera device keeps the resolution A unchanged, and the image of resolution A is resampled to obtain the image of resolution B; the other is Switching the camera device, the input image is changed from the original resolution A image to the resolution B image; the purpose of these two solutions is to obtain an image of the new resolution B, but the implementation is different.

In order to enable the decoding end to switch to the target resolution synchronously with the encoding end, the switching information needs to be sent to the decoding end before the formal switching, so that the decoding end can obtain relevant switching information such as the target resolution. That is, the method further includes: step S105.

Step S105: Generate and send switching information, where the switching information includes the synchronization switching identifier and the target resolution.

Among them, the synchronization switching identifier includes the switching resolution status identifier and the synchronization identifier.

In an embodiment, the switching resolution state identifier includes the identifier of the state before switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.

In an embodiment, the synchronization identifier includes a frame number statistical information identifier at the target resolution after switching.

For example, the pre-defined switching information is: the switch resolution status identifier (status, 8bit), the frame number statistical information identifier under the new resolution (fidx, 32bit), and the target resolution size after switching (width, 16bit; height, 16bit); status=0 means no switching (that is, status before switching), status=1 means switching (that is, switching intermediate status), status=2 means after switching (that is, switching completion status), after the decoder receives the switching information, if it detects The switch resolution status bit status of the switching information is 1 to enter the switching intermediate state, and it is detected that the switch resolution status bit of the switching information is 2 to enter the switching completion state. If the resolution 1920x1080 is switched to the target resolution 1280x720 from the Nth frame, the related switching information of the N-1th frame can be (status=0, fidx=N-1, width=1920, height=1080), which means The N-1th frame is in the state before switching, and the switching information of the Nth frame may be (status=1, fidx=N, width=1280, height=720), which indicates that the Nth frame is in the intermediate state of switching. Illustratively, in the intermediate state of resolution switching, especially during the re-sampling process, both ends will also send a synchronization switching flag in one or two directions to ensure that the output of the re-sampling module at the decoding end is consistent with the re-sampling output at the encoding end.

The easier ways to implement the handover information include: the handover information can be included in the communication protocol or supplemented in the Supplemental Enhancement Information (SEI), that is, the method for sending the handover information in step S105 can be easily implemented in the following two ways: Ways:

One is: the switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.

The other is to generate and send supplementary enhancement information SEI including handover information.

Normally, step S105 is triggered to execute a switching instruction, that is, switching information is generated and sent according to the switching instruction. Specifically, the generation of the switching instruction may be automatically generated by the encoding end (for example, when the quality of the data transmission channel is detected to be degraded), or may be generated according to a user instruction. The specific description is as follows:

In an application, the method further includes: step S106.

Step S106: When it is detected that the quality of the data transmission channel does not meet the preset condition, a switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.

Wherein, in step S106, when it is detected that the quality of the data transmission channel does not meet the preset condition, the switching instruction is generated, which may specifically include: sub-step S1061, sub-step S1062, and sub-step S1063.

Sub-step S1061: Detect and determine the magnitude of the data transmission channel quality, the first preset channel quality requirement, and the second preset channel quality requirement.

Sub-step S1062: When it is detected that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, a first switching instruction is generated, wherein the target resolution of the first switching instruction is greater than the resolution of the first image data.

Sub-step S1063: When it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.

Further, the first preset channel quality requirement is the same as the second preset channel quality requirement, and both are preset channel quality requirements. The preset channel quality requirement includes a preset channel quality threshold T and a preset duration of t seconds. At this time, the sub-step S1062 may specifically include: when it is detected that the quality of the data transmission channel is greater than or equal to the preset channel quality threshold T for t seconds, generating a first handover instruction; the sub-step S1063 may specifically include: when the quality of the data transmission channel is detected If the duration of t seconds is less than the preset channel quality threshold T, a second switching instruction is generated. When the first preset channel quality requirement is different from the second preset channel quality requirement, it can also be detected that the data transmission channel quality is less than the first preset channel quality requirement and greater than or equal to the second preset channel quality requirement , The resolution remains unchanged.

In another application, the method further includes: receiving an input user instruction, where the user instruction includes the target resolution; and generating a switching instruction according to the user instruction. In this way, a suitable target resolution can be matched according to the actual needs of the user, and the user experience can be further improved.

Refer to Figure 3, which is a schematic flow diagram of another embodiment of the data transmission method of the present application. It should be noted that the method of the embodiment of the present application is a data transmission method of the decoding end. For the content associated with the encoding end, please refer to the above-mentioned encoding end. The part of the data transmission method will not be repeated here. The following specifically introduces the content that is not related to or different from the encoding end.

The data transmission method at the decoding end includes: step S201, step S202, step S203, and step S204.

Step S201: Obtain the target resolution.

Step S202: Obtain the third image data reconstructed by the first decoder.

Step S203: Obtain the second code stream data encoded by the third encoder, where the second code stream data corresponds to the target resolution.

Step S204: Obtain fourth image data after the second decoder decodes the second code stream data.

The third image data refers to the reconstructed image data obtained from the first decoder according to the decoding principle (corresponding to the encoding principle). According to the encoding and decoding principle, it can be known that the third image data at the decoding end matches the first image data at the encoding end. The second code stream data can be obtained by encoding the image data corresponding to the target resolution in the third encoder, and the fourth image data can be obtained by decoding the second code stream data by the second decoder. The resolution of the third image data and the target resolution can be the same, for example, before the resolution switching; the resolution of the third image data and the target resolution can also be different, for example, during the resolution switching process and the switching It can be different after completion. Among them, the first decoder and the second decoder can be two independent hardware decoders, or the same hardware decoder can be multiplexed, that is, the same hardware decoder is used for time-sharing decoding: in certain periods as the first The decoder performs decoding, and performs decoding as a second decoder in some other periods.

Normally, after the decoder obtains the target resolution, it can decode the code stream data corresponding to the target resolution. However, after obtaining the target resolution in the embodiment of the present application, the third image data reconstructed by the first decoder is obtained, and the second code stream data corresponding to the target resolution encoded by the third encoder is obtained, which is equivalent to analog encoding at the encoding end. The latter bitstream data is then decoded. Since the third image data can be used as the basic image data of the reference image data when decoding the bitstream data of the subsequent image data, in this way, it can provide correct and effective follow-up image data in time. In particular, it can provide correct and effective reference image data to provide support in time for the subsequent decoding of the image data after the resolution has been changed. In the case of poor channel quality, the encoder does not need to retransmit the complete, large-rate reference data stream data, and the decoder can also obtain correct and effective reference image data in time, thereby providing support for correct decoding , Provide support to ensure the image quality after decoding. Since the decoding end simulates the encoding process of the encoding end corresponding to the target resolution and performs decoding, in this way, technical support can be provided to ensure seamless connection between the decoding end and the encoding end. Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data match, and the decoding end simulates the encoding process of the encoding end and performs decoding. In this way, It can provide technical support for the encoding end to only generate error correction frames to the decoding end when the subsequent error correction is required.

Referring to Fig. 4, in one embodiment, when switching resolutions, in order to facilitate the synchronization of the encoding end and the decoding end, maintain a switching process of switching resolutions before the formal switching (that is, the intermediate state of switching), that is, through The image data corresponding to the target resolution is obtained by simulation, so as to maintain a switching process of switching resolution at the decoding end, that is, in step S202, after obtaining the third image data reconstructed by the first decoder, the method further includes: step S205 .

Step S205: Obtain fifth image data after resampling the third image data, where the fifth image data is image data corresponding to the second code stream data.

In this embodiment, the resolution of the fifth image data is the target resolution, which is image data obtained by re-sampling the third image data (that is, obtained by simulation). Through the re-sampling module on the encoding end and the decoding end, it is possible to ensure that the images output by the two ends in the resolution switching match. In particular, the degree of resampling at the encoding end and the decoding end can be the same to ensure the consistency of image transmission during the resolution switching process. Then, step S203 is to encode the fifth image data by the third encoder to obtain the second code stream data.

Specifically, step S205 may have the following two situations:

The first type: if the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain the fifth image data of the target resolution.

The second type: if the target resolution is less than the resolution of the third image data, down-sampling the third image data to obtain the fifth image data of the target resolution.

Wherein, whether the fourth image data obtained after decoding in step S204 is retained depends on the current state of the decoding end:

(1) If the current state includes the switching intermediate state, discard the fourth image data.

The switching intermediate state means that the resolution switching has not been officially completed during the switching of the resolution. Therefore, the fourth image data does not need to be retained at this time, and the fourth image data is discarded, which can prevent the fourth image data from occupying storage space . In one application, the switching intermediate state of the decoder includes maintaining the state of decoding with different resolutions.

(2) If the current state includes the switching completion state, receive the first code stream data.

The switching completion state means that the resolution switching is officially completed. At this time, the first bit stream data from the encoding end can be received. On the one hand, it can ensure the completion of the resolution switching, and on the other hand, it can ensure the synchronous switching with the encoding end. In an application, the switching completion state of the decoding end includes the state of maintaining the resolution decoding after switching.

In another embodiment, if the current state includes the switching completion state, the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new code stream data of the new input image data is decoded. Five image data is the image data corresponding to the second code stream data.

In this embodiment, the fifth image data is the image data corresponding to the target resolution, and is the image data corresponding to the second bit stream data; wherein, the fifth image data may also be the image data after re-sampling the third image data. . After the switch is completed, the resolution of the input image data is the target resolution. When it is necessary to decode other video frames other than the I frame (for example: P frame) corresponding to the input image data, the fifth image data can be used as the new The reference image data of the input image data decodes other video frames other than the I frame corresponding to the new input image data.

Referring to FIG. 5, in an embodiment, before acquiring the target resolution, the decoding end further includes: step S206.

Step S206: Receive switching information, where the switching information includes the synchronization switching identifier and the target resolution.

For example, the pre-defined switching information is: the switch resolution status identifier (status, 8bit), the frame number statistical information identifier under the new resolution (fidx, 32bit), and the target resolution size after switching (width, 16bit; height, 16bit); status=0 means no switching (that is, status before switching), status=1 means switching (that is, switching intermediate status), status=2 means after switching (that is, switching completion status), after the decoder receives the switching information, if it detects The switch resolution status bit status of the switching information is 1 to enter the switching intermediate state, and it is detected that the switch resolution status bit of the switching information is 2 to enter the switching completion state. If the resolution 1920x1080 is switched to the target resolution 1280x720 from the Nth frame, the relevant switching information of the N-1th frame can be (status=0, fidx=N-1, width=1920, height=1080), which means The N-1th frame is in the state before switching, and the switching information of the Nth frame may be (status=1, fidx=N, width=1280, height=720), which indicates that the Nth frame is in the intermediate state of switching. Illustratively, in the intermediate state of resolution switching, especially during the re-sampling process, both ends will also send a synchronization switching flag in one or two directions to ensure that the output of the re-sampling module at the decoding end is consistent with the re-sampling output at the encoding end.

The easier ways to implement the switching information include: the switching information can be included in the communication protocol (together with the code stream data) or supplemented in the supplementary enhancement information SEI, that is, the switching information receiving method in step S206 can be easily implemented in the following ways Two ways:

One is: receiving the first code stream data including switching information. The first stream data comes from the encoding end. The other is: receiving supplementary enhancement information SEI including handover information.

The following describes in detail the specific details of an application that combines the data transmission method at the encoding end with the data transmission method at the decoding end.

Please refer to Figures 6 to 8. Figure 6 is a flow diagram of the sending and receiving ends of the application of the data transmission method 1 of this application before switching the resolution. Figure 7 is the sending and receiving ends of the application of Figure 6 during the switching of the resolution. Fig. 8 is a flowchart of the sending end and the receiving end of the application of Fig. 6 after the resolution switching is completed.

As shown in FIG. 6, the first encoder at the encoding end encodes and sends the input image data (resolution PA) before the resolution is switched, and the decoder receives the code stream data SA and decodes the output image data (resolution PA). When receiving the switching instruction, it switches to the intermediate state of resolution switching as shown in Figure 7. The switching command varies according to the switching purpose. Generally, there are two sources: automatic switching according to the data transmission channel quality detection result and switching according to user instructions. The former can be used in the scenario where the transmission code stream is adaptively matched to the channel bandwidth. According to the channel quality detection result, it is measured whether to switch the resolution. For example, when the channel quality is relatively good, the input image is encoded and sent according to the size of 1920x1080. When the channel quality is detected to be worse and is lower than the channel quality threshold T for t seconds, the resolution switch is started, and the input image is downsampled Encode and send in a size of 1280x720 to relieve transmission pressure. Similarly, when it is detected that the communication conditions are better, it can be up-sampled from a lower resolution to a higher resolution for encoding and sending. It is worth mentioning that the degree of resampling can be flexibly configured, but the encoding end and the decoding end must be consistent.

As shown in Figure 7, during the switching process of switching resolution, the part inside the dashed frame plays a role in maintaining the switching intermediate state. The switching process is mainly composed of two parts: the encoding end switching intermediate state maintenance and the decoding end switching intermediate state respectively maintain. The function of the intermediate state maintenance part of the encoding end switching is to simulate the decoded image at the decoding end and use it as a reference frame to encode the image after the size change. In this way, it can smoothly enter the encoding transmission state after the resolution is switched. The function of the switch intermediate state maintenance part of the decoding end is to simulate receiving and decoding the bit stream data after the resolution is switched. The biggest advantage of this scheme is that the encoding end and the decoding end can be seamlessly connected, and it can avoid sending large-rate I frames when switching resolutions; this resolution switching scheme is channel-friendly, has short transmission delay, and effectively avoids traditional The screen resolution, jitter, flicker and other issues caused by cutting resolution. The traditional cut resolution scheme usually uses I frame as the start of the new resolution. In order to achieve better subjective quality, a larger volume of I frame is usually required. The code rate of I frame is positively correlated with the image quality after decoding. The higher the bit rate, the greater the transmission pressure, especially in the case of poor channel quality, the transmission delay is very large, so traditional cutting resolution usually brings bad subjective such as blurring, jitter, and flicker. Experience.

The above scheme will be described in further detail below according to the scenario of adaptively matching the channel bandwidth of the transmission code stream.

First introduce the scheme of the encoding end. When switching the resolution, first encode the input image according to the pre-switching procedure. As shown in Figure 7, the image data of the resolution PA is input to the first encoder, and the code stream data SA output by the first encoder is sent. At the same time, the reconstructed image data (that is, the first image data with a resolution of PA) is taken from the first encoder and input to the resampling module, and the matching resolution is selected according to the level of channel quality ( That is, the target resolution), and update the communication protocol. The communication protocol includes the resolution status, the frame number statistics information under the new resolution, and the resolution size information after switching (that is, the target resolution). The reconstructed image data (that is, the first image data) is down-sampling and outputs the image data of the resolution PB' (that is, the second image data), and the image data of the resolution PB' is input to the second encoder for processing Encoding, the coded stream data SB' (that is, the first stream data) is temporarily discarded. This part of the solution realizes the state of maintaining the encoding of different resolutions (PA and PB').

Then introduce the solution of the decoding end. When switching resolutions, first decode the received bitstream data according to the procedure before switching. As shown in Figure 7, the first decoder decodes the received code stream data SA to obtain the image data of the resolution PA, and at the same time obtains the resolution switching state from the state of the communication protocol at both ends of the codec. When the current state is detected When switching the intermediate state for the resolution, take out the decoded reconstructed image (that is, the third image data with a resolution of PA) from the first decoder, and input it to the resampling module, and configure the resampling module according to the information carried in the communication protocol, The function of this part is to ensure that the output of the re-sampling module at the decoding end is consistent with the output of the re-sampling module at the encoding end, and then input the resampled image data with a resolution of PB' (that is, the fifth image data) into the third encoder for encoding. Then the code stream data (ie, the second code stream data) encoded by the third encoder is input to the second decoder, and the image data of the resolution PB' (ie, the fourth image data) decoded by the second decoder is temporarily Discarded, this part of the solution realizes the state of maintaining the decoding of different resolutions (PA and PB').

It is worth noting that since the input image at the encoder is sequentially encoded frame by frame, there is a one-to-one correspondence between the image frames in the reconstruction buffer (buffer) at the encoder and the decoder, so there is no need to re-encode when switching resolutions. The I frame is sent to the decoder, and the decoder only needs to take the corresponding frame from the buffer as a reference frame. This feature can also be adapted to different error recovery strategies, such as Ondemand_I, Ondemand_Ref. In the scenario where Ondemand_I is applied, when the decoder makes an error, it requests an error correction I frame from the encoder; in the scenario where Ondemand_ref is applied, when an error occurs, the decoder requests an error correction P frame from the encoder. If the P frame exceeds the queue range, then request an I frame. .

As shown in Figure 8, after the resolution switching is completed, the second encoder on the encoding end corresponds to the second encoder on the encoding end in Figure 7 (switching intermediate state), and the encoding end encodes the code stream data SB (ie The first bit stream data) is sent, and the input image data of resolution PB after switching corresponds to the image data of resolution PB' input in the switching process as shown in Figure 7. The resolution of the two is the same, but the image data of resolution PB It is the newly input video frame, and the image data of the resolution PB' is obtained by re-sampling the reconstructed video frame after encoding the original video frame.

As shown in Figure 8, after the resolution switching is completed, the second decoder at the decoding end corresponds to the second decoder at the decoding end in Figure 7 (switching intermediate state), and the decoding end receives the code stream data SB (that is, the first code stream data). ) Is directly input to the second decoder for decoding and output resolution PB image data. The received bit stream data SB corresponds to the bit stream data encoded and output by the third encoder at the decoding end. The resolution of the two is the same, but SB is wireless The received video code stream of the new resolution, and the code stream data encoded and output by the third encoder at the decoding end is the video code stream of the image after re-sampling the original resolution image. Through the solution in the embodiment of the present application, the consistency of the reference frame sequence of the encoding end and the decoding end can be ensured, so when the resolution is switched, the encoding end does not need to re-encode the I frame and send it to the decoding end.

The encoder input image source has two schemes: one is to switch the camera (ie camera device), the input image resolution is changed from PA to PB; the other is to keep the PA of the camera output image unchanged, and resample the image data of the resolution PA Obtain image data with a resolution of PB. The purpose of the two is to obtain the image data of the new resolution PB, but the realization method is different. In addition, the switching information can be included in the communication protocol part and sent as bit stream data, or it can be supplemented with the SEI information part sent in the H264 coding standard.

Refer to Figure 9. Figure 9 is a schematic structural diagram of an embodiment of the data transmission device of the present application. The data transmission device of this embodiment can execute the data transmission method on the encoding side. For detailed description of the relevant content, please refer to the above data transmission method on the encoding side. I won't repeat it again.

The device 10 includes: a memory 11 and a processor 12; the memory 11 and the processor 12 are connected by a bus 13.

Among them, the processor 12 may be a micro control unit, a central processing unit, or a digital signal processor, and so on.

Among them, the memory 11 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk, or a mobile hard disk, etc.

The memory 11 is used to store a computer program; the processor 12 is used to execute the computer program and when the computer program is executed, the following steps are implemented:

Obtain the target resolution; obtain the first image data reconstructed by the first encoder; obtain the first code stream data encoded by the second encoder, the first code stream data corresponding to the target resolution.

Wherein, when the processor executes the computer program, it implements the following steps: acquiring second image data after re-sampling the first image data, where the second image data is image data corresponding to the first code stream data.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes switching the intermediate state, the first code stream data is discarded.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the first code stream data is sent.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data It is the image data corresponding to the first code stream data.

Wherein, the new input image data includes the image data of the target resolution obtained by re-sampling or the image data of the target resolution directly output by the camera device.

Wherein, when the processor executes the computer program, the following steps are implemented: if the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.

Wherein, when the processor executes the computer program, the following steps are implemented: if the target resolution is less than the resolution of the first image data, the first image data is down-sampled to obtain the second image data of the target resolution.

Among them, the first encoder and the second encoder are the same encoder.

Among them, the target resolution matches the quality of the data transmission channel.

Wherein, when the processor executes the computer program, it implements the following steps: generating and sending switching information, the switching information includes a synchronous switching identifier and a target resolution.

Wherein, the switching resolution state identifier includes the identifier of the state before the switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.

Wherein, the synchronization identifier includes the frame number statistical information identifier at the target resolution after the switching.

Wherein, when the processor executes the computer program, the following steps are implemented: the switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.

Wherein, when the processor executes the computer program, it implements the following steps: generating and sending the supplementary enhancement information SEI including the switching information.

Wherein, when the processor executes the computer program, it implements the following steps: generating and sending switching information according to the switching instruction.

Wherein, when the processor executes the computer program, the following steps are implemented: when it is detected that the quality of the data transmission channel does not meet the preset condition, a switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.

Wherein, when the processor executes the computer program, it implements the following steps: when detecting that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, it generates a first switching instruction, wherein the target resolution of the first switching instruction is greater than the first switching instruction. A resolution of image data; when it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.

The first preset channel quality requirement is the same as the second preset channel quality requirement. The preset channel quality requirement includes a preset channel quality threshold T and a preset duration of t seconds; when the processor executes the computer program, the following steps are implemented : When it is detected that the quality of the data transmission channel for t seconds is greater than or equal to the preset channel quality threshold T, then the first switching instruction is generated; when it is detected that the quality of the data transmission channel for t seconds is less than the preset channel quality threshold T, then the second switch is generated. Switch instructions.

Wherein, when the processor executes the computer program, the following steps are implemented: receiving an input user instruction, where the user instruction includes the target resolution; and generating a switching instruction according to the user instruction.

This application also provides a data transmission device, which is a data transmission device on the decoding side. It should be noted that the device in this embodiment can execute the steps in the data transmission method on the decoding side as described above. For detailed descriptions of related content, please refer to The above-mentioned data transmission method at the decoding end will not be repeated here.

The device includes: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and when the computer program is executed, the following steps are implemented:

Obtain the target resolution; Obtain the third image data reconstructed by the first decoder; Obtain the second code stream data encoded by the third encoder, the second code stream data corresponding to the target resolution; Obtain the second decoder pair The fourth image data after the code stream data is decoded.

Wherein, when the processor executes the computer program, it implements the following steps: acquiring fifth image data after re-sampling the third image data, where the fifth image data is image data corresponding to the second code stream data.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes the switching intermediate state, the fourth image data is discarded.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the first code stream data is received.

Wherein, when the processor executes the computer program, the following steps are implemented: if the current state includes the switching completed state, the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new input image data is new The code stream data is decoded, and the fifth image data is the image data corresponding to the second code stream data.

Wherein, when the processor executes the computer program, the following steps are implemented: if the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain the fifth image data of the target resolution.

Wherein, when the processor executes the computer program, the following steps are implemented: if the target resolution is less than the resolution of the third image data, the third image data is down-sampled to obtain the fifth image data of the target resolution.

Among them, the first decoder and the second decoder are the same decoder.

Wherein, when the processor executes the computer program, the following steps are implemented: receiving switching information, the switching information includes a synchronous switching identifier and a target resolution.

Wherein, when the processor executes the computer program, it implements the following steps: receiving the first code stream data including the switching information.

Wherein, when the processor executes the computer program, it implements the following steps: receiving the supplementary enhancement information SEI including the switching information.

The present application also provides a data transmission system, which includes a data transmission device at any encoding end and a data transmission device at any decoding end as described above. For a detailed description of the relevant content, please refer to the above-mentioned data transmission device on the encoding end and the data transmission device on the decoding end, which will not be repeated here.

The present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor realizes the data transmission method at any one of the encoding ends. For a detailed description of the relevant content, please refer to the data transmission method section of the above-mentioned encoding end, which is not repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of the data transmission device at the encoding end, such as a hard disk or memory of the data transmission device at the encoding end. The computer-readable storage medium may also be an external storage device of the data transmission device at the encoding end, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc., equipped on the data transmission device at the encoding end.

This application also provides another computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor realizes the data transmission method at any one of the decoding ends. For a detailed description of the relevant content, please refer to the data transmission method section of the above-mentioned decoder, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of any of the foregoing data transmission devices at the decoding end, such as a hard disk or memory of the data transmission device at the decoding end. The computer-readable storage medium may also be an external storage device of the data transmission device at the decoding end, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc., equipped on the data transmission device at the decoding end.

It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application.

It should also be understood that the term "and/or" used in the specification of this application and the appended claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations.

The above are only specific embodiments of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A data transmission method, characterized in that it comprises:

Obtain the target resolution;

Acquiring the first image data reconstructed by the first encoder;

Obtain the first code stream data encoded by the second encoder, where the first code stream data corresponds to the target resolution.
The method according to claim 1, wherein after the obtaining the first image data reconstructed by the first encoder, the method further comprises: obtaining second image data after re-sampling the first image data , The second image data is image data corresponding to the first code stream data.
The method according to claim 1, wherein the method further comprises:

If the current state includes the switching intermediate state, the first code stream data is discarded.
The method according to claim 1, wherein the method further comprises:

If the current state includes the switching completion state, the first code stream data is sent.
The method according to claim 1, wherein the method further comprises:

If the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data is corresponding to the first code stream data Image data.
The method according to claim 5, wherein the new input image data comprises image data of a target resolution obtained by re-sampling or image data of a target resolution directly output by a camera device.
The method according to claim 2, wherein said acquiring second image data after re-sampling said first image data comprises:

If the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.
The method according to claim 2, wherein said acquiring second image data after re-sampling said first image data comprises:

If the target resolution is less than the resolution of the first image data, then down-sampling the first image data to obtain second image data of the target resolution.
The method according to claim 1, wherein:

The first encoder and the second encoder are the same encoder.
The method according to claim 1, wherein:

The target resolution matches the quality of the data transmission channel.
The method according to claim 1, wherein the method further comprises:

Generate and send switching information, where the switching information includes a synchronous switching identifier and a target resolution.
The method according to claim 11, wherein the synchronization switching identifier includes a switching resolution state identifier and a synchronization identifier.
The method according to claim 12, wherein the switching resolution state identifier includes the identifier of the state before switching, the identifier of the switching intermediate state, and the identifier of the switching completed state.
The method according to claim 12, wherein the synchronization identifier comprises a frame number statistical information identifier at the target resolution after switching.
The method according to claim 11, wherein said generating and sending handover information comprises:

The switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.
The method according to claim 11, wherein said generating and sending handover information comprises:

Generate and send the supplementary enhancement information SEI including the handover information.
The method according to claim 11, wherein said generating and sending handover information comprises:

Generate and send switching information according to switching instructions.
The method according to claim 17, wherein the method further comprises:

When it is detected that the quality of the data transmission channel does not meet the preset condition, the switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.
The method according to claim 18, wherein the generating the switching instruction when it is detected that the quality of the data transmission channel does not meet a preset condition comprises:

When it is detected that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, a first switching instruction is generated, wherein the target resolution of the first switching instruction is greater than the resolution of the first image data;

When it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.
The method according to claim 19, wherein the first preset channel quality requirement is the same as the second preset channel quality requirement, and the preset channel quality requirement includes a preset channel quality threshold T and a preset duration. Time t seconds;

When it is detected that the quality of the data transmission channel is greater than or equal to a preset channel quality requirement, generating a first handover instruction includes:

When it is detected that the quality of the data transmission channel for t seconds is greater than or equal to the preset channel quality threshold T, a first switching instruction is generated;

When it is detected that the quality of the data transmission channel is less than a preset channel quality requirement, generating a second handover instruction includes:

When it is detected that the quality of the data transmission channel for t seconds is less than the preset channel quality threshold T, a second switching instruction is generated.
The method according to claim 17, wherein the method further comprises:

Receiving an input user instruction, where the user instruction includes a target resolution;

The switching instruction is generated according to the user instruction.
A data transmission method, characterized in that it comprises:

Obtain the target resolution;

Acquiring the third image data reconstructed by the first decoder;

Acquiring second code stream data encoded by the third encoder, where the second code stream data corresponds to the target resolution;

Obtain the fourth image data after the second decoder decodes the second code stream data.
The method according to claim 22, wherein after said obtaining the third image data reconstructed by the first decoder, the method further comprises:

Acquiring fifth image data after resampling the third image data, where the fifth image data is image data corresponding to the second code stream data.
The method according to claim 22, wherein the method further comprises:

If the current state includes the switching intermediate state, the fourth image data is discarded.
The method according to claim 22, wherein the method further comprises:

If the current state includes the switching completion state, the first code stream data is received.
The method according to claim 22, wherein the method further comprises:

If the current state includes the switching completion state, the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new code stream data of the new input image data is decoded. The fifth image data Is the image data corresponding to the second code stream data.
The method according to claim 23, wherein said acquiring fifth image data after re-sampling said third image data comprises:

If the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain fifth image data of the target resolution.
The method according to claim 23, wherein said acquiring fifth image data after re-sampling said third image data comprises:

If the target resolution is less than the resolution of the third image data, down-sampling the third image data to obtain fifth image data of the target resolution.
The method of claim 22, wherein:

The first decoder and the second decoder are the same decoder.
The method of claim 22, wherein:

The target resolution matches the quality of the data transmission channel.
The method according to claim 22, wherein before said obtaining the target resolution, the method further comprises:

Receive switching information, where the switching information includes a synchronous switching identifier and a target resolution.
The method according to claim 31, wherein the synchronization switching identifier includes a switching resolution state identifier and a synchronization identifier.
The method according to claim 32, wherein the switching resolution state identifier includes the identifier of the state before switching, the identifier of the switching intermediate state, and the identifier of the switching completed state.
The method according to claim 32, wherein the synchronization identifier comprises a frame number statistical information identifier at the target resolution after switching.
The method according to claim 31, wherein said receiving handover information comprises:

Receiving the first code stream data including the switching information.
The method according to claim 31, wherein said receiving handover information comprises:

Receiving the supplementary enhancement information SEI including the handover information.
A data transmission device, characterized in that, the device includes: a memory and a processor;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Obtain the target resolution;

Acquiring the first image data reconstructed by the first encoder;

Obtain the first code stream data encoded by the second encoder, where the first code stream data corresponds to the target resolution.
The device according to claim 37, wherein the processor implements the following steps when executing the computer program:

Acquiring second image data after resampling the first image data, where the second image data is image data corresponding to the first bitstream data.
The device according to claim 37, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching intermediate state, the first code stream data is discarded.
The device according to claim 37, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching completion state, the first code stream data is sent.
The device according to claim 37, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data is corresponding to the first code stream data Image data.
The device according to claim 41, wherein the new input image data comprises image data of a target resolution obtained by re-sampling or image data of a target resolution directly output by a camera device.
The device according to claim 38, wherein the processor implements the following steps when executing the computer program:

If the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.
The device according to claim 38, wherein the processor implements the following steps when executing the computer program:

If the target resolution is less than the resolution of the first image data, then down-sampling the first image data to obtain second image data of the target resolution.
The device according to claim 37, wherein the first encoder and the second encoder are the same encoder.
The device according to claim 37, wherein the target resolution matches the quality of the data transmission channel.
The device according to claim 37, wherein the processor implements the following steps when executing the computer program:

Generate and send switching information, where the switching information includes a synchronous switching identifier and a target resolution.
The apparatus according to claim 47, wherein the synchronization switching identifier comprises a switching resolution state identifier and a synchronization identifier.
The device according to claim 48, wherein the switching resolution state identifier includes an identifier of a state before switching, an identifier of an intermediate state of switching, and an identifier of a switching completed state.
The device according to claim 48, wherein the synchronization identifier comprises a frame number statistical information identifier at the target resolution after switching.
The device according to claim 47, wherein the processor implements the following steps when executing the computer program:

The switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.
The device according to claim 47, wherein the processor implements the following steps when executing the computer program:

Generate and send the supplementary enhancement information SEI including the handover information.
The device according to claim 47, wherein the processor implements the following steps when executing the computer program:

Generate and send switching information according to switching instructions.
The device according to claim 53, wherein the processor implements the following steps when executing the computer program:

When it is detected that the quality of the data transmission channel does not meet the preset condition, the switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.
The device according to claim 54, wherein the processor implements the following steps when executing the computer program:

When it is detected that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, generating a first switching instruction, wherein the target resolution of the first switching instruction is greater than the resolution of the first image data;

When it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.
The apparatus according to claim 55, wherein the first preset channel quality requirement is the same as the second preset channel quality requirement, and the preset channel quality requirement includes a preset channel quality threshold T and a preset duration. Time t seconds;

When the processor executes the computer program, the following steps are implemented:

When it is detected that the quality of the data transmission channel for t seconds is greater than or equal to the preset channel quality threshold T, a first switching instruction is generated;

When it is detected that the quality of the data transmission channel for t seconds is less than the preset channel quality threshold T, a second switching instruction is generated.
The device according to claim 53, wherein the processor implements the following steps when executing the computer program:

Receiving an input user instruction, where the user instruction includes a target resolution;

The switching instruction is generated according to the user instruction.
A data transmission device, characterized in that the device includes: a memory and a processor;

The memory is used to store a computer program;

The processor is used to execute the computer program, and when executing the computer program, implement the following steps:

Obtain the target resolution;

Acquiring the third image data reconstructed by the first decoder;

Acquiring second code stream data encoded by the third encoder, where the second code stream data corresponds to the target resolution;

Obtain the fourth image data after the second decoder decodes the second code stream data.
The device according to claim 58, wherein the processor implements the following steps when executing the computer program:

Acquiring fifth image data after resampling the third image data, where the fifth image data is image data corresponding to the second code stream data.
The device according to claim 58, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching intermediate state, the fourth image data is discarded.
The device according to claim 58, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching completion state, the first code stream data is received.
The device according to claim 58, wherein the processor implements the following steps when executing the computer program:

If the current state includes the switching completion state, the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new code stream data of the new input image data is decoded. The fifth image data Is the image data corresponding to the second code stream data.
The device according to claim 59, wherein the processor implements the following steps when executing the computer program:

If the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain fifth image data of the target resolution.
The device according to claim 59, wherein the processor implements the following steps when executing the computer program:

If the target resolution is less than the resolution of the third image data, down-sampling the third image data to obtain fifth image data of the target resolution.
The device of claim 58, wherein the first decoder and the second decoder are the same decoder.
The device according to claim 58, wherein the target resolution matches the quality of the data transmission channel.
The device according to claim 58, wherein the processor implements the following steps when executing the computer program:

Receive switching information, where the switching information includes a synchronous switching identifier and a target resolution.
The device according to claim 67, wherein the synchronization switching identifier comprises a switching resolution state identifier and a synchronization identifier.
The device according to claim 68, wherein the switching resolution state identifier includes the identifier of the state before switching, the identifier of the switching intermediate state, and the identifier of the switching completed state.
The device according to claim 68, wherein the synchronization identifier comprises a frame number statistical information identifier at the target resolution after switching.
The device according to claim 67, wherein the processor implements the following steps when executing the computer program:

Receiving the first code stream data including the switching information.
The device according to claim 67, wherein the processor implements the following steps when executing the computer program:

Receiving the supplementary enhancement information SEI including the handover information.
A data transmission system, characterized in that the system comprises the data transmission device according to any one of claims 37-57 and the data transmission device according to any one of claims 58-72.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes any one of claims 1-21 Data transfer method.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes any one of claims 22-36 Data transfer method.