WO2021102992A1 - Procédé, dispositif et système de transmission de données, et support d'enregistrement - Google Patents

Procédé, dispositif et système de transmission de données, et support d'enregistrement Download PDF

Info

Publication number
WO2021102992A1
WO2021102992A1 PCT/CN2019/122168 CN2019122168W WO2021102992A1 WO 2021102992 A1 WO2021102992 A1 WO 2021102992A1 CN 2019122168 W CN2019122168 W CN 2019122168W WO 2021102992 A1 WO2021102992 A1 WO 2021102992A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
switching
resolution
data
target resolution
Prior art date
Application number
PCT/CN2019/122168
Other languages
English (en)
Chinese (zh)
Inventor
周士贞
苏文艺
赵亮
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980061264.9A priority Critical patent/CN112771830A/zh
Priority to PCT/CN2019/122168 priority patent/WO2021102992A1/fr
Publication of WO2021102992A1 publication Critical patent/WO2021102992A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems

Definitions

  • This application relates to the field of communication technology, and in particular to a data transmission method, device, system and storage medium.
  • I frames In a video coding sequence, there are mainly three types of coding frames: I frames, P frames, and B frames.
  • the I frame does not refer to other image frames, so usually when the code stream state is switched, such as switching the resolution, an I frame is sent first to ensure that the referenced video frame is correct and valid when the subsequent video frame is decoded.
  • the wireless channel is a complex, changeable and unstable communication condition. After the channel quality deteriorates, on the one hand, the bit stream data sent by the encoding end cannot be sent to the decoding end in a timely manner, and error packets may occur. On the other hand, a reduced-resolution transmission strategy can be adopted.
  • this application provides a data transmission method, a data transmission device, a data transmission system, and a storage medium.
  • this application provides a data transmission method, including:
  • this application provides a data transmission method, including:
  • the present application provides a data transmission device, the device includes: a memory and a processor;
  • the memory is used to store a computer program
  • the processor is used to execute the computer program and when executing the computer program, implement the following steps:
  • the present application provides a data transmission device, the device includes: a memory and a processor;
  • the memory is used to store a computer program
  • the processor is used to execute the computer program and when executing the computer program, implement the following steps:
  • the present application provides a data transmission system, which includes the data transmission device described in the third aspect and the data transmission device described in the fourth aspect.
  • the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes the data described in the first aspect. Transmission method.
  • the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes the data described in the second aspect above. Transmission method.
  • the embodiments of the present application provide a data transmission method, device, system and storage medium.
  • the data transmission sending end (encoding end): obtains the target resolution; obtains the first image data reconstructed by the first encoder; obtains the second code
  • the first code stream data corresponding to the target resolution after encoding by the encoder; since the encoding end first obtains the reconstructed first image data after obtaining the target resolution, it is equivalent to the image data decoded by the analog decoding end, the first image data Matching with the decoded image data at the decoding end, the first image data can be used as the basic image data of the reference image data of the subsequent image data.
  • Data transmission receiving end (decoding end): Obtain the target resolution; Obtain the third image data reconstructed by the first decoder; Obtain the second code stream data corresponding to the target resolution encoded by the third encoder; Obtain the second decoding The fourth image data after the decoder decodes the second code stream data; since the decoder first obtains the reconstructed third image data after obtaining the target resolution, the third image data can be used as reference image data for subsequent image data In this way, it can provide correct and effective reference image data support for subsequent image data in a timely manner, in particular, it can provide correct and effective reference image data for decoding the subsequent image data after the resolution is changed in time.
  • the decoding end can also obtain correct and effective reference image data in time, thereby ensuring that the decoded Image quality provides support; since the decoding end simulates the encoding process of the encoding end corresponding to the target resolution and performs decoding, in this way, technical support can be provided to ensure the seamless connection between the decoding end and the encoding end. Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data are matched. In this way, the encoding end can only send the Provide technical support for error-corrected frames at the decoding end.
  • FIG. 1 is a schematic flowchart of an embodiment of a data transmission method according to the present application
  • FIG. 2 is a schematic flowchart of another embodiment of the data transmission method of the present application.
  • FIG. 3 is a schematic flowchart of another embodiment of the data transmission method of the present application.
  • FIG. 5 is a schematic flowchart of another embodiment of the data transmission method of the present application.
  • Fig. 6 is a flow diagram of the sending end and the receiving end of the first application of the data transmission method of this application before the resolution is switched;
  • FIG. 7 is a flowchart of the sending end and the receiving end during the switching process of the switching resolution of the application of FIG. 6;
  • Fig. 8 is a flow chart of the sending end and the receiving end of the application of Fig. 6 after the resolution switching is completed;
  • I frames In a video coding sequence, there are mainly three types of coding frames: I frames, P frames, and B frames.
  • the I frame does not refer to other image frames.
  • an I frame is sent first to ensure that the reference video frame is correct and effective when the subsequent video frame is decoded.
  • the wireless channel In the field of real-time video wireless transmission, the wireless channel is a complex, changeable and unstable communication condition. After the channel quality deteriorates, the correct and effective reference video frames of the encoder and decoder cannot be guaranteed in time.
  • the data transmission sending end (encoding end) of the embodiment of the present application obtains the target resolution; obtains the first image data reconstructed by the first encoder; obtains the first code stream data corresponding to the target resolution encoded by the second encoder; Since the encoding end first obtains the reconstructed first image data after acquiring the target resolution, which is equivalent to the image data decoded by the analog decoding end.
  • the first image data matches the decoded image data of the decoding end, and the first image data can be As the basic image data of the reference image data of the subsequent image data, in this way, it can provide correct and effective reference image data for the subsequent image data in time, especially the correct and effective image data for the subsequent image data after the resolution is changed.
  • Data transmission receiving end (decoding end): Obtain the target resolution; Obtain the third image data reconstructed by the first decoder; Obtain the second code stream data corresponding to the target resolution encoded by the third encoder; Obtain the second decoding The fourth image data after the decoder decodes the second code stream data; since the decoder first obtains the reconstructed third image data after obtaining the target resolution, the third image data can be used as reference image data for subsequent image data In this way, it can provide correct and effective reference image data support for subsequent image data in a timely manner, in particular, it can provide correct and effective reference image data support for the subsequent decoding of image data after the resolution is changed in time, In the case of poor channel quality, the encoder does not need to retransmit the complete, large-rate reference data stream data, and the decoder can also obtain correct and effective reference image data in time, thereby ensuring the decoded image Quality support; since the decoding end
  • the decoding end Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data match, and the decoding end simulates the encoding process of the encoding end and performs decoding. In this way, It can provide technical support for the encoding end to only generate error correction frames to the decoding end when the subsequent error correction is required.
  • FIG. 1 is a schematic flowchart of an embodiment of a data transmission method according to the present application. It should be noted that the method in the embodiment of the present application is a data transmission method at the encoding end.
  • the data transmission method at the encoding end includes: step S101, step S102, and step S103.
  • Step S101 Obtain a target resolution.
  • Step S102 Obtain the first image data reconstructed by the first encoder.
  • Step S103 Obtain the first code stream data encoded by the second encoder, where the first code stream data corresponds to the target resolution.
  • the target resolution refers to the resolution of image data that currently needs to be encoded.
  • the target resolution matches the quality of the data transmission channel.
  • the quality of the data transmission channel can be current or pre-calculated. Specifically, when the quality of the data transmission channel is good, the target resolution can be increased to transmit as large an image as possible, and the size of the decoding end image can be increased as much as possible on the basis of ensuring the image quality; when the quality of the data transmission channel is poor At this time, the target resolution can be reduced to achieve the encoding reduction of the output code stream, reduce the error packet loss as much as possible, and ensure that the image quality of the decoding end is basically acceptable.
  • a list of correspondences between the quality level of the data transmission channel and the matched target resolution can be preset, and the matched target resolution can be determined according to the currently detected quality level of the data transmission channel.
  • the quality of the data transmission channel can be detected by the encoding end, so that timely feedback on changes in channel quality can be obtained.
  • the first image data refers to the reconstructed image data obtained from the first encoder according to the coding principle.
  • the first code stream data can be obtained after encoding the image data corresponding to the target resolution in the second encoder.
  • the resolution of the first image data may be the same as the target resolution, for example, before the resolution switching; the resolution of the first image data may also be different from the target resolution, for example, during the resolution switching process and the switching It can be different after completion.
  • the first encoder and the second encoder can be two independent hardware encoders, or multiplex the same hardware encoder, that is, use the same hardware encoder for time-sharing encoding: as the first
  • the encoder performs encoding, and performs encoding as a second encoder in some other periods.
  • the first code stream data can be obtained after encoding the image data corresponding to the target resolution.
  • the target resolution in the embodiment of this application, before encoding, it also obtains the first image data reconstructed by the first encoder, which is equivalent to simulating the image data decoded by the decoder.
  • the first image data can be used as the basic image data of the reference image data of the subsequent image data.
  • the reference image data of the encoding end is the same as the reference image data of the decoding end after the resolution is switched, so that it can be timely Provide correct and effective reference image data for subsequent image data, in particular, provide support for correct and effective reference image data for encoding of image data after subsequent resolution changes in time, so as to ensure correct decoding at the decoder.
  • the method may further include: S104.
  • Step S104 Obtain second image data after resampling the first image data, where the second image data is image data corresponding to the first code stream data.
  • the resolution of the second image data is the target resolution, which is image data obtained by re-sampling the first image data (that is, obtained by simulation).
  • the re-sampling module on the encoding end and the decoding end it is possible to ensure that the images output by the two ends in the resolution switching match.
  • the degree of resampling at the encoding end and the decoding end can be the same to ensure the consistency of image transmission during the resolution switching process.
  • step S103 is to encode the second image data by the second encoder to obtain the first code stream data.
  • step S104 may have the following two situations:
  • the first type if the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.
  • Downsampling also known as reduced image or downsampling (downsampled)
  • the redundancy can be removed, and the occupied bandwidth can be reduced when the data is transmitted after encoding.
  • the redundancy can be removed by down-sampling, and the packet loss and error packet can be minimized when the data is transmitted after encoding. , As far as possible to ensure the image quality of the decoding end.
  • Down-sampling principle For an image I with a size of M*N, perform s times downsampling to obtain a resolution image of (M/s)*(N/s) size. Of course, s should be M and N The common divisor works. If you consider the image in matrix form, the image in the s*s window of the original image is turned into a pixel, and the value of this pixel is the mean value of all pixels in the window.
  • the second type if the target resolution is less than the resolution of the first image data, down-sampling the first image data to obtain the second image data of the target resolution.
  • Image magnification almost always uses interpolation methods, that is, on the basis of the original image pixels, a suitable interpolation algorithm is used to insert new elements between pixels.
  • step S103 whether the first code stream data obtained by encoding in step S103 is sent depends on the current state of the encoding end:
  • Switching the intermediate state means that the resolution switching has not been officially completed during the switching process of the resolution. Therefore, the first bit stream data at this time does not need to be sent, and the first bit stream data is discarded, which can avoid the first bit stream data. Take up storage space.
  • the switching intermediate state of the encoding end includes the state of maintaining encoding with different resolutions.
  • the switching completion state means that the resolution switching is officially completed. At this time, the first bit stream data is sent to the decoding end. On the one hand, it can ensure that the resolution switching is completed, and on the other hand, it can ensure the synchronous switching with the decoding end.
  • the switching completion state of the encoding end includes the state of maintaining the resolution encoding after switching.
  • the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data corresponds to the first bitstream data.
  • Image data if the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data corresponds to the first bitstream data.
  • the second image data is the image data corresponding to the target resolution, and is the image data corresponding to the first bitstream data; wherein, the second image data may also be the image data after the first image data is resampled.
  • the resolution of the input image data is the target resolution.
  • the second image data can be used as the new input image
  • the reference image data of the data encodes the new input image data.
  • the new input image data includes the image data of the target resolution obtained by re-sampling or the image data of the target resolution directly output by the camera device.
  • the image source input by the encoding end can have two schemes: one is that the output image of the camera device keeps the resolution A unchanged, and the image of resolution A is resampled to obtain the image of resolution B; the other is Switching the camera device, the input image is changed from the original resolution A image to the resolution B image; the purpose of these two solutions is to obtain an image of the new resolution B, but the implementation is different.
  • the method further includes: step S105.
  • Step S105 Generate and send switching information, where the switching information includes the synchronization switching identifier and the target resolution.
  • the synchronization switching identifier includes the switching resolution status identifier and the synchronization identifier.
  • the switching resolution state identifier includes the identifier of the state before switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.
  • the synchronization identifier includes a frame number statistical information identifier at the target resolution after switching.
  • both ends will also send a synchronization switching flag in one or two directions to ensure that the output of the re-sampling module at the decoding end is consistent with the re-sampling output at the encoding end.
  • the easier ways to implement the handover information include: the handover information can be included in the communication protocol or supplemented in the Supplemental Enhancement Information (SEI), that is, the method for sending the handover information in step S105 can be easily implemented in the following two ways: Ways:
  • SEI Supplemental Enhancement Information
  • the switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.
  • step S105 is triggered to execute a switching instruction, that is, switching information is generated and sent according to the switching instruction.
  • the generation of the switching instruction may be automatically generated by the encoding end (for example, when the quality of the data transmission channel is detected to be degraded), or may be generated according to a user instruction.
  • the specific description is as follows:
  • the method further includes: step S106.
  • Step S106 When it is detected that the quality of the data transmission channel does not meet the preset condition, a switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.
  • Sub-step S1061 Detect and determine the magnitude of the data transmission channel quality, the first preset channel quality requirement, and the second preset channel quality requirement.
  • Sub-step S1062 When it is detected that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, a first switching instruction is generated, wherein the target resolution of the first switching instruction is greater than the resolution of the first image data.
  • Sub-step S1063 When it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.
  • the first preset channel quality requirement is the same as the second preset channel quality requirement, and both are preset channel quality requirements.
  • the preset channel quality requirement includes a preset channel quality threshold T and a preset duration of t seconds.
  • the sub-step S1062 may specifically include: when it is detected that the quality of the data transmission channel is greater than or equal to the preset channel quality threshold T for t seconds, generating a first handover instruction; the sub-step S1063 may specifically include: when the quality of the data transmission channel is detected If the duration of t seconds is less than the preset channel quality threshold T, a second switching instruction is generated.
  • the first preset channel quality requirement is different from the second preset channel quality requirement, it can also be detected that the data transmission channel quality is less than the first preset channel quality requirement and greater than or equal to the second preset channel quality requirement , The resolution remains unchanged.
  • the method further includes: receiving an input user instruction, where the user instruction includes the target resolution; and generating a switching instruction according to the user instruction.
  • a suitable target resolution can be matched according to the actual needs of the user, and the user experience can be further improved.
  • FIG 3 is a schematic flow diagram of another embodiment of the data transmission method of the present application.
  • the method of the embodiment of the present application is a data transmission method of the decoding end.
  • the content associated with the encoding end please refer to the above-mentioned encoding end.
  • the part of the data transmission method will not be repeated here. The following specifically introduces the content that is not related to or different from the encoding end.
  • the data transmission method at the decoding end includes: step S201, step S202, step S203, and step S204.
  • Step S201 Obtain the target resolution.
  • Step S202 Obtain the third image data reconstructed by the first decoder.
  • Step S203 Obtain the second code stream data encoded by the third encoder, where the second code stream data corresponds to the target resolution.
  • Step S204 Obtain fourth image data after the second decoder decodes the second code stream data.
  • the first decoder and the second decoder can be two independent hardware decoders, or the same hardware decoder can be multiplexed, that is, the same hardware decoder is used for time-sharing decoding: in certain periods as the first The decoder performs decoding, and performs decoding as a second decoder in some other periods.
  • the decoder can decode the code stream data corresponding to the target resolution.
  • the third image data reconstructed by the first decoder is obtained, and the second code stream data corresponding to the target resolution encoded by the third encoder is obtained, which is equivalent to analog encoding at the encoding end.
  • the latter bitstream data is then decoded. Since the third image data can be used as the basic image data of the reference image data when decoding the bitstream data of the subsequent image data, in this way, it can provide correct and effective follow-up image data in time. In particular, it can provide correct and effective reference image data to provide support in time for the subsequent decoding of the image data after the resolution has been changed.
  • the encoder does not need to retransmit the complete, large-rate reference data stream data, and the decoder can also obtain correct and effective reference image data in time, thereby providing support for correct decoding , Provide support to ensure the image quality after decoding.
  • the decoding end simulates the encoding process of the encoding end corresponding to the target resolution and performs decoding, in this way, technical support can be provided to ensure seamless connection between the decoding end and the encoding end. Since the encoding end and the decoding end both obtain the reconstructed first image data and third image data, the first image data and the third image data match, and the decoding end simulates the encoding process of the encoding end and performs decoding. In this way, It can provide technical support for the encoding end to only generate error correction frames to the decoding end when the subsequent error correction is required.
  • the method when switching resolutions, in order to facilitate the synchronization of the encoding end and the decoding end, maintain a switching process of switching resolutions before the formal switching (that is, the intermediate state of switching), that is, through The image data corresponding to the target resolution is obtained by simulation, so as to maintain a switching process of switching resolution at the decoding end, that is, in step S202, after obtaining the third image data reconstructed by the first decoder, the method further includes: step S205 .
  • Step S205 Obtain fifth image data after resampling the third image data, where the fifth image data is image data corresponding to the second code stream data.
  • step S205 may have the following two situations:
  • the first type if the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain the fifth image data of the target resolution.
  • the second type if the target resolution is less than the resolution of the third image data, down-sampling the third image data to obtain the fifth image data of the target resolution.
  • the switching intermediate state means that the resolution switching has not been officially completed during the switching of the resolution. Therefore, the fourth image data does not need to be retained at this time, and the fourth image data is discarded, which can prevent the fourth image data from occupying storage space .
  • the switching intermediate state of the decoder includes maintaining the state of decoding with different resolutions.
  • the switching completion state means that the resolution switching is officially completed. At this time, the first bit stream data from the encoding end can be received. On the one hand, it can ensure the completion of the resolution switching, and on the other hand, it can ensure the synchronous switching with the encoding end.
  • the switching completion state of the decoding end includes the state of maintaining the resolution decoding after switching.
  • the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new code stream data of the new input image data is decoded.
  • Five image data is the image data corresponding to the second code stream data.
  • the fifth image data is the image data corresponding to the target resolution, and is the image data corresponding to the second bit stream data; wherein, the fifth image data may also be the image data after re-sampling the third image data.
  • the resolution of the input image data is the target resolution.
  • the fifth image data can be used as the new The reference image data of the input image data decodes other video frames other than the I frame corresponding to the new input image data.
  • the decoding end before acquiring the target resolution, the decoding end further includes: step S206.
  • Step S206 Receive switching information, where the switching information includes the synchronization switching identifier and the target resolution.
  • the synchronization switching identifier includes the switching resolution status identifier and the synchronization identifier.
  • the switching resolution state identifier includes the identifier of the state before switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.
  • the synchronization identifier includes a frame number statistical information identifier at the target resolution after switching.
  • both ends will also send a synchronization switching flag in one or two directions to ensure that the output of the re-sampling module at the decoding end is consistent with the re-sampling output at the encoding end.
  • the easier ways to implement the switching information include: the switching information can be included in the communication protocol (together with the code stream data) or supplemented in the supplementary enhancement information SEI, that is, the switching information receiving method in step S206 can be easily implemented in the following ways Two ways:
  • One is: receiving the first code stream data including switching information.
  • the first stream data comes from the encoding end.
  • the other is: receiving supplementary enhancement information SEI including handover information.
  • Figure 6 is a flow diagram of the sending and receiving ends of the application of the data transmission method 1 of this application before switching the resolution.
  • Figure 7 is the sending and receiving ends of the application of Figure 6 during the switching of the resolution.
  • Fig. 8 is a flowchart of the sending end and the receiving end of the application of Fig. 6 after the resolution switching is completed.
  • the resolution switch When the channel quality is detected to be worse and is lower than the channel quality threshold T for t seconds, the resolution switch is started, and the input image is downsampled Encode and send in a size of 1280x720 to relieve transmission pressure. Similarly, when it is detected that the communication conditions are better, it can be up-sampled from a lower resolution to a higher resolution for encoding and sending. It is worth mentioning that the degree of resampling can be flexibly configured, but the encoding end and the decoding end must be consistent.
  • the switching process is mainly composed of two parts: the encoding end switching intermediate state maintenance and the decoding end switching intermediate state respectively maintain.
  • the function of the intermediate state maintenance part of the encoding end switching is to simulate the decoded image at the decoding end and use it as a reference frame to encode the image after the size change. In this way, it can smoothly enter the encoding transmission state after the resolution is switched.
  • the function of the switch intermediate state maintenance part of the decoding end is to simulate receiving and decoding the bit stream data after the resolution is switched.
  • the biggest advantage of this scheme is that the encoding end and the decoding end can be seamlessly connected, and it can avoid sending large-rate I frames when switching resolutions; this resolution switching scheme is channel-friendly, has short transmission delay, and effectively avoids traditional The screen resolution, jitter, flicker and other issues caused by cutting resolution.
  • the traditional cut resolution scheme usually uses I frame as the start of the new resolution. In order to achieve better subjective quality, a larger volume of I frame is usually required.
  • the code rate of I frame is positively correlated with the image quality after decoding. The higher the bit rate, the greater the transmission pressure, especially in the case of poor channel quality, the transmission delay is very large, so traditional cutting resolution usually brings bad subjective such as blurring, jitter, and flicker. Experience.
  • first encode the input image according to the pre-switching procedure As shown in Figure 7, the image data of the resolution PA is input to the first encoder, and the code stream data SA output by the first encoder is sent.
  • the reconstructed image data (that is, the first image data with a resolution of PA) is taken from the first encoder and input to the resampling module, and the matching resolution is selected according to the level of channel quality (That is, the target resolution), and update the communication protocol.
  • the communication protocol includes the resolution status, the frame number statistics information under the new resolution, and the resolution size information after switching (that is, the target resolution).
  • the reconstructed image data (that is, the first image data) is down-sampling and outputs the image data of the resolution PB' (that is, the second image data), and the image data of the resolution PB' is input to the second encoder for processing Encoding, the coded stream data SB' (that is, the first stream data) is temporarily discarded.
  • This part of the solution realizes the state of maintaining the encoding of different resolutions (PA and PB').
  • the first decoder decodes the received code stream data SA to obtain the image data of the resolution PA, and at the same time obtains the resolution switching state from the state of the communication protocol at both ends of the codec.
  • the code stream data (ie, the second code stream data) encoded by the third encoder is input to the second decoder, and the image data of the resolution PB' (ie, the fourth image data) decoded by the second decoder is temporarily Discarded, this part of the solution realizes the state of maintaining the decoding of different resolutions (PA and PB').
  • the second encoder on the encoding end corresponds to the second encoder on the encoding end in Figure 7 (switching intermediate state), and the encoding end encodes the code stream data SB (ie The first bit stream data) is sent, and the input image data of resolution PB after switching corresponds to the image data of resolution PB' input in the switching process as shown in Figure 7.
  • the resolution of the two is the same, but the image data of resolution PB It is the newly input video frame, and the image data of the resolution PB' is obtained by re-sampling the reconstructed video frame after encoding the original video frame.
  • the second decoder at the decoding end corresponds to the second decoder at the decoding end in Figure 7 (switching intermediate state), and the decoding end receives the code stream data SB (that is, the first code stream data). ) Is directly input to the second decoder for decoding and output resolution PB image data.
  • the received bit stream data SB corresponds to the bit stream data encoded and output by the third encoder at the decoding end.
  • the resolution of the two is the same, but SB is wireless
  • the received video code stream of the new resolution, and the code stream data encoded and output by the third encoder at the decoding end is the video code stream of the image after re-sampling the original resolution image.
  • the consistency of the reference frame sequence of the encoding end and the decoding end can be ensured, so when the resolution is switched, the encoding end does not need to re-encode the I frame and send it to the decoding end.
  • the encoder input image source has two schemes: one is to switch the camera (ie camera device), the input image resolution is changed from PA to PB; the other is to keep the PA of the camera output image unchanged, and resample the image data of the resolution PA Obtain image data with a resolution of PB.
  • the purpose of the two is to obtain the image data of the new resolution PB, but the realization method is different.
  • the switching information can be included in the communication protocol part and sent as bit stream data, or it can be supplemented with the SEI information part sent in the H264 coding standard.
  • Figure 9 is a schematic structural diagram of an embodiment of the data transmission device of the present application.
  • the data transmission device of this embodiment can execute the data transmission method on the encoding side.
  • the relevant content please refer to the above data transmission method on the encoding side. I won't repeat it again.
  • the device 10 includes: a memory 11 and a processor 12; the memory 11 and the processor 12 are connected by a bus 13.
  • the processor 12 may be a micro control unit, a central processing unit, or a digital signal processor, and so on.
  • the memory 11 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk, or a mobile hard disk, etc.
  • the memory 11 is used to store a computer program; the processor 12 is used to execute the computer program and when the computer program is executed, the following steps are implemented:
  • the processor when the processor executes the computer program, it implements the following steps: acquiring second image data after re-sampling the first image data, where the second image data is image data corresponding to the first code stream data.
  • the processor executes the computer program, the following steps are implemented: if the current state includes switching the intermediate state, the first code stream data is discarded.
  • the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the first code stream data is sent.
  • the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the second image data is used as the reference image data of the new input image data, and the new input image data is encoded, and the second image data It is the image data corresponding to the first code stream data.
  • the new input image data includes the image data of the target resolution obtained by re-sampling or the image data of the target resolution directly output by the camera device.
  • the processor executes the computer program, the following steps are implemented: if the target resolution is greater than the resolution of the first image data, the first image data is up-sampled to obtain the second image data of the target resolution.
  • the processor executes the computer program, the following steps are implemented: if the target resolution is less than the resolution of the first image data, the first image data is down-sampled to obtain the second image data of the target resolution.
  • the first encoder and the second encoder are the same encoder.
  • the target resolution matches the quality of the data transmission channel.
  • the processor when the processor executes the computer program, it implements the following steps: generating and sending switching information, the switching information includes a synchronous switching identifier and a target resolution.
  • the synchronization switching identifier includes the switching resolution status identifier and the synchronization identifier.
  • the switching resolution state identifier includes the identifier of the state before the switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.
  • the switching information is packaged in the code stream header in the form of a communication protocol, and the first code stream data including the switching information is generated and sent.
  • the processor when the processor executes the computer program, it implements the following steps: generating and sending the supplementary enhancement information SEI including the switching information.
  • the processor when the processor executes the computer program, it implements the following steps: generating and sending switching information according to the switching instruction.
  • the processor executes the computer program, the following steps are implemented: when it is detected that the quality of the data transmission channel does not meet the preset condition, a switching instruction is generated, and the switching instruction includes a target resolution matching the quality of the data transmission channel.
  • the processor when the processor executes the computer program, it implements the following steps: when detecting that the quality of the data transmission channel is greater than or equal to the first preset channel quality requirement, it generates a first switching instruction, wherein the target resolution of the first switching instruction is greater than the first switching instruction. A resolution of image data; when it is detected that the channel quality is less than the second preset channel quality requirement, a second switching instruction is generated, wherein the target resolution of the second switching instruction is less than the resolution of the first image data.
  • the first preset channel quality requirement is the same as the second preset channel quality requirement.
  • the preset channel quality requirement includes a preset channel quality threshold T and a preset duration of t seconds; when the processor executes the computer program, the following steps are implemented : When it is detected that the quality of the data transmission channel for t seconds is greater than or equal to the preset channel quality threshold T, then the first switching instruction is generated; when it is detected that the quality of the data transmission channel for t seconds is less than the preset channel quality threshold T, then the second switch is generated. Switch instructions.
  • the processor executes the computer program, the following steps are implemented: receiving an input user instruction, where the user instruction includes the target resolution; and generating a switching instruction according to the user instruction.
  • This application also provides a data transmission device, which is a data transmission device on the decoding side. It should be noted that the device in this embodiment can execute the steps in the data transmission method on the decoding side as described above. For detailed descriptions of related content, please refer to The above-mentioned data transmission method at the decoding end will not be repeated here.
  • the device includes: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and when the computer program is executed, the following steps are implemented:
  • the processor when the processor executes the computer program, it implements the following steps: acquiring fifth image data after re-sampling the third image data, where the fifth image data is image data corresponding to the second code stream data.
  • the processor executes the computer program, the following steps are implemented: if the current state includes the switching intermediate state, the fourth image data is discarded.
  • the processor executes the computer program, the following steps are implemented: if the current state includes the switching completion state, the first code stream data is received.
  • the processor executes the computer program, the following steps are implemented: if the current state includes the switching completed state, the fifth image data is used as the reference image data of the new code stream data of the new input image data, and the new input image data is new The code stream data is decoded, and the fifth image data is the image data corresponding to the second code stream data.
  • the processor executes the computer program, the following steps are implemented: if the target resolution is greater than the resolution of the third image data, the third image data is up-sampled to obtain the fifth image data of the target resolution.
  • the processor executes the computer program, the following steps are implemented: if the target resolution is less than the resolution of the third image data, the third image data is down-sampled to obtain the fifth image data of the target resolution.
  • the first decoder and the second decoder are the same decoder.
  • the target resolution matches the quality of the data transmission channel.
  • the processor executes the computer program, the following steps are implemented: receiving switching information, the switching information includes a synchronous switching identifier and a target resolution.
  • the synchronization switching identifier includes the switching resolution status identifier and the synchronization identifier.
  • the switching resolution state identifier includes the identifier of the state before the switching, the identifier of the intermediate state of the switching, and the identifier of the switching completed state.
  • the synchronization identifier includes the frame number statistical information identifier at the target resolution after the switching.
  • the processor when the processor executes the computer program, it implements the following steps: receiving the first code stream data including the switching information.
  • the processor when the processor executes the computer program, it implements the following steps: receiving the supplementary enhancement information SEI including the switching information.
  • the present application also provides a data transmission system, which includes a data transmission device at any encoding end and a data transmission device at any decoding end as described above.
  • a data transmission system which includes a data transmission device at any encoding end and a data transmission device at any decoding end as described above.
  • the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor realizes the data transmission method at any one of the encoding ends.
  • a computer-readable storage medium which stores a computer program, and when the computer program is executed by a processor, the processor realizes the data transmission method at any one of the encoding ends.
  • the computer-readable storage medium may be an internal storage unit of the data transmission device at the encoding end, such as a hard disk or memory of the data transmission device at the encoding end.
  • the computer-readable storage medium may also be an external storage device of the data transmission device at the encoding end, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc., equipped on the data transmission device at the encoding end.
  • the computer-readable storage medium stores a computer program.
  • the processor realizes the data transmission method at any one of the decoding ends.
  • the relevant content please refer to the data transmission method section of the above-mentioned decoder, which will not be repeated here.
  • the computer-readable storage medium may be an internal storage unit of any of the foregoing data transmission devices at the decoding end, such as a hard disk or memory of the data transmission device at the decoding end.
  • the computer-readable storage medium may also be an external storage device of the data transmission device at the decoding end, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc., equipped on the data transmission device at the decoding end.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne un procédé, un dispositif et un système de transmission de données, ainsi qu'un support d'enregistrement. Le procédé consiste à : obtenir une résolution cible (S101) ; obtenir des premières données d'image après qu'un premier code a été reconstruit (S102) ; obtenir des premières données de flux de code codées par un second codeur, les premières données de flux de code correspondant à la résolution cible (S103).
PCT/CN2019/122168 2019-11-29 2019-11-29 Procédé, dispositif et système de transmission de données, et support d'enregistrement WO2021102992A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980061264.9A CN112771830A (zh) 2019-11-29 2019-11-29 数据传输方法、装置、系统及存储介质
PCT/CN2019/122168 WO2021102992A1 (fr) 2019-11-29 2019-11-29 Procédé, dispositif et système de transmission de données, et support d'enregistrement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/122168 WO2021102992A1 (fr) 2019-11-29 2019-11-29 Procédé, dispositif et système de transmission de données, et support d'enregistrement

Publications (1)

Publication Number Publication Date
WO2021102992A1 true WO2021102992A1 (fr) 2021-06-03

Family

ID=75693769

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/122168 WO2021102992A1 (fr) 2019-11-29 2019-11-29 Procédé, dispositif et système de transmission de données, et support d'enregistrement

Country Status (2)

Country Link
CN (1) CN112771830A (fr)
WO (1) WO2021102992A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080279532A1 (en) * 2004-11-30 2008-11-13 Matsushita Electric Industrial Co., Ltd. Recording and Reproducing Apparatus
CN101860749A (zh) * 2010-04-20 2010-10-13 中兴通讯股份有限公司 一种视频图像编码和解码方法及装置
CN108243339A (zh) * 2016-12-27 2018-07-03 浙江大学 图像编解码方法及装置
CN108737825A (zh) * 2017-04-13 2018-11-02 腾讯科技（深圳）有限公司 视频数据编码方法、装置、计算机设备和存储介质
CN110149515A (zh) * 2019-04-01 2019-08-20 西安万像电子科技有限公司 数据传输方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100208086A1 (en) * 2009-02-19 2010-08-19 Texas Instruments Incorporated Reduced-memory video stabilization
CN106131573B (zh) * 2016-06-27 2017-07-07 中南大学 一种hevc空间分辨率转码方法
CN109618159A (zh) * 2018-11-06 2019-04-12 建湖云飞数据科技有限公司 一种基于分辨率对应关系的自适应编码方法
CN109618154A (zh) * 2018-11-13 2019-04-12 建湖云飞数据科技有限公司 一种视频帧解码方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080279532A1 (en) * 2004-11-30 2008-11-13 Matsushita Electric Industrial Co., Ltd. Recording and Reproducing Apparatus
CN101860749A (zh) * 2010-04-20 2010-10-13 中兴通讯股份有限公司 一种视频图像编码和解码方法及装置
CN108243339A (zh) * 2016-12-27 2018-07-03 浙江大学 图像编解码方法及装置
CN108737825A (zh) * 2017-04-13 2018-11-02 腾讯科技（深圳）有限公司 视频数据编码方法、装置、计算机设备和存储介质
CN110149515A (zh) * 2019-04-01 2019-08-20 西安万像电子科技有限公司 数据传输方法及装置

Also Published As

Publication number Publication date
CN112771830A (zh) 2021-05-07

Similar Documents

Publication Publication Date Title
JP6087940B2 (ja) 復号化ピクチャ・バッファおよび参照ピクチャ・リストのための状態情報のシグナリング
WO2018196790A1 (fr) Procédé de lecture de vidéo, dispositif et système
US8675728B2 (en) Transmitting apparatus and method, and receiving apparatus and method
US20110002376A1 (en) Latency Minimization Via Pipelining of Processing Blocks
CN101310533A (zh) 服务获取的方法和设备
CN111221491A (zh) 交互控制方法及装置、电子设备、存储介质
US20150089367A1 (en) System and method for forwarding an application user interface
EP2804143A1 (fr) Système et procédé permettant de transférer un flux de commande graphique
TWI423674B (zh) 視訊接收器上之抹消方法以及利用視訊伺服器傳輸連流至接收器之方法
RU2689187C2 (ru) Декодирование и синтезирование кадров для неполных видеоданных
US10728563B2 (en) Data stream encoder configuration
CN101312536B (zh) 一种用于视频传输差错控制的视频编码方法及装置
TWI628958B (zh) 在低延遲視頻通訊系統中改善視頻表現之全訊框緩衝
US20140333640A1 (en) System and method for forwarding a graphics command stream
CN115708356A (zh) 视频处理方法、装置、设备及计算机可读存储介质
WO2024022317A1 (fr) Procédé et appareil de traitement de flux vidéo, support de stockage et dispositif électronique
US9426460B2 (en) Electronic devices for signaling multiple initial buffering parameters
WO2021102992A1 (fr) Procédé, dispositif et système de transmission de données, et support d'enregistrement
CN105657448B (zh) 一种编码视频流的转发方法、装置及系统
WO2023142665A1 (fr) Procédé et appareil de traitement d'image, et dispositif informatique, support de stockage et produit de programme
JP2001016584A (ja) ビデオ伝送方法および装置
JP4829867B2 (ja) 画像符号化装置及び画像復号装置
JP5200979B2 (ja) 画像転送装置、方法及びプログラム
US20140333639A1 (en) System and method for forwarding a graphics command stream
JP2024517915A (ja) データ処理方法、装置、コンピュータ機器及びコンピュータプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19954258

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19954258

Country of ref document: EP

Kind code of ref document: A1