WO2020062184A1 - Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations - Google Patents

Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations Download PDF

Info

Publication number
WO2020062184A1
WO2020062184A1 PCT/CN2018/108866 CN2018108866W WO2020062184A1 WO 2020062184 A1 WO2020062184 A1 WO 2020062184A1 CN 2018108866 W CN2018108866 W CN 2018108866W WO 2020062184 A1 WO2020062184 A1 WO 2020062184A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
code stream
target
frames
idr
Prior art date
Application number
PCT/CN2018/108866
Other languages
English (en)
Chinese (zh)
Inventor
刘细华
Original Assignee
深圳市大疆创新科技有限公司
大疆互娱科技（北京）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司, 大疆互娱科技（北京）有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201880037071.5A priority Critical patent/CN110741647A/zh
Priority to PCT/CN2018/108866 priority patent/WO2020062184A1/fr
Publication of WO2020062184A1 publication Critical patent/WO2020062184A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements

Definitions

  • the present invention relates to the field of computer technology, and in particular, to an image processing method, device, movable platform, and storage medium.
  • GDR Gradual Decoding Refresh
  • the receiving device Due to the characteristics of the inter-frame encoding frame used to record the difference between the current frame and the previous frame, after the receiving device receives and stores the bit stream transmitted via GDR, if the receiving device needs to perform image processing operations such as editing or random position playback of the original video , It needs to be decoded frame by frame for the code stream transmitted through GDR, which results in low image processing efficiency.
  • Embodiments of the present invention provide an image processing method, a device, a movable platform, and a storage medium, which are beneficial to improving image processing efficiency.
  • a first aspect of an embodiment of the present invention is to provide an image processing method, including:
  • the original code stream of the target video file where the original code stream includes the code streams of multiple frames obtained by encoding the target video file, and the first frame in the frame sequence corresponding to any one of the multiple frames is intra-frame encoding Frames, other frames are inter-coded frames;
  • a second aspect of an embodiment of the present invention is another image processing method provided, including:
  • a third aspect of the embodiments of the present invention is still another image processing method, including:
  • a fourth aspect of the embodiments of the present invention is to provide an image processing apparatus, including:
  • the memory is used to store program code
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • the original code stream of the target video file where the original code stream includes the code streams of multiple frames obtained by encoding the target video file, and the first frame in the frame sequence corresponding to any one of the multiple frames is intra-frame encoding Frames, other frames are inter-coded frames;
  • a fifth aspect of the embodiments of the present invention provides another image processing apparatus, including:
  • the memory is used to store program code
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • Obtaining a current code stream of a target video file where the current code stream is a code stream including at least one inter-coded frame and at least two instant decoding refresh IDR frames;
  • a sixth aspect of the embodiments of the present invention provides another image processing apparatus, including:
  • the memory is used to store program code
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • a power system mounted on the fuselage and configured to provide power to the movable platform
  • the image processing method, device, and mobile platform provided by the embodiments of the present invention can obtain the decoded data of each frame included in the original bitstream by decoding the obtained original bitstream of the target video file, so as to further analyze the original bitstream.
  • the decoded data is encoded to obtain the current code stream of the target video file, so that the user can decode the image of any position in the target video file and obtain the position according to the intra-coded frames and IDR frames included in the current code stream.
  • the corresponding decoded image makes the image processing efficiency improve.
  • FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an original code stream provided by an embodiment of the present invention.
  • 5a is a schematic diagram of a target frame sequence according to an embodiment of the present invention.
  • 5b is a schematic diagram of a target frame sequence according to another embodiment of the present invention.
  • FIG. 6 is a schematic diagram of an image processing method according to an embodiment of the present invention.
  • FIG. 7 is a schematic flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 8 is a schematic block diagram of an image processing apparatus of a movable platform provided by an embodiment of the present invention.
  • FIG. 9 is a schematic block diagram of an image processing apparatus of a movable platform according to another embodiment of the present invention.
  • FIG. 10 is a schematic block diagram of an image processing apparatus of a movable platform according to another embodiment of the present invention.
  • a component when a component is called “fixed to” another component, it may be directly on another component or a centered component may exist. When a component is considered to be “connected” to another component, it can be directly connected to another component or a centered component may exist at the same time.
  • the image processing method, device, and mobile platform provided by the embodiments of the present invention can obtain the decoded data of each frame included in the original bitstream by decoding the obtained original bitstream of the target video file, so as to further analyze the original bitstream.
  • the decoded data is encoded to obtain the current code stream of the target video file, so that the user can decode the image of any position in the target video file and obtain the position according to the intra-coded frames and IDR frames included in the current code stream.
  • the corresponding decoded image makes the image processing efficiency improve.
  • the original bitstream of the transmitted video file (such as the target video file) includes only frames.
  • Inter-coded frames such as B-frames and / or P-frames, excluding intra-frame coded frames, which may be, for example, I-frames and / or instantaneous decode refreshes IDR frame (Instantaneous Decoding, Refresh, Fram, IDR frame), in one embodiment, I frame, which is a kind of independent ordinary intra frame encoded I frame with all the information, ordinary I frame can be independently decoded, without reference to other frames The decoded image is obtained.
  • the first ordinary I frame is an IDR frame.
  • the role of the IDR frame is to refresh immediately. Since a new sequence is recalculated from the IDR frame to start encoding, sequence errors do not continue to propagate during decoding, and I Frames do not have the capability of random access, this function is only undertaken by the IDR.
  • IDR frames will cause DPB (Decoded Picture Buffer, Reference Frame List) to be emptied, but I frames will not.
  • IDR frames must be I frames, but I frames are not necessarily IDR frames.
  • the P frame is a forward predictive coding frame, and the P frame represents the difference between this frame and the previous frame. Therefore, when decoding a P frame to obtain a decoded image, the image needs to be decoded from the beginning until it is decoded to P The decoded image is obtained at the corresponding position of the frame; the B frame is a bidirectional predictive interpolation coded frame. The B frame records the difference between the current frame and the previous and subsequent frames. When decoding the B frame to decode the image, you need to refer to the code before and after the B frame. Stream and decode to get the decoded image corresponding to the B frame.
  • the data size of each frame is relatively stable, but only the first frame in the original code stream is an intra-coded frame.
  • Mobile terminals such as drones
  • After decoding the original bitstream if you want to decode a part of the file to obtain a decoded image, you must decode from the first intra-coded frame to the frame position corresponding to the required decoded image. It can be seen that the code that uses GDR for image transmission When decoding a stream, decoding at any position in the target video file needs to refer to the first intra-coded frame of the corresponding sequence, resulting in inefficient decoding of the video file.
  • the present invention proposes an image processing method, which can replace one or more inter-encoded frames in the H264 code stream transmitted by the GDR image with intra-encoded frames ( (Ie IDR frame), so that when the mobile platform decodes frames at any position in the H264 code stream, it does not need to refer to the first intra-coded frame in the sequence every time, and it can start decoding from the middle position of the H264 code stream.
  • Ie IDR frame intra-encoded frames
  • the movable platform When the movable platform detects that the encoder is initialized, it obtains the original code stream of the target video file and determines the original code. The stream is decoded to obtain decoded data.
  • the decoded data may be, for example, YUV data, so as to determine whether the data amount of the obtained decoded data meets a preset condition.
  • the preset condition may be, for example, a multiple of 45 frames.
  • YUV data is a color coding method, which is mainly used in the field of television systems and analog video. It refers to separating luminance parameters (Y: Luminance or Luma) and chrominance parameters (UV: Chrominance or Chroma). The pixel encoding format for the representation.
  • the corresponding encoder when the mobile platform performs an encoding operation, may be initialized according to the first parameter information of the H264 stream data, where the first parameter information includes a sequence parameter set SPS (Sequence Paramater Set, SPS) and picture parameter set PPS (Picture, Paramter Set, PPS).
  • SPS Sequence Paramater Set
  • PPS Picture parameter set
  • the encoder is initialized, the SPS and PPS data such as profile, level, num_ref_frames, pic_init_qp_minus26 and other parameters must be completely consistent, and then set the encoder to encode YUV data as For IDR frames, the encoded data needs to be reparsed and repacked according to the SPS and PPS of the original bitstream, and then stored in the storage medium.
  • Subsequent stream data may be changed due to the insertion of a new IDR frame and log2_max_frame_num_minus4. You need to parse and update frame_num, modify it to new parameters, and then packetize it. Finally, mux becomes a playable mp4 file.
  • the above encoder may be implemented by software encoding, and the above software encoding may be libx264 or openh26, or any other software encoding scheme.
  • the general display saving process includes four major modules: split analysis, decoder decoding, display, and encoder re-encoding.
  • the data parsing module receives the stream of H264 packets, it needs to unpack and merge the same frame of the stream into a complete stream data packet to prepare the decoder for decoding; the decoder module receives the stream data Then, according to the normal decoding sequence and decoding logic, the code stream data is decoded into the correct YUV data; then the display module converts the YUV data into RGB data that the display can normally display, and displays it on the display end; finally the encoder re-encodes the module The YUV data decoded by the decoder is re-encoded into a H264 code stream.
  • the working mode of the encoder re-encoding module is: the encoder needs some information related to the original code stream, because the original code stream is not completely re-encoded, but only re-encoded after a certain data frame, generally
  • the encoder needs some information related to the original code stream, because the original code stream is not completely re-encoded, but only re-encoded after a certain data frame, generally
  • This time interval is generally 1-2s.
  • the value can be 1.5. s.
  • an IDR frame is required approximately every 45 frames, that is, every 45 frames, the decoded YUV data is re-encoded.
  • the encoder may perform specific steps when re-encoding: the first frame of the original H264 code stream and the decoded YUV data are transmitted to the encoding module at the same time. If the encoder has not been initialized, the SPS in the H264 code stream is used And / or PPS information, initialize the encoder to encode YUV data, generate new H264 data, modify the new H264's packet parameters and repack with the parameters of the original stream, and hand them to the multiplexer (mux, mux ) The module writes to the file. If the number of frames is not a multiple of 45, modify the parameters of the old code stream, such as frame_num and other parameters, then perform entropy encoding, and then write to the file by mux.
  • the image processing method of the movable platform in the embodiment of the present invention is described as an example below.
  • FIG. 2 is a flowchart of an image processing method for a mobile platform according to an embodiment of the present invention. As shown in FIG. 2, the method includes the following steps:
  • S201 Obtain an original code stream of a target video file, where the original code stream includes code streams of multiple frames obtained by encoding the target video file, and any one of the multiple frames corresponds to a first frame in a frame sequence as a frame. Intra-coded frames, other frames are inter-coded frames.
  • the mobile platform can obtain the original code stream of the target video file transmitted through GDR (Gradual Decoding, Refresh, GDR), and can use the GDR format to encode the transmitted target video file.
  • the original code stream Corresponds to the H264 code stream, in which the GDR is completely refreshed from a frame, which realizes the gradual refresh by the method of P frames including I block groups, which will bring better network adaptability.
  • the first frame of any frame sequence is an intra-frame encoded frame, such as an I frame, and the other frames are inter-frame encoded frames, such as a P frame and / or a B frame.
  • inter-coded frames such as B-frames and / or P-frames
  • the mobile platform decodes any frame sequence in the original code stream to obtain a decoded image.
  • the frame to be decoded may be the closest I to the frame to be decoded.
  • the number of frames spaced between frames is large, which results in a low decoding efficiency of the mobile platform for the decoded frames, making the mobile platform's random access speed to the target video file lower, and weakening the user experience. Therefore, subsequent steps S202 and S203 may be performed to obtain a new code stream (that is, the current code stream), thereby improving the access speed of the target video file.
  • intra-coded frames can be independently decoded, and the characteristics of decoded images can be obtained without reference to other frames.
  • Intra-coded frames can be inserted into the original code stream. Therefore, in order to achieve The intra-coded frame is inserted into the original code stream to achieve random access to the target video file. After the original code stream is obtained, the original code stream is decoded to obtain decoded data of each of the multiple frames. .
  • the decoded data may be, for example, YUV data, and the YUV data may be converted into RGB data that can be normally displayed by a display and displayed on the display end.
  • the movable platform may The target video file is re-encoded, that is, S203 is performed instead, and an intra-coded frame is inserted into the current code stream obtained by re-encoding the target video file.
  • the original code stream When the original code stream is decoded, the original code stream may be parsed and processed to obtain the code stream of each frame, so that the code stream of each frame may be decoded and decoded for each of the frames. data.
  • S203 Encode the decoded data according to the first parameter information of the original code stream to obtain a current code stream of the target video file, where the current code stream includes at least one inter-encoded frame and at least two Instant decoding refreshes the bit stream of the IDR frame.
  • the first parameter information includes code stream parameters, such as sequence parameter set SPS and / or image parameter set PPS, used to encode decoded data, and the sequence parameter set SPS and / or image parameter set PPS Both are important parameter sets in the H264 stream, and are saved together in the header file of the video file.
  • code stream parameters such as sequence parameter set SPS and / or image parameter set PPS
  • the sequence parameter set SPS contains parameters for a continuously encoded video sequence, such as frame number, reference frame number, decoded image Size and frame field coding mode selection identification;
  • the image parameter set PPS corresponds to a certain image or several images in a sequence, parameters such as identifier, entropy encoding mode selection identification, number of slices, initial quantization parameters Wait, if there is no PPS or SPS in the H264 stream, the decoder will not be able to parse the stream data, which will cause the target video file to be unplayable.
  • the related parameters included in the PPS and / or SPS may also be used to initialize an encoder, so that the encoder re-encodes decoded data (such as YUV data) obtained by decoding the original bitstream.
  • decoded data such as YUV data
  • the interval at which each frame sequence is inserted into the intra-coded frame can be determined, and the decoded data at the corresponding position is encoded into the intra-frame based on the interval Encode frames so that each frame sequence includes at least two intra-encoded frames.
  • the two intra-encoded frames included can be, for example, instant decode refresh IDR frames, and other frames of the current code stream obtained by re-encoding are intra.
  • Encoded frames such as the P and / or B frames described above.
  • the current bitstream obtained after encoding the decoded data includes at least two IDR frames, based on the characteristics of the IDR frame immediately refreshed, when decoding any frame to be decoded in the frame sequence to obtain a decoded image, refer to the frame sequence
  • the IDR frame closest to the frame to be decoded, and decoding is started from the latest IDR frame until it is decoded to the corresponding position of the frame to be decoded, so that the decoded image corresponding to the frame to be decoded can be decoded without using the original stream frame sequence
  • the first intra-coded frame is decoded to obtain a decoded image, which realizes random access to the target video file and can improve the processing efficiency of the image.
  • the image processing method, device, and mobile platform provided by the embodiments of the present invention can obtain the decoded data of each frame included in the original bitstream by decoding the obtained original bitstream of the target video file, so as to further analyze the original bitstream.
  • the decoded data is encoded to obtain the current code stream of the target video file, so that the user can decode the image of any position in the target video file and obtain the position according to the intra-coded frames and IDR frames included in the current code stream.
  • the corresponding decoded image makes the image processing efficiency improve.
  • FIG. 3 is a flowchart of an image processing method for a mobile platform according to another embodiment of the present invention. As shown in FIG. 3, the method includes the following steps:
  • S301-S302 can be referred to the descriptions of S201-S202, which will not be repeated here.
  • the specific execution steps are:
  • the target frame sequence is any frame sequence in the original code stream of the target video file.
  • the length of the target frame sequence refers to the target frame sequence.
  • the number of all frames included in the frame, the length of the target frame sequence corresponds to the playback duration of the target video file means: the duration required to finish playing all the frames in the target frame sequence, correspondingly, the first
  • the insertion period is a time period. For example, if the length of the target frame sequence corresponds to a playback duration of the target video file of 2 seconds, the first insertion period may correspond to 1s or 1.5s, etc.
  • the target frame sequence length corresponds to the playback duration of the target video file and the first insertion cycle, and a target frame that needs to be encoded into an IDR frame is determined from the plurality of frames.
  • the original bitstream of any of the target video files includes at least one frame sequence
  • the at least one frame sequence may be determined by the first parameter information (the first (The parameter information includes PPS and / or SPS), and / or the at least one frame sequence may also be distinguished by second parameter information included in the code stream, and when distinguished by the second parameter information,
  • the second parameter information may be set to 0 to identify a start position of each frame sequence included in the code stream.
  • the mobile platform when the mobile platform determines a target frame that needs to be encoded into an IDR frame among the multiple frames, it may further perform steps:
  • S23 Determine a target frame to be encoded into an IDR frame among the multiple frames according to the second insertion period and the sum of the number of frames.
  • the total number of frames of the target frame sequence refers to: the number of all frames included in the target frame sequence.
  • coding needs to be obtained according to the total number of frames of the target frame sequence.
  • the second insertion cycle of the completed IDR frame in the target frame sequence is a frame number cycle.
  • the second insertion cycle refers to the insertion of a target frame sequence every frame number indicated by the second insertion cycle.
  • the IDR frame is to determine the position of the IDR frame to be encoded in the multiple frames, that is, to determine the target frame to be encoded into the IDR frame in the multiple frames.
  • the total number of frames of the target frame sequence is assumed to be 50, and the corresponding second insertion period is 45 or the like.
  • the mobile platform when the mobile platform determines a target frame to be encoded into an IDR frame among the multiple frames, it may further perform steps:
  • s32 Determine a target frame to be encoded into an IDR frame among the multiple frames according to a sum of the third insertion period and the number of the multiple frames.
  • the number of frames spaced between any two IDR frames inserted may also be determined according to the total number of frames included in the code stream, that is, among multiple frames of the original code stream. , Inserting at least one IDR frame according to the third insertion period.
  • S304 may be performed again.
  • a step may be performed first: determining that the length of the target frame sequence corresponds to the playback duration of the target video file divided by the first Whether the first remainder obtained during an insertion period is zero; when the first remainder is not zero, the IDR frame (that is, the IDR frame that needs to be encoded) required for insertion determined according to the first insertion period is described
  • the position cannot guarantee that the required IDR frame is inserted in each insertion cycle and / or each frame sequence, and then the second parameter information of the original code stream needs to be adjusted, and the second parameter information is adjusted, that is, The position where the required IDR frame is inserted is adjusted, that is, the target frame that needs to be encoded into the IDR frame is re-determined among the multiple frames.
  • the second parameter information corresponding to the position where the required IDR frame is inserted may be set to 0, and the second parameter information corresponding to the frame after the required IDR frame that is
  • the decoded data of the target frame at the determined position may be performed according to the first parameter information of the original code stream and the adjusted second parameter information of the original code stream. Encoding to obtain the code stream of the at least one IDR frame, and inserting the obtained code stream of the at least one IDR frame into a desired position.
  • the target frame sequence length is 50, and if the target frame sequence length corresponds to the playback time of the target video file is 3 seconds, the required The first insertion cycle of the IDR frame in the target frame sequence is 1.2 seconds, because the length of the target frame sequence corresponds to the playback duration of the target video file of 2 seconds divided by the first insertion cycle of 1.2 seconds If the remainder is not 0, the second parameter information of the original bitstream is adjusted.
  • the adjusted second parameter information of the target frame sequence may be as shown in FIG. 5b. As shown in FIG. 5b, The position of the target frame sequence whose original second parameter information is 45 is adjusted to 0, and the second parameter information of the subsequent frame position is modified. In one embodiment, the position adjusted to 0 is the IDR frame inserted. Position, that is, the position in the original code stream where the target frame to be encoded into the IDR frame is located.
  • a step may further be performed: determining the total number of frames of the target frame sequence divided by the second insertion period. Whether the second remainder is zero; when the second remainder is not zero, it also indicates that the position of the IDR frame required for insertion determined according to the second insertion period cannot guarantee each insertion period and / or each frame
  • the required IDR frames are inserted in the sequence, and the second parameter information of the original bitstream needs to be adjusted.
  • the second parameter information corresponding to the position corresponding to the required IDR frame may also be inserted. Set it to 0, and sequentially adjust the second parameter information corresponding to the second parameter information corresponding to the frame after the inserted required IDR frame.
  • the frames at the determined positions may also be paired according to the first parameter information of the original code stream and the adjusted second parameter information of the original code stream.
  • Encode the decoded data to obtain a code stream of at least one IDR frame, and insert the obtained code stream of the at least one IDR frame into a desired position.
  • the second parameter information may be, for example, fram_num, and the modification process of the second parameter information may refer to the description in FIG. 5a and FIG. 5b, and details are not described herein again.
  • a step may further be performed: determining a sum of the number of the plurality of frames divided by a third remainder obtained by a third insertion cycle. Whether it is 0, and when the third is then not 0, adjusting the second parameter information of the original code stream so that the sum of the number of the plurality of frames divided by the adjusted third insertion period is the remainder It is 0, that is, IDR frames are inserted at a certain interval among multiple frames of the original code stream, and the interval may be 45, for example.
  • the movable platform executes S304, first obtain the first parameter information of the code stream of the first frame in the frame sequence corresponding to the multiple frames of the original code stream, and according to the first of the code streams of the first frame, Parameter information to initialize the encoder; and then use the encoder after the initialization process to encode the decoded data of the target frame to obtain an updated code stream, so that it can be based on the first parameter information of the original code stream,
  • the first parameter information of the updated code stream is adjusted, and the updated code stream is further encapsulated according to the adjusted first parameter information, so as to obtain a code stream of at least one IDR frame.
  • the first remainder when the first remainder is not zero, it can be further judged whether the insertion position corresponding to the first insertion period is an intra-coded frame. If so, the original code stream is indicated according to the first insertion period. There are already one or more intra-frame encoding frames in the determined insertion position, so it is not necessary to re-insert IDR frames at this insertion position, so the corresponding positions of the existing intra-frame encoding frames in the original code stream are not required. If the second parameter information of the original bitstream is not adjusted, the second parameter information of the original bitstream is adjusted. If the second remainder or the third remainder is not zero, the adjustment may also be performed. First determine whether there is an intra-coded frame at the insertion position determined according to the second insertion cycle or the insertion position determined according to the third insertion cycle, and if not, perform the above-mentioned step of adjusting the second parameter information.
  • each frame sequence included in the original bitstream may be processed sequentially according to the foregoing steps.
  • the encoder Before processing any frame sequence, the encoder needs to be processed according to the first parameter information corresponding to the sequence. Initialization processing.
  • the current code stream including the IDR frame can also be entropy-coded, so that a multiplexer (mux) is written to a file for storage.
  • a multiplexer multiplexer
  • the decoded data of the target frame is encoded according to the first parameter information of the original code stream to obtain a code stream of at least one IDR frame, so that the code of the at least one IDR frame can be
  • the position of the target frame in the original bitstream is replaced by the stream to obtain the current bitstream of the target video file, so that the target video file can be decoded according to the IDR frame included in the current bitstream to quickly obtain a decoded image and improve The efficiency of image processing.
  • the mobile device may obtain the target video file from a device (such as a camera) or a module having a video camera function, and perform the image processing method according to the image processing method shown in FIG. 2 or FIG. 3.
  • the target video file is processed to obtain a current bitstream of the target video file.
  • decoding starts with the first IDR frame before the target frame, and With reference to the inter-encoded frame between the target true and the first IDR frame, the decoded image corresponding to the target frame is decoded to achieve fast decoding of the target video file's current bitstream at any position, thereby improving mobility Image processing efficiency of the terminal.
  • the device with a video camera function may be, for example, a smart terminal that establishes a communication connection with the mobile platform, such as a smart phone, a tablet computer, or a smart wearable device.
  • the movable platform may be, for example, a drone or a car.
  • the movable platform and the device with a video camera function are not limited.
  • the movable platform obtains a current stream of a target video file, and when a selection operation of a target frame in the target video file is detected, it is determined to be located before the target frame.
  • the first IDR frame and the inter-frame encoded frame between the target frame and the IDR frame decode the code stream of the target frame to obtain the image corresponding to the target frame, and achieve the target frame at any position in the current code stream Fast decoding, because the decoding of a target frame at an arbitrary position can no longer refer to the first intra-coded frame of the frame sequence corresponding to the target frame, which improves the image processing efficiency of the mobile platform.
  • the method includes the following steps:
  • a target frame that needs to be encoded into an instant decoding and refreshing IDR frame is determined according to an effect position of the selected operation on an original bitstream of the target video file.
  • S702 Perform decoding processing on the original code stream until the decoded data of the target frame is obtained.
  • S703 Encode the decoded data of the target frame according to the first parameter information of the original code stream to obtain the current code stream of the target frame.
  • S704 Replace the code stream of the target frame in the original code stream with the current code stream to obtain the current code stream of the target video file.
  • the selected operation may be sent to the mobile platform by a smart terminal that establishes a communication connection with the mobile platform, or may be a selected operation directly acting on the mobile platform.
  • the detection of the selection operation of the target video file indicates that the user has a need to edit the target file.
  • the selection operation inserts an IDR frame at the position of the original stream of the target video file.
  • the position at which the selected operation acts is the desired position for inserting the IDR frame
  • the frame that the selected operation plays in the original code stream is the target frame for inserting the IDR frame. Therefore, at the desired position When inserting an IDR frame, only the original code stream can be decoded to the target frame to obtain the target frame and the decoded data of all frames before the target frame, so that the target frame can be decoded according to the first parameter information of the original code stream
  • the data is encoded to obtain the current code stream of the target frame, so that the target code stream in the original code stream can be replaced with the re-encoded current code stream to obtain the current code stream of the target video file, according to the actual needs of users Processing of the target video file.
  • the movable platform when the movable platform detects the selected operation on the target video file, it can determine the action position of the selected operation on the original stream of the target video file, that is, determine the selected operation.
  • the original code stream After the target frame acting in the original code stream, the original code stream is decoded until the decoded data of the target frame is obtained, and then the decoded data is encoded and processed according to the first parameter information to obtain the current code stream of the target frame.
  • the bitstream of the target frame in the original bitstream is replaced with the current bitstream obtained by re-encoding, which realizes the personalized processing of the target video file according to user needs and improves user satisfaction.
  • FIG. 8 is a structural diagram of an image processing apparatus of a movable platform provided by an embodiment of the present invention.
  • the image processing apparatus 800 of the movable platform includes The memory 801 and the processor 802.
  • the memory 802 stores program code.
  • the processor 802 calls the program code in the memory.
  • the processor 802 performs the following operations:
  • the original code stream of the target video file where the original code stream includes the code streams of multiple frames obtained by encoding the target video file, and the first frame in the frame sequence corresponding to any one of the multiple frames is intra-frame encoding Frames, other frames are inter-coded frames;
  • the first parameter information includes a sequence parameter set SPS and / or an image parameter set PPS
  • the processor 802 encodes the decoded data according to the first parameter information of the original code stream, When the current bitstream of the target video file is obtained, the following operations are performed:
  • the processor 802 when the processor 802 determines a target frame to be encoded into an IDR frame among the multiple frames, the processor 802 performs the following operations:
  • a target frame that needs to be encoded into an IDR frame is determined among the multiple frames.
  • processor 802 when the processor 802 calls the program code, it also performs the following operations:
  • And encoding the decoded data of the target frame according to the first parameter information of the original code stream to obtain the code stream of the at least one IDR frame includes:
  • the processor 802 when the processor 802 determines a target frame to be encoded into an IDR frame among the multiple frames, the processor 802 performs the following operations:
  • a target frame that needs to be encoded into an IDR frame is determined among the multiple frames.
  • processor 802 when the processor 802 calls the program code, it also performs the following operations:
  • the encoding the decoded data of the target frame according to the first parameter information of the original code stream to obtain the code stream of the at least one IDR frame includes:
  • the processor 802 when the processor 802 encodes the decoded data of the target frame according to the first parameter information of the original code stream to obtain the code stream of the at least one IDR frame, it performs the following operations :
  • processor 802 when the processor 802 calls the program code, it also performs the following operations:
  • the decoded data of the target frame is encoded by using the encoder after the initialization process to obtain the updated code stream.
  • processor 802 when the processor 802 calls the program code, it also performs the following operations:
  • processor 802 when the processor 802 calls the program code, it also performs the following operations:
  • Decoding the original bitstream to obtain decoded data of each of the frames includes:
  • the image processing apparatus of the movable platform provided by this embodiment can execute the image processing methods shown in FIG. 2 and FIG. 3 provided by the foregoing embodiments, and the execution manners and beneficial effects thereof are similar, and will not be repeated here.
  • FIG. 9 is a structural diagram of an image processing apparatus for a movable platform according to another embodiment of the present invention.
  • the image processing apparatus 900 includes a memory 901 and a processor 902.
  • the memory 902 stores program code.
  • the processor 902 calls the program code in the memory.
  • the processor 902 performs the following operations:
  • Obtaining a current code stream of a target video file where the current code stream is a code stream including at least one inter-coded frame and at least two instant decoding refresh IDR frames;
  • the image processing apparatus of the movable platform provided by this embodiment can execute the image processing method shown in FIG. 6 provided by the foregoing embodiment, and the execution manner and beneficial effects thereof are similar, and will not be repeated here.
  • FIG. 10 is a structural diagram of an image processing apparatus for a movable platform according to another embodiment of the present invention.
  • the image processing apparatus 100 includes a memory 101 and a processor 102.
  • the memory 102 stores program code.
  • the processor 102 calls the program code in the memory.
  • the processor 102 performs the following operations:
  • the image processing apparatus of the movable platform provided by this embodiment can execute the image processing method shown in FIG. 7 provided by the foregoing embodiment, and the execution manner and beneficial effects thereof are similar, and will not be repeated here.
  • An embodiment of the present invention further provides a movable platform.
  • the movable platform includes:
  • a power system mounted on the fuselage and configured to provide power to the movable platform
  • an image processing device of a movable platform as shown in FIG. 8 or an image processing device of a movable platform as shown in FIG. 9, or an image processing device of a movable platform as shown in FIG. 10.
  • the movable platform further includes:
  • An image sensor is mounted on the body and used to obtain a target video file.
  • the image sensor includes a video image sensor and / or a text image sensor.
  • the movable platform further includes:
  • a communication device is installed on the fuselage and is used for information interaction with the control terminal.
  • the movable platform includes at least one of the following: a drone and a car.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon les modes de réalisation, la présente invention concerne un procédé de traitement d'images pour une plateforme mobile, comprenant : l'acquisition d'un flux de codes d'origine d'un fichier vidéo cible, le flux de codes d'origine comprenant des flux de codes de multiple trames obtenues par le codage du fichier vidéo cible, une première trame d'une séquence de trames correspondant à l'une quelconque des multiples trames étant une trame codée intra-trame, et les autres trames étant des trames codées inter-trame ; le décodage du flux de codes d'origine pour obtenir des données décodées de chacune des trames ; et le codage des données décodées en fonction de premières informations de paramètres du flux de codes d'origine pour obtenir un flux de codes en cours du fichier vidéo cible, le flux de codes en cours étant un flux de codes comprenant au moins une trame codée inter-trame et au moins deux trames de rafraîchissement de décodage instantané (IDR). Ainsi, la présente invention est bénéfique pour améliorer l'efficacité de traitement d'images de la plateforme mobile.
PCT/CN2018/108866 2018-09-29 2018-09-29 Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations WO2020062184A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880037071.5A CN110741647A (zh) 2018-09-29 2018-09-29 一种图像处理方法、装置、可移动平台及存储介质
PCT/CN2018/108866 WO2020062184A1 (fr) 2018-09-29 2018-09-29 Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/108866 WO2020062184A1 (fr) 2018-09-29 2018-09-29 Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations

Publications (1)

Publication Number Publication Date
WO2020062184A1 true WO2020062184A1 (fr) 2020-04-02

Family

ID=69236604

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/108866 WO2020062184A1 (fr) 2018-09-29 2018-09-29 Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations

Country Status (2)

Country Link
CN (1) CN110741647A (fr)
WO (1) WO2020062184A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111343503B (zh) * 2020-03-31 2022-03-04 北京金山云网络技术有限公司 视频的转码方法、装置、电子设备及存储介质
CN111770390B (zh) * 2020-07-09 2024-02-09 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014165653A2 (fr) * 2013-04-05 2014-10-09 Qualcomm Incorporated Alignements d'image en codage vidéo multicouche
CN104396265A (zh) * 2012-04-27 2015-03-04 高通股份有限公司 在视频译码中从清晰随机存取图片的全面随机存取
CN105657448A (zh) * 2014-11-24 2016-06-08 科腾科技(北京)有限公司 一种编码视频流的转发方法、装置及系统
CN107333133A (zh) * 2016-04-28 2017-11-07 浙江大华技术股份有限公司 一种码流接收设备的码流编码的方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100959538B1 (ko) * 2006-03-30 2010-05-27 엘지전자 주식회사 비디오 신호를 디코딩/인코딩하기 위한 방법 및 장치
CN101184230A (zh) * 2006-11-13 2008-05-21 中兴通讯股份有限公司 在视频序列中插入切换帧的方法
CN102630013B (zh) * 2012-04-01 2013-10-16 北京捷成世纪科技股份有限公司 基于场景切换的码率控制视频压缩方法和装置
CN103873869A (zh) * 2012-12-18 2014-06-18 北京市勤能通达科技有限公司 一种码率控制方法
CN106791875B (zh) * 2016-11-30 2020-03-31 华为技术有限公司 视频数据解码方法、编码方法以及相关设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104396265A (zh) * 2012-04-27 2015-03-04 高通股份有限公司 在视频译码中从清晰随机存取图片的全面随机存取
WO2014165653A2 (fr) * 2013-04-05 2014-10-09 Qualcomm Incorporated Alignements d'image en codage vidéo multicouche
CN105657448A (zh) * 2014-11-24 2016-06-08 科腾科技(北京)有限公司 一种编码视频流的转发方法、装置及系统
CN107333133A (zh) * 2016-04-28 2017-11-07 浙江大华技术股份有限公司 一种码流接收设备的码流编码的方法及装置

Also Published As

Publication number Publication date
CN110741647A (zh) 2020-01-31

Similar Documents

Publication Publication Date Title
US10728594B2 (en) Method and apparatus for transmitting data of mobile terminal
US11395000B2 (en) Dependent random access point pictures
US8615038B2 (en) Video coding, decoding and hypothetical reference decoder
US10602190B2 (en) Temporal sub-layer descriptor
US20160234522A1 (en) Video Decoding
US20160261878A1 (en) Signaling information for coding
WO2018010662A1 (fr) Procédé et dispositif de transcodage de fichier vidéo et support d'informations
US20060239563A1 (en) Method and device for compressed domain video editing
WO2020108033A1 (fr) Procédé et dispositif de transcodage et support de stockage lisible par ordinateur
EP2713620B1 (fr) Appareil de décodage vidéo
CN112073737A (zh) 在直播视频流应用中重新编码预测的图像帧
US20140375889A1 (en) Method, apparatus, and system for switching transport stream
KR20230098717A (ko) 인코딩 방법, 인코딩된 비트스트림 및 인코딩 디바이스
WO2019128668A1 (fr) Procédé et appareil de traitement d'un train de bits vidéo, dispositif informatique et support d'enregistrement lisible
WO2020062184A1 (fr) Procédé et dispositif de traitement d'images, plateforme mobile et support d'informations
JP2002320228A (ja) 信号処理装置
CN111225211A (zh) 转码方法、转码装置和计算机可读存储介质
CN105379281B (zh) 用于使用图形处理器的视频解码的图片参考控制
WO2015052938A1 (fr) Liste des plus grandes sous-couches temporelles
US20110299591A1 (en) Video processing apparatus and method
KR20240001212A (ko) 비디오 스트리밍에서의 파라미터 세트 지시 방법
KR20220144241A (ko) 서버 및 그 제어 방법
CN112422974A (zh) 视频编码方法、装置、计算机设备及存储介质
JP2013093713A (ja) 符号化装置及び方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18935353

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18935353

Country of ref document: EP

Kind code of ref document: A1