WO2020177724A1 - 编码方法、图像编码器及图像传输系统 - Google Patents

编码方法、图像编码器及图像传输系统 Download PDF

Info

Publication number
WO2020177724A1
WO2020177724A1 PCT/CN2020/077824 CN2020077824W WO2020177724A1 WO 2020177724 A1 WO2020177724 A1 WO 2020177724A1 CN 2020077824 W CN2020077824 W CN 2020077824W WO 2020177724 A1 WO2020177724 A1 WO 2020177724A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
image
coded
sequence
code stream
Prior art date
Application number
PCT/CN2020/077824
Other languages
English (en)
French (fr)
Inventor
李昭早
Original Assignee
深圳市道通智能航空技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术有限公司 filed Critical 深圳市道通智能航空技术有限公司
Publication of WO2020177724A1 publication Critical patent/WO2020177724A1/zh
Priority to US17/467,364 priority Critical patent/US12003887B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/39—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/12—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
    • H04N7/122—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal involving expansion and subsequent compression of a signal segment, e.g. a frame, a line
    • H04N7/125—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal involving expansion and subsequent compression of a signal segment, e.g. a frame, a line the signal segment being a picture element
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/114—Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2387—Stream processing in response to a playback request from an end-user, e.g. for trick-play
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/93—Regeneration of the television signal or of selected parts thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • This application relates to the field of image processing technology, and in particular to an encoding method, an image encoder, and an image transmission system.
  • embodiments of the present invention provide an encoding method, image encoder, and image transmission system that can meet diverse service requirements.
  • an encoding method for aerial cameras In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions: an encoding method for aerial cameras.
  • the encoding method includes: processing original image data to obtain at least two different output images; performing multi-stream encoding on the at least two output images to form corresponding at least two encoding data to support at least two services demand.
  • the encoding method includes: processing the original image data to obtain at least two output images; performing one bit stream encoding on each output image of the at least two output images to form at least two encoded data; wherein, Different channels of output images correspond to different code streams; the at least two channels of coded data are used to support at least two service requirements, and each channel of coded data in the at least two channels of coded data is used to support the at least two service requirements A business requirement in
  • the service requirements include video recording, online playback, and remote live broadcasting;
  • the at least two output images include: a first output image, a second output image, and a third output image; then,
  • the step of performing multi-stream encoding on at least two channels of the output image to form corresponding at least two channels of encoded data to support at least two image operations specifically includes: performing a first code stream on the first output image Encoding to form first coded data supporting video recording; performing a second code stream encoding on the second output image to form second coded data supporting online playback; performing a third code stream encoding on the third output image , To form the third coded data supporting remote live broadcast.
  • the first output image is a first image frame sequence with an original resolution
  • the step of performing a first code stream encoding on the first output image to form first encoded data supporting video recording specifically includes: determining that at least one image frame in the first image frame sequence is a first key frame , Wherein the image frame between the two first key frames is the first non-key frame; intra-frame compression encoding is performed on the first key frame to obtain a complete encoded frame; and the previous image frame is referred to,
  • the non-key frame is subjected to inter-frame compression coding to obtain a forward predictive coded frame; based on the complete coded frame and the forward predictive coded frame, several consecutive coding sequences are formed, wherein each coded sequence is located in the sequence
  • the first complete coded frame and the forward predictive coded frame arranged in sequence are composed.
  • the method further includes: setting an upper limit of encoding quality for encoding of the first code stream, so that the data flow of the first encoded data is kept within a set range.
  • the first bitstream encoding is variable rate encoding.
  • the second output image is a second image frame sequence with a second resolution, wherein the second resolution is lower than the original resolution; then,
  • the step of performing a second bitstream encoding on the second output image to form second encoded data supporting online playback specifically includes: at least one image frame in the second image frame sequence is a second The key frame, the image frame between the two second key frames is the second non-key frame; the second key frame is intra-frame compression-encoded to obtain a complete coded frame; and in the form of frame skipping reference, the The second non-key frame is subjected to inter-frame compression coding to obtain a forward predictive coded frame; based on the complete coded frame and the forward predictive coded frame, several consecutive coding sequences are formed, wherein each code sequence is It is composed of a complete coded frame at the first position of the sequence and a forward predictive coded frame arranged in sequence.
  • the step of performing inter-frame compression coding on the second non-key frame in the form of frame skipping reference to obtain a forward prediction coded frame specifically includes: selecting the second non-key frame to be coded ; According to the playback magnification setting of online playback, determine the previous one or more image frames of the second non-key frame to be encoded as a reference frame; calculate the difference between the second non-key frame to be encoded and the reference frame ; Encoding the difference according to a preset compression algorithm to obtain a forward prediction encoded frame corresponding to the second non-key frame to be encoded.
  • the playback magnification of the online playback is a positive integer between 1 and K; then, the previous one or more images of the second non-key frame to be encoded are determined according to the playback magnification of the online playback
  • the step of the frame being a reference frame specifically includes: determining that the first 2k image frames of the second non-key frame to be encoded are reference frames, and k is a positive integer from 0 to K; the k is calculated by the following formula:
  • the time axis of the first output image and the second output image are synchronized, and have the same start time and end time.
  • the third output image is a third image frame sequence with a third resolution, and the third resolution is lower than the original resolution; then,
  • the step of performing a third code stream encoding on the third output image to form encoded data supporting remote live broadcast specifically includes: determining at least one third key frame in the third image frame sequence, wherein , The image frame between the two third key frames is the third non-key frame; intra-frame compression coding is performed on the third key frame to obtain a complete coded frame; and in the form of continuous reference, the third key frame is Perform inter-frame compression coding on non-key frames to obtain forward predictive coding frames;
  • each segment of coded sequence is composed of a complete coded frame at the first position of the sequence and a forward predictive coded frame arranged in sequence;
  • the macroblocks of the complete coded frame of each coded sequence are scattered to one or more of the forward predictive coded frames in the coded sequence.
  • the embodiments of the present invention further provide the following technical solutions: an image encoder.
  • the image encoder includes a processor and a memory communicatively connected with the processor;
  • the memory stores computer program instructions, and the computer program instructions are called by the processor to make the processor execute the encoding method as described above to form at least two channels of encoded data to support at least two business requirements.
  • an image transmission system includes:
  • the image encoder is used to execute the encoding method described above to form at least two corresponding encoded data; the image decoder is used to receive part or all of the encoded data and decode it according to the operation instruction; the communication module, The communication module is connected with the image encoder and the image decoder, and is used to form a communication channel to transmit the encoded data.
  • the image encoder includes: a first code stream encoding unit, a second code stream encoding unit, and a third code stream encoding unit; the first code stream encoding unit is used to perform the first code stream encoding to form Support the first coded data for video recording; the second code stream encoding unit is used to encode the second code stream to form second coded data that supports online playback; the third code stream encoding unit is used to perform the third code stream Encoding to form the third encoded data supporting remote live broadcast.
  • the image transmission system further includes: a memory connected to the image encoder and configured to store the first code stream encoded data and the second code stream encoded data.
  • the image decoder when performing online playback, is further configured to: according to the current playback magnification, in receiving the second encoded data, the complete encoded frame of the key frame and the image frame sequence number and the remainder of the division of 2k are The forward prediction coding frame corresponding to the non-key frame of 0, k is the current playback magnification.
  • the image decoder when performing online playback, is further configured to: when the current playback magnification is greater than K, receive all complete encoded frames in the second encoded data and discard all forward predictive encoded frames .
  • the image decoder when performing online playback, is further configured to: in the second encoded data, search for the key frame closest to the designated playback time and use the closest key frame as the starting point, Receive the subsequent coding sequence.
  • the encoding method provided in the embodiments of the present invention uses a multi-stream encoding method, adopts different encoding strategies for different business needs, can be compatible with different business needs at the same time, and make business realization smoother.
  • Figure 1 is a schematic diagram of an application environment of an embodiment of the present invention.
  • FIG. 2 is a structural block diagram of an image transmission system provided by an embodiment of the present invention.
  • Fig. 3 is a method flowchart of an encoding method provided by an embodiment of the present invention.
  • FIG. 4 is a method flowchart of a first encoding method provided by an embodiment of the present invention.
  • Figure 5 is a schematic diagram of first encoded data provided by an embodiment of the present invention.
  • FIG. 6 is a method flowchart of a second encoding method provided by an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of second encoded data provided by an embodiment of the present invention.
  • FIG. 8 is a method flowchart of a third encoding method provided by an embodiment of the present invention.
  • Fig. 9 is a schematic diagram of third coded data provided by an embodiment of the present invention.
  • Business requirements refer to functional applications that use cameras or image capture devices to collect image data, which are realized through data stream transmission, and are used to meet user needs. In the process of using a camera to shoot images to achieve one or more business requirements, there will be corresponding requirements for image compression coding and compressed data transmission for the characteristics of different business requirements.
  • the code stream of the encoded data is required to be stable without major fluctuations; when performing video review, the form of the encoded data is required to support functions such as progress bar drag, fast forward, or fast forward.
  • Figure 1 is an application scenario for realizing business requirements provided by an embodiment of the present invention.
  • the application scenario taking an aerial camera carried by a drone as an example, as shown in FIG. 1, the application scenario includes a drone 10 equipped with an aerial camera, a smart terminal 20 and a wireless network 30.
  • the UAV 10 may be an unmanned aerial vehicle driven by any type of power, including but not limited to a four-axis UAV, a fixed-wing aircraft, and a helicopter model. It can have the corresponding volume or power according to the needs of the actual situation, so as to provide the load capacity, flight speed and flight range that can meet the needs of use.
  • the aerial camera can be any type of image acquisition device, including a sports camera, a high-definition camera, or a wide-angle camera.
  • its aerial camera can be installed and fixed on the drone through a fixed bracket such as a pan/tilt, and controlled by the drone 10 to perform image collection tasks.
  • one or more functional modules can be added to the drone to enable the drone to achieve corresponding functions, such as the built-in main control chip, which is used as the core of the drone flight and data transmission control or the image transmission device , Upload the collected image information to the equipment connected with the drone.
  • the smart terminal 20 may be any type of smart device used to establish a communication connection with the drone, such as a mobile phone, a tablet computer, or a smart remote control.
  • the smart terminal 20 may be equipped with one or more different user interaction devices to collect user instructions or display and feedback information to the user.
  • buttons, display screens, touch screens, speakers, and remote control joysticks are included in the smart terminal 20 .
  • the smart terminal 20 may be equipped with a touch screen, through which the user’s remote control instructions for the drone are received and the image information obtained by the aerial camera is displayed to the user through the touch screen. The user can also use the remote control The touch screen switches the image information currently displayed on the display.
  • the wireless network 30 may be a wireless communication network based on any type of data transmission principle for establishing a data transmission channel between two nodes, such as a Bluetooth network, a WiFi network, a wireless cellular network, or a combination thereof located in a specific signal frequency band.
  • FIG. 2 is a structural block diagram of an image transmission system provided by an embodiment of the present invention. As shown in FIG. 2, the image transmission system includes: an image encoder 11, an image decoder 21, and opposite communication modules (12, 22).
  • the image encoder 11 is a functional module installed on the side of the drone 10. It can execute the encoding method provided in the embodiment of the present invention to form multiple channels of encoded data, which are respectively used to support different service requirements.
  • Fig. 3 is a method flowchart of an encoding method provided by an embodiment of the present invention. As shown in Figure 3, the method includes:
  • Each output image can be represented in the form of a sequence of continuous image frames.
  • the output images are independent of each other and can have different image parameters or properties. For example, two output images may have different resolutions or a certain image may be a gray value image, depending on the actual processing process performed by the image encoder 11.
  • Multi-stream encoding refers to the use of different encoding strategies or encoding methods for different output images, which ultimately form multiple different encoding data encoding methods. Based on the fact that each channel of coded data has a different code stream to meet the characteristics of different service requirements, the term “multi-stream coding” is used here to refer to the aforementioned coding method that uses multiple coding strategies at the same time.
  • the technician can choose to use the corresponding software, hardware, or a combination of software and hardware to implement one or more image processing (such as noise reduction, smoothing). Filtering or cropping, etc.) and coding steps.
  • image processing such as noise reduction, smoothing. Filtering or cropping, etc.
  • the image encoder 11 can be realized by a conventionally used processor and a memory communicating with the processor. Technicians can write computer program instructions, and then burn and store them in the memory. When the computer program instructions are called by the processor, the processor executes corresponding multi-stream encoding to form at least two channels of encoded data to support at least two service requirements.
  • the communication module 12 is a functional module set on the drone 10 and adapted to the wireless network 30.
  • the UAV 10 can be added to the wireless network 30 through the communication module 12, as one of the nodes of the network, sending or receiving data via the communication channel provided by the wireless network 30.
  • the image decoder 21 is configured to receive and decode part or all of the encoded data according to the operation instruction. Similar to the image encoder 11, the image decoder 21 can also be implemented using corresponding software, hardware, or a combination of software and hardware.
  • Operation instructions refer to instructions input by the user on the smart terminal 20 that are related to the realization of business requirements, used to select functions or control adjustable parameters. Based on different operating instructions, the image decoder 21 may request to receive or download the corresponding encoded data, and use a decoding algorithm adapted to the encoded data to obtain the required image data.
  • the communication module 22 is a functional module installed opposite to the communication module 12.
  • the smart terminal 20 can also be added to the wireless network 30 through the communication module 22 as another node of the wireless network. Based on the communication channel established between the communication modules (12, 22), the image encoder 11 and the image decoder 21 realize data interaction, and perform the issuance of encoded data or the upload of operation instructions.
  • the drone 10 can implement a video recording function.
  • the video recording function refers to storing the image data collected by the aerial camera in one or more storage devices so that it can be read or copied to other terminal devices for playback.
  • the specific implementation process is roughly to first compress and encode the image information captured by the drone 10 through the aerial camera. Then the compressed and encoded data is stored in a large-capacity storage device (such as an SD card) of the drone to form a video file.
  • a large-capacity storage device such as an SD card
  • online playback can also be implemented between the drone 10 and the smart terminal 20.
  • Online playback refers to the function of the user on the smart terminal 20 to quickly browse the video files previously shot or collected by the drone 10 to roughly understand the content of the video files.
  • the implementation process is roughly as follows: First, the image information collected and captured by the drone 10 through the aerial camera is compressed, encoded and saved to form a video file. Then, the smart terminal 20 can obtain these video files online through the wireless communication network and display them to the user.
  • the online playback function needs to support drag, fast forward or fast playback.
  • remote live broadcast can be implemented between the drone 10 and the smart terminal 20.
  • the remote live broadcast refers to the function of real-time broadcast of the images captured by the aerial camera on the smart terminal 20 through the wireless transmission of the communication module.
  • the specific implementation process is as follows: First, the image information collected by the drone 10 through the aerial camera is compressed and encoded, and then provided to the smart terminal 20 via wireless communication. Then, after being decoded by the smart terminal 20, it is displayed to the user in real-time synchronously through the display screen to realize the function of remote live broadcast.
  • the image encoder 11 may at least include a first code stream encoding unit 111, a second code stream encoding unit 112, and a third Code stream encoding unit 113.
  • the first code stream encoding unit 111 is configured to perform first code stream encoding to form first coded data supporting video recording.
  • the second code stream encoding unit 112 is used to perform second code stream encoding to form second coded data supporting online playback.
  • the third code stream encoding unit 113 is configured to perform third code stream encoding to form third coded data supporting remote live broadcast.
  • the image encoder 11 adopts different code stream coding units to perform targeted coding according to different functional requirements and the characteristics of their applications. Through the form of multi-stream coding, all functional requirements can be met. Being taken into account, the code stream transmission is smoother.
  • the image transmission system may further include a storage device 13.
  • the storage device 13 is connected to the image encoder 11 and is used to store the first code stream coded data and the second code stream coded data. It can send at least a part of the coded data to the smart terminal 20 for display through the communication module when the user issues a corresponding operation instruction through the smart terminal 20.
  • the storage device 13 may be any suitable type of non-volatile storage device for providing sufficient storage space to store all the original data obtained by collection, including but not limited to SD card, SSD hard disk, mechanical hard disk or flash memory. In this embodiment, the storage device 13 adopts an SD card.
  • the storage device 13 may also select an appropriate number according to actual needs, for example, it may be set to two, which are used for storing the first code stream encoded data and the second code stream encoded data respectively.
  • the storage device 13 can also directly use storage space or storage devices provided by other systems in the drone, as long as it can meet the requirements of data storage capacity and data read and write speed.
  • the application environment shown in Figure 1 only shows the application of the image transmission system on the UAV.
  • the image transmission system can also be mounted on other types of mobile vehicles (such as remote control cars) to perform the same functions.
  • the inventive idea about the image transmission system disclosed in the embodiment of the present invention is not limited to being applied to the unmanned aerial vehicle shown in FIG. 1.
  • the following describes the first code stream encoding unit 111, the second code stream encoding unit 112, and the third code stream encoding unit 113 to perform the first code stream encoding, the second code stream encoding unit 113 and the data transmission characteristics of remote live broadcast, video recording, and video playback.
  • the specific process of code stream encoding and the third code stream encoding is described.
  • FIG. 4 is a flowchart of a method for encoding a first code stream provided by an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a data structure of the first coded data.
  • the output image used for the first code stream encoding is the first image frame sequence with original resolution and very high definition. As shown in Figure 4, the method includes the following steps:
  • the first key frame refers to an image frame serving as an anchor point in an image frame sequence, and can be used to correct accumulated errors. It is usually an image frame with a significant change in the image frame in the sequence of image frames.
  • the first non-key frame is the image frame located between these key frames.
  • Intra-frame compression coding refers to image compression and coding only for the spatial domain of the first key frame itself.
  • the obtained complete coded frame I can be decoded independently to recover the corresponding key frame image.
  • the compression rate of such intra-frame compression coding is relatively small.
  • Inter-frame compression coding refers to a method of compression coding based on the difference between two image frames before and after.
  • the finally obtained forward predictive coding frame P only records the difference between the image frame and the previous image frame, and it must rely on the reference image frame to complete the decoding.
  • the coded data obtained by performing inter-frame compression coding on each first non-key frame is called a "forward predictive coded frame”.
  • the first image frame sequence can be organized to form several consecutive coding sequences.
  • each coded sequence is composed of a complete coded frame I at the first position of the sequence and a forward predictive coded frame P arranged in sequence.
  • the above steps can also be adjusted or integrated according to actual conditions, and only need to organize to form the coded data structure shown in FIG. 5.
  • it can be a common IP mode code stream encoded by H264/H265.
  • the video files obtained by encoding in this way can be exported by users to different platforms and players for playback, which has better compatibility and adapts to multiple scenarios.
  • the first bitstream encoding is variable bitrate encoding to ensure the stability of the image quality of the bitstream.
  • the first encoded data adopts the original resolution, which has a higher resolution.
  • the code rate of its code stream is very high, close to the limit write rate of storage devices (such as SD cards). Therefore, the upper limit of the encoding quality can be set for the first code stream encoding to keep the data flow of the first coded data within the set range, and the storage limit of the memory card will not be exceeded due to the sudden abnormality of the code stream.
  • the specific upper limit of encoding quality is an empirical value, which can be set or adjusted by those skilled in the art according to actual conditions.
  • FIG. 6 is a flowchart of a method for encoding a second code stream provided by an embodiment of the present invention
  • FIG. 7 is a schematic diagram of second encoded data provided by an embodiment of the present invention.
  • the second output image used by the second bitstream encoding is a second image frame sequence with a lower resolution (such as 720p) to realize the function of online playback.
  • a lower resolution such as 720p
  • the time axis of the first output image and the second output image are synchronized, and have the same start time and end time.
  • the method includes the following steps:
  • At least one image frame in the second image frame sequence is a second key frame, and an image frame between two second key frames is a second non-key frame.
  • the second image frame sequence is organized into multiple different units by selecting and determining the second key frame during encoding of the second code stream.
  • each non-key frame has the previous image frame as the reference frame.
  • frame skip reference means that the second non-key frame is not The previous image frame is used as the reference frame, but the previous one or several image frames are used as the reference frame.
  • the specific form of frame skipping reference may be defined by the playback magnification of online playback.
  • the specific method flow is:
  • the playback magnification refers to the ratio between the playback speed of the smart terminal 20 and the normal playback speed of the video file during online playback. It is used to provide an adjustment range of the playback speed. Generally, the playback magnification is an integer value, such as 1 times, 2 times, or 3 times.
  • the difference is encoded according to a preset compression algorithm to obtain the forward prediction encoded frame corresponding to the second non-key frame to be encoded.
  • any suitable compression algorithm can be selected to encode the difference between two image frames according to actual needs.
  • each coded sequence consists of a complete coded frame I at the first position of the sequence and a forward predictive coded frame P arranged in sequence, and the reference frame of each forward predictive coded frame is the image frame pointed by the arrow .
  • the encoded data structure shown in FIG. 7 is actually very beneficial to the realization of the playback speed adjustment or the drag operation during online playback.
  • the image decoder determines the designated playback time corresponding to the drag instruction input by the user, it can search for the key frame closest to the designated playback time. Then, taking the closest key frame as the starting point, only the subsequent coding sequence needs to be received, which reduces the amount of data that the communication module needs to transmit.
  • the image decoder can skip several consecutive forward predictive encoded frames according to the playback magnification selected by the user, and only receive part of the forward predictive encoded frames for decoding. It can also reduce the amount of data that the communication module needs to transmit.
  • the following describes in detail how the second encoded data can reduce the network transmission capacity when dragging and adjusting the playback speed with reference to the specific example shown in FIG. 7.
  • one or more image frames can be skipped to achieve online playback with high playback magnification.
  • the playback magnification supported by the online playback function is a positive integer between 1 and K
  • the image frame number of the second key frame is set to 0, and the image frame number of the second non-key frame to be encoded Is n.
  • the corresponding index can be found in the image frame sequence without requesting other redundant second non-key frames.
  • the first 2k image frames of the second non-key frame to be encoded can be determined as reference frames.
  • the value of k is calculated by the following piecewise function:
  • z mod(n, 2K)
  • Fig. 7 is a schematic diagram of a reference frame when the playback magnification K is set to 2. As shown in FIG. 7, for each non-key frame, the image frame pointed by the arrow is the reference frame of the second non-key frame.
  • the second non-key frames can be divided into three categories: the first category is non-key frames with odd serial numbers, which are located at the bottom layer, even if they are discarded, they will not affect the decoding of other frames and can be discarded at will.
  • the second category is non-key frames with sequence numbers 2, 6, 10, etc. (not divisible by 4), which are located in the middle level and will affect the decoding of adjacent frames when they are discarded.
  • the third category is non-key frames with sequence numbers 4 and 8 (divisible by 4), which cannot be discarded.
  • the image decoder can select the complete coded frame I of the received key frame and the image frame number and 2k according to the current playback magnification k. Instead of receiving all the forward predictive coded frames P corresponding to the second non-key frame whose division remainder is 0.
  • the image decoder can only receive the complete coded frame 0 and forward predictive coded frames 2, 4, 6, and 6 in each image frame sequence. 8, 10 and 12, and based on the reference dependency, complete the decoding of these encoded data so that the video file is played at a playback rate that is 1 times greater than the normal playback speed.
  • the image decoder may only receive the complete encoded frame I and discard all forward predictive encoded frames, thereby performing online playback at a faster playback speed.
  • FIG. 8 is a flowchart of a method for encoding a third code stream provided by an embodiment of the present invention
  • FIG. 9 is a schematic diagram of third coded data provided by an embodiment of the present invention. Due to the limitations of the remote live broadcast network traffic, etc., the third output image used by the third code stream encoding is a third image frame sequence with a lower resolution (such as 720p).
  • the third codestream encoding method may include the following steps:
  • Continuous reference means that each third non-key frame uses the previous image frame as a reference frame, which uses the same reference method as the first bitstream encoding shown in FIG. 6.
  • each segment of the coded sequence is composed of a complete coded frame at the first position of the sequence and a forward predictive coded frame arranged in sequence as a constituent unit of the third coded data.
  • the complete coded frame and the forward predictive coded frame adopt different coding methods, and the amount of data or the size of the frame are significantly different.
  • Micro block is a part of data divided from a complete coded frame according to a specific data division criterion. As shown in Figure 9, by dispersing the macroblocks of the complete coded frame I into each forward predictive coded frame P, the size of each frame can be relatively even, and the bit rate during transmission is very stable, which has a relatively large impact on the network. Small, suitable for long-distance wireless communication transmission.
  • the image encoding method provided by the embodiments of the present invention is aimed at recording files that need to be frequently read by users, and is suitable for decoding and playing characteristics in different environments, using the most common IP mode encoding (such as HX265/HX264), And take into account the write capability of the storage device.
  • the frame skipping method is used to determine the reference frame of the second non-key frame, so as to reduce the number of image frames that the image decoder needs to receive and reduce the download of the wireless communication network. burden.
  • the macroblock dispersion method is adopted to disperse the macroblocks of the complete coded frame into each forward prediction frame.
  • the code rate remains stable, reducing the impact on the network.
  • the computer software may be stored in a computer readable storage medium, and when the program is executed, it may include the processes of the above-mentioned method embodiments.
  • the storage medium can be a magnetic disk, an optical disc, a read-only storage memory or a random storage memory, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明实施例涉及一种编码方法、图像编码器及图像传输系统。该编码方法包括：处理原始图像数据，以获得至少两路不同的输出图像；对至少两路的所述输出图像进行多码流编码，以形成对应的至少两路编码数据以支持至少两种业务需求。该编码方法使用多码流编码的方式，针对不同的业务需要采用不同的编码策略，可以同时兼容不同的业务需要，令业务的实现更为顺利。

Description

编码方法、图像编码器及图像传输系统
本申请要求于2019年3月6日提交中国专利局、申请号为201910168160.X、申请名称为“编码方法、图像编码器及图像传输系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种编码方法、图像编码器及图像传输系统。
背景技术
随着电子技术的不断进步，相机的性能越来越强大，能够采集的图像质量和图像分辨率等也越来越高。由此，相机采集得到的录像文件或者图片的数据量也越来越大。
按照相机的用途或者应用领域的不同，基于这些图像数据会存在许多不同的业务需求。例如，在航拍无人机的传输过程中，需要实现地面终端远距离实时直播视频，远距离控制相机录制4K甚至更高分辨率视频到存储卡以及地面终端远程回放相机中录像文件，支持拖拽、快放等的业务需求。
较大的数据量为实现业务需求带来了非常严峻的挑战，尤其是传输带宽限制较大的无线传输而言。因此，通常需要提供合适的图像编码方法，对图像数据进行编码压缩，以降低所需要占用的带宽资源。
但是不同的业务需求而言，都具有不同的使用要求特点和网络传输要求。如何提升图像编码方法的性能，使其能够兼容多样性的业务需求是迫切需要解决的技术问题。
发明内容
为了解决上述技术问题，本发明实施例提供一种能够满足多样性的业务需求的编码方法、图像编码器及图像传输系统。
为解决上述技术问题，本发明实施例提供以下技术方案：一种编码方法，用于航拍相机。
该编码方法包括：处理原始图像数据，以获得至少两路不同的输出图像；对至少两路的所述输出图像进行多码流编码，以形成对应的至少两路编码数据以支持至少两种业务需求。
或者，该编码方法包括：处理原始图像数据，以获得至少两路输出图像；对所述至少两路输出图像中每路输出图像进行一种码流编码，以形成至少两路编码数据；其中，不同路的输出图像对应不同的码流；所述至少两路编码数据 用于支持至少两种业务需求，所述至少两路编码数据中的每路编码数据用于支持所述至少两种业务需求中的一种业务需求。
可选地，所述业务需求包括录像、在线回放以及远程直播；所述至少两路输出图像包括：第一输出图像，第二输出图像以及第三输出图像；则，
所述对至少两路的所述输出图像进行多码流编码，形成对应的至少两路编码数据以支持至少两种图像操作的步骤，具体包括：对所述第一输出图像进行第一码流编码，以形成支持录像的第一编码数据；对所述第二输出图像进行第二码流编码，以形成支持在线回放的第二编码数据；对所述第三输出图像进行第三码流编码，以形成支持远程直播的第三编码数据。
可选地，所述第一输出图像为具有原始分辨率的第一图像帧序列；则，
所述对所述第一输出图像进行第一码流编码，形成支持录像的第一编码数据的步骤，具体包括：确定在所述第一图像帧序列中的至少一个图像帧为第一关键帧，其中，两个第一关键帧之间的图像帧为第一非关键帧；对所述第一关键帧进行帧内压缩编码，以获得完整编码帧；以及参考在前的图像帧，对所述非关键帧进行帧间压缩编码，以获得前向预测编码帧；基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成。
可选地，所述方法还包括：为所述第一码流编码设置编码质量上限，以使所述第一编码数据的数据流量保持在设定范围内。
可选地，所述第一码流编码为可变码率编码。
可选地，所述第二路输出图像为具有第二分辨率的第二图像帧序列，其中，所述第二分辨率低于原始分辨率；则，
所述对所述第二路输出图像进行第二码流编码，以形成支持在线回放的第二编码数据的步骤，具体包括：在所述第二图像帧序列中的至少一个图像帧为第二关键帧，两个第二关键帧之间的图像帧为第二非关键帧；对所述第二关键帧进行帧内压缩编码，以获得完整编码帧；并且以跳帧参考的形式，对所述第二非关键帧进行帧间压缩编码，以获得前向预测编码帧；基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成。
可选地，所述以跳帧参考的形式，对所述第二非关键帧进行帧间压缩编码，以获得前向预测编码帧的步骤，具体包括：选定待编码的第二非关键帧；根据在线回放的播放倍率设置，确定所述待编码的第二非关键帧的前一个或者多个图像帧为参考帧；计算所述待编码的第二非关键帧与参考帧之间的差别；对所述差别，按照预设的压缩算法进行编码，以获得所述待编码的第二非关键帧对应的前向预测编码帧。
可选地，所述在线回放的播放倍率为1至K之间的正整数；则，所述根据在线回放的播放倍率，确定所述待编码的第二非关键帧的前一个或者多个图像帧为参考帧的步骤，具体包括：确定所述待编码的第二非关键帧的前2k个图 像帧为参考帧，k为0至K的正整数；所述k通过如下算式计算：
k＝0    mod(n,2)＝1时；
k＝K    mod(n,2K)＝0时；
k＝z/2    mod(n,2)＝0时；
其中，第二关键帧的图像帧序号设置为0，待编码的第二非关键帧的图像帧序号为n，z＝mod(n,2K)。
可选地，所述第一输出图像与第二输出图像的时间轴同步，具有相同的起始时刻和终止时刻。
可选地，所述第三路输出图像为具有第三分辨率的第三图像帧序列，所述第三分辨率低于原始分辨率；则，
所述对所述第三路输出图像进行第三码流编码，以形成支持远程直播的编码数据的步骤，具体包括：在所述第三图像帧序列中，确定至少一个第三关键帧，其中，两个第三关键帧之间的图像帧为第三非关键帧；对所述第三关键帧进行帧内压缩编码，以获得完整编码帧；并且以连续参考的形式，对所述第三非关键帧进行帧间压缩编码，以获得前向预测编码帧；
基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成；
将每段编码序列的完整编码帧的宏块分散到所在编码序列中的一个或者多个所述前向预测编码帧。
为解决上述技术问题，本发明实施例还提供以下技术方案：一种图像编码器。所述图像编码器包括：处理器以及与所述处理器通信连接的存储器；
所述存储器存储有计算机程序指令，所述计算机程序指令被所述处理器调用以使所述处理器执行如上所述的编码方法，形成至少两路编码数据以支持至少两种业务需求。
为解决上述技术问题，本发明实施例还提供以下技术方案：一种图像传输系统。所述图像传输系统包括：
图像编码器，用于执行如上所述的编码方法，形成对应的至少两路编码数据；图像解码器，用于根据操作指令，接收部分或者全部所述编码数据并进行解码；通信模组，所述通信模组与所述图像编码器和图像解码器连接，用于形成通信信道以传输所述编码数据。
可选地，所述图像编码器包括：第一码流编码单元、第二码流编码单元以及第三码流编码单元；所述第一码流编码单元用于进行第一码流编码，形成支持录像的第一编码数据；所述第二码流编码单元用于进行第二码流编码，形成支持在线回放的第二编码数据；所述第三码流编码单元用于进行第三码流编码，形成支持远程直播的第三编码数据。
可选地，所述图像传输系统还包括：存储器，所述存储器与所述图像编码器连接，用于存储所述第一码流编码数据和第二码流编码数据。
可选地，在执行在线回放时，所述图像解码器还用于：根据当前的播放倍率，接收所述第二编码数据中，关键帧的完整编码帧以及图像帧序号与2k相除余数为0的非关键帧对应的前向预测编码帧，k为当前的播放倍率。
可选地，在执行在线回放时，所述图像解码器还用于：在当前的播放倍率大于K时，接收所述第二编码数据中所有的完整编码帧并丢弃所有的前向预测编码帧。
可选地，在执行在线回放时，所述图像解码器还用于：在所述第二编码数据中，搜索与指定播放时刻最接近的关键帧并且以所述最接近的关键帧为起点，接收后续的编码序列。
与现有技术相比较，本发明实施例的提供编码方法使用多码流编码的方式，针对不同的业务需要采用不同的编码策略，可以同时兼容不同的业务需要，令业务的实现更为顺利。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1为本发明实施例的应用环境示意图；
图2为本发明实施例提供的图像传输系统的结构框图；
图3为本发明实施例提供的编码方法的方法流程图；
图4为本发明实施例提供第一编码方法的方法流程图；
图5为本发明实施例提供的第一编码数据的示意图；
图6为本发明实施例提供的第二编码方法的方法流程图；
图7为本发明实施例提供的第二编码数据的示意图；
图8为本发明实施例提供的第三编码方法的方法流程图；
图9为本发明实施例提供的第三编码数据的示意图。
具体实施方式
为了便于理解本发明，下面结合附图和具体实施例，对本发明进行更详细的说明。需要说明的是，当元件被表述“固定于”另一个元件，它可以直接在另一个元件上、或者其间可以存在一个或多个居中的元件。当一个元件被表述“连接”另一个元件，它可以是直接连接到另一个元件、或者其间可以存在一个或多个居中的元件。本说明书所使用的术语“上”、“下”、“内”、“外”、“底部”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语“第一”、“第二”“第三”等仅用于描述目的，而不能理解为指示或暗示 相对重要性。
除非另有定义，本说明书所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本说明书中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是用于限制本发明。本说明书所使用的术语“和/或”包括一个或多个相关的所列项目的任意的和所有的组合。
此外，下面所描述的本发明不同实施例中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
“业务需求”是指利用相机或者图像采集设备采集图像数据，通过数据流传输实现，用于满足用户使用需要的功能应用。而在利用相机拍摄图像，实现一种或者多种业务需求的过程中，对于不同的业务需求的特点，对于图像压缩编码以及压缩数据的传输会有相应的要求。
例如，进行远程直播时，要求编码数据的码流平稳，不发生大的波动；进行录像回顾时，则需要编码数据形式支持进度条拖拽、快进或者快放等功能。
图1为本发明实施例提供的实现业务需求的应用场景。在本应用场景中，以无人机搭载的航拍相机为例，如图1所示，所述应用场景包括搭载了航拍相机的无人机10、智能终端20以及无线网络30。
无人机10可以是以任何类型的动力驱动的无人飞行载具，包括但不限于四轴无人机、固定翼飞行器以及直升机模型等。其可以根据实际情况的需要，具备相应的体积或者动力，从而提供能够满足使用需要的载重能力、飞行速度以及飞行续航里程等。
航拍相机可以是任何类型的图像采集设备，包括运动相机、高清相机或者广角相机。其航拍相机作为无人机上搭载的其中一种功能模块，可以通过云台等安装固定支架，安装固定在无人机上，并受控于无人机10，执行图像采集的任务。
当然，无人机上还可以添加有一种或者多种功能模块，令无人机能够实现相应的功能，例如内置的主控芯片，作为无人机飞行和数据传输等的控制核心或者是图传装置，将采集获得的图像信息上传至与无人机建立连接的设备中。
智能终端20可以是任何类型，用以与无人机建立通信连接的智能设备，例如手机、平板电脑或者智能遥控器等。该智能终端20可以装配有一种或者多种不同的用户交互装置，用以采集用户指令或者向用户展示和反馈信息。
这些交互装置包括但不限于：按键、显示屏、触摸屏、扬声器以及遥控操作杆。例如，智能终端20可以装配有触控显示屏，通过该触控显示屏接收用户对无人机的遥控指令并通过触控显示屏向用户展示由航拍相机获得的图像信息，用户还可以通过遥控触摸屏切换显示屏当前显示的图像信息。
无线网络30可以是基于任何类型的数据传输原理，用于建立两个节点之间的数据传输信道的无线通信网络，例如位于特定信号频段的蓝牙网络、WiFi网络、无线蜂窝网络或者其结合。
无人机10与智能终端20之间可以通过专用的图像传输系统(也可以被简称为“图传”)来实现图像数据传输。图2为本发明实施例提供的图像传输系统的结构框图。如图2所示，该图像传输系统包括：图像编码器11，图像解码器21以及对向设置的通信模组(12，22)。
其中，图像编码器11是设置在无人机10一侧的功能模块。其可以执行本发明实施例提供的编码方法以形成多路编码数据，分别用于支持不同的业务需求。图3为本发明实施例提供的编码方法的方法流程图。如图3所示，该方法包括：
310、处理航拍相机采集获得的原始图像数据，获得至少两路不同的输出图像。
每一路输出图像可以以连续图像帧序列的形式表示。输出图像之间相互独立，可以具有不同的图像参数或者性质。例如，两路输出图像之间可以具有不同的分辨率或者某一路图像可以是灰度值图像，具体取决于图像编码器11实际执行的处理过程。
320、以多码流编码的方式对所述至少两路输出图像进行编码，形成对应的多路编码数据。
“多码流编码”是指对不同的输出图像使用不同的编码策略或者编码方式，最终形成多路不同的编码数据的编码方式。基于每路编码数据具有不同的码流情况，以满足不同的业务需求的特点，在此使用“多码流编码”这样的术语表示上述同时使用多种编码策略的编码方式。
例如，对所述至少两路输出图像中每路输出图像进行一种码流编码，以形成至少两路编码数据；其中，不同路的输出图像对应不同的码流；所述至少两路编码数据用于支持至少两种业务需求，所述至少两路编码数据中的每路编码数据用于支持所述至少两种业务需求中的一种业务需求。
基于根据本实施例中揭露的图像编码器11需要执行的功能步骤，技术人员可以选择使用对应的软件、硬件或者软硬件结合的方式来实现，执行一个或者多个图像处理(例如降噪、平滑滤波或者裁切等)和编码步骤。
例如，该图像编码器11可以通过惯常使用的处理器以及与所述处理器通信连接的存储器这样的结构形式来实现。技术人员可以编写计算机程序指令，然后烧录、存储在存储器中。当所述计算机程序指令被所述处理器调用时，以使处理器执行对应的多码流编码以形成至少两路编码数据以支持至少两种业务需求。
通信模组12是设置在无人机10上，与无线网络30相适配的功能模组。无人机10可以通过通信模组12加入到无线网络30中，作为网络的其中一个节点，经由无线网络30提供的通信信道发送或者接收数据。
图像解码器21用于根据操作指令，接收部分或者全部所述编码数据并进行解码。与图像编码器11相类似地，该图像解码器21也可以选择使用对应的软件、硬件或者软硬件结合的方式来实现。
操作指令是指用户在智能终端20上输入的，与实现业务需求相关的指令，用以选择功能或者控制可调参数等。基于不同的操作指令，图像解码器21可以请求接收或者下载对应的编码数据，并使用与编码数据相适配的解码算法以获得所需要的图像数据。
通信模组22是与通信模组12对向设置的功能模组。智能终端20也可以通过通信模组22加入到无线网络30中，作为无线网络的另一个节点。基于通信模组(12，22)之间建立的通信信道，图像编码器11与图像解码器21实现数据交互，进行编码数据的下发或者操作指令的上传等。
基于以上的图像传输系统，在该应用场景中可以提供或者实现许多不同的功能应用，满足对应的业务需求。
例如，无人机10可以实现录像功能。录像功能是指将航拍相机采集到的图像数据存储在一个或者多个存储设备中，使其可以被读取或者拷贝到其它的终端设备上播放。其具体的实现过程大致是先对无人机10通过航拍相机采集拍摄的图像信息进行压缩编码。然后将压缩编码后的数据存储在无人机的大容量存储设备(如SD卡)中，形成录像文件。
在一些实施例中，无人机10与智能终端20之间还可以实现在线回放。
在线回放是指用户在智能终端20上，对无人机10在先拍摄或者采集的录像文件进行快速浏览以大致了解已经录像文件内容的功能。其实现过程大致如下：首先，对无人机10通过航拍相机采集拍摄的图像信息进行压缩编码并保存，形成录像文件。然后，智能终端20可以通过无线通信网络，在线获取这些录像文件并向用户展示。在线回放功能中，需要支持拖拽、快进或者快速回放的功能。
在另一些实施例中，无人机10与智能终端20之间可以实现远程直播。远程直播是指在智能终端20上，通过通信模组的无线传输，将航拍相机拍摄获得的图像实时播放的功能。其具体实现的过程如下：首先，无人机10通过航拍相机采集拍摄的图像信息经过压缩编码以后，经由无线通信方式，提供至智能终端20。然后，由智能终端20进行解码以后，通过显示屏向用户实时同步展示从而实现远程直播的功能。
与以上三种功能需求(远程直播、录像以及在线回放)相对应的，请继续参阅图2，图像编码器11至少可以包括第一码流编码单元111、第二码流编码单元112以及第三码流编码单元113。
其中，所述第一码流编码单元111用于进行第一码流编码，形成支持录像的第一编码数据。所述第二码流编码单元112则用于进行第二码流编码，形成支持在线回放的第二编码数据。所述第三码流编码单元113用于进行第三码流编码，形成支持远程直播的第三编码数据。
在本实施例中，图像编码器11针对不同的功能需要及其应用的特点，分别采用不同的码流编码单元分别进行针对性的编码，通过多码流编码的形式使得各项功能需要均能够被兼顾，码流传输更为顺畅。
请继续参阅图2，在实现录像和录像回放的功能时，所述图像传输系统还可以包括存储设备13。
该存储设备13与所述图像编码器11连接，用于存储所述第一码流编码数据和第二码流编码数据。其可以在用户通过智能终端20发出相应的操作指令时，通过通信模块将至少一部分的编码数据发送至智能终端20中进行展示。
存储设备13可以是任何合适类型的非易失性的存储设备，用于提供足够的存储空间以保存采集获得的所有原始数据，包括但不限于SD卡、SSD硬盘、机械硬盘或者闪存等。在本实施例中，该存储设备13采用SD卡。
具体的，存储设备13还可以根据实际需要选择合适的数量，例如可以设置为两个，分别用于第一码流编码数据和第二码流编码数据的存储。当然，存储设备13也可以直接使用位于无人机中的其它系统提供的存储空间或者存储设备，只需要能够满足数据存储容量和数据读写速度的要求即可。
图1所示的应用环境仅显示了图像传输系统在无人机上的应用。本领域技术人员可以理解的是，该图像传输系统还可以被搭载在其它类型的移动载具(如遥控车)上，执行相同的功能。本发明实施例公开的关于图像传输系统的发明思路并不限于在图1所示的无人机上应用。
以下结合远程直播、录像以及录像回放等功能的数据传输特点，分别描述第一码流编码单元111、第二码流编码单元112以及第三码流编码单元113进行第一码流编码、第二码流编码以及第三码流编码的具体过程。
图4为本发明实施例提供的第一码流编码的方法流程图，图5为第一编码数据的数据结构示意图。第一码流编码使用的输出图像为具有原始分辨率，清晰度非常高的第一图像帧序列。如图4所示，该方法包括如下步骤：
410、确定在所述第一图像帧序列中的至少一个图像帧为第一关键帧，两个第一关键帧之间的图像帧为第一非关键帧。
在一段连续的图像帧序列中，由于采样率高等的原因。因此，序列中大部分的图像帧之间是非常相似的。许多的编码形式利用这样的特性，通过关键帧和非关键的选取和压缩来尽可能的压缩图像帧数据(如H264/H265编码)。
其中，第一关键帧是指在图像帧序列中作为锚点的图像帧，可以用于校正累积误差。其通常是图像帧序列中，图像画面出现显著变化的图像帧。第一非关键帧则是位于这些关键帧之间的图像帧。
420、对所述第一关键帧进行帧内压缩编码，以获得完整编码帧。
帧内压缩编码是指仅针对第一关键帧自身的空间域进行图像压缩和编码。其获得的完整编码帧I可以被独立解码而恢复出相应的关键帧图像。这样的帧内压缩编码的压缩率是相对较小的。
430、参考在所述第一非关键帧之前的图像帧，对所述第一非关键帧进行帧间压缩编码，以获得前向预测编码帧。
帧间压缩编码是指根据前后两个图像帧之间的差别进行压缩编码的方式。最终获得前向预测编码帧P只记录了图像帧与前一个图像帧之间的差别，其必 须依赖于参考的图像帧才能够完成解码。在本实施例中，将每一个第一非关键帧进行帧间压缩编码获得的编码数据称为“前向预测编码帧”。
440、基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列。
如图5所示，通过步骤410至430，第一图像帧序列可以被组织形成若干个连续的编码序列。其中，每段编码序列由位于序列首位的完整编码帧I和依次排列的前向预测编码帧P组成。
当然，上述步骤也可以根据实际情况进行调整或者整合，只需要组织形成图5所示的编码数据结构即可。例如，可以是H264/H265编码的普通IP模式的码流。
以这样编码方式获得的录像文件可以被用户导出到不同的平台和播放器进行播放，具有更好的兼容性并适应于多个场景。较佳的是，所述第一码流编码为可变码率编码，以保证码流图像质量的稳定性。
在一些实施例中，该第一编码数据采用的是原始分辨率，分辨率较高。其码流的码率非常高，接近于存储设备(如SD卡)的极限写入速率。因此，可以为所述第一码流编码设置编码质量上限以使所述第一编码数据的数据流量保持在设定范围内，不会因码流突发异常而超出存储卡的存储极限。该具体的编码质量上限是一个经验性数值，可以由本领域技术人员根据实际情况而设置或者进行调整。
图6为本发明实施例提供的第二码流编码的方法流程图，图7为本发明实施例提供的第二编码数据的示意图。第二码流编码使用的第二输出图像为分辨率较低(如720p)的第二图像帧序列，用以实现在线回放的功能。在一些实施例中，为了在线回放功能可以与录像数据保持对齐和一致，所述第一输出图像与第二输出图像的时间轴同步，具有相同的起始时刻和终止时刻。
如图6所示，该方法包括如下步骤：
610、在所述第二图像帧序列中的至少一个图像帧为第二关键帧，两个第二关键帧之间的图像帧为第二非关键帧。
与第一码流编码的方式相类似地，在第二码流编码时同样也通过选择确定第二关键帧的方式来将第二图像帧序列组织为多个不同的单元。
620、对所述第二关键帧进行帧内压缩编码，以获得完整编码帧。
630、以跳帧参考的形式，对所述第二非关键帧进行帧间压缩编码，获得前向预测编码帧。
进行帧间压缩编码时，针对的数据是当前第二非关键帧与在前的参考帧之间的差别。而与第一码流编码方式所采用的，每一个非关键帧都已前一个图像帧作为参考帧所不同的是，如图7所示，“跳帧参考”是指第二非关键帧不是以在前的图像帧作为参考帧，而是以在前的一个或几个图像帧作为参考帧。
在一些实施例中，跳帧参考的具体形式可以由在线回放的播放倍率所定义。其具体的方法流程为：
首先，选定待编码的第二非关键帧。然后，根据在线回放的播放倍率设置，确定所述待编码的第二非关键帧的前一个或者多个图像帧为参考帧并计算所述待编码的第二非关键帧与参考帧之间的差别。
播放倍率是指智能终端20在进行在线回放时，播放速度与与录像文件的正常播放速度之间的比例。其用于提供一个播放速度的调整范围。一般的，播放倍率是整数值，例如1倍、2倍或者3倍等。
最后，对所述差别，按照预设的压缩算法进行编码，获得所述待编码的第二非关键帧对应的前向预测编码帧。
具体可以根据实际情况的需要选用任何合适的压缩算法对两个图像帧之间的差别进行编码。
640、基于所述完整编码帧和所述前向预测编码帧，以形成若干段连续的编码序列。
其中，如图7所示，每段编码序列由位于序列首位的完整编码帧I和依次排列的前向预测编码帧P组成，而每个前向预测编码帧的参考帧为箭头指向的图像帧。
由于图像解码器在对每个前向预测编码帧P进行解码时，必须依赖于其参考帧，而只有完整编码帧I的解码不需要依赖其它的图像帧。因此，如图7所示的编码数据结构实际上非常有利于在线回放时的播放速度调整或者拖拽操作的实现。
例如，在用户在智能终端20上进行拖拽操作时，图像解码器确定与用户输入的拖拽指令对应的指定播放时刻后，可以搜索与指定播放时刻最接近的关键帧。然后，以所述最接近的关键帧为起点，只需要接收后续的编码序列即可，减少了通信模组需要传输的数据量。
而当用户需要加快回放的速度时，图像解码器可以根据用户选定的播放倍率，跳过中间连续的数个前向预测编码帧，仅接收部分的前向预测编码帧进行解码即可，同样也可以减少通信模组需要传输的数据量。
以下结合图7所示的具体实例，详细的描述第二编码数据如何能够在拖拽和调整播放速度时，降低网络传输容量的过程。
对于正常播放速度的图像帧序列，可以通过跳过一个或者多个图像帧的形式来实现高播放倍率的在线回放功能。假设在线回放功能支持的播放倍率为1至K之间的正整数，在每一个图像帧序列中，第二关键帧的图像帧序号设置为0，待编码的第二非关键帧的图像帧序号为n。
为了满足不同的播放倍率都能够在图像帧序列中找到相应的索引，不需要请求其它多余的第二非关键帧，可以确定待编码的第二非关键帧的前2k个图像帧为参考帧，k的值通过如下的分段函数计算：
k＝0    mod(n,2)＝1时；
k＝K    mod(n,2K)＝0时；
k＝z/2    mod(n,2)＝0时。
其中，mod是求余函数，用于计算两者相除的余数，z＝mod(n,2K)。通过这样的参考帧确定方式，实际上依据参考依赖关系，将所有的第二非关键帧分为了K+1类。
图7为播放倍率K取2时的参考帧示意图。如图7所示，对于每个非关键帧，其箭头指向的图像帧为该第二非关键帧的参考帧。
相应地，第二非关键帧可以被分为三类：第一类是序号为奇数的非关键帧，其位于最底层，即使抛弃也不会对其它帧的解码造成影响，可以被随意抛弃。第二类是序号为2、6、10等(不能被4整除)的非关键帧，其位于中间的层级，被丢弃时会影响到相邻帧的解码。第三类是序号为4和8(能够被4整除)的非关键帧，其不能被丢弃。
基于上述“跳帧参考”方式形成的第二编码数据，在智能终端20执行在线回放时，图像解码器可以根据当前的播放倍率k，选择接收关键帧的完整编码帧I以及图像帧序号与2k相除余数为0的第二非关键帧对应的前向预测编码帧P而不是接收全部的前向预测编码帧。
例如，当k(即当前的播放倍率)取1时，如图4所示，图像解码器可以仅接收每个图像帧序列中的完整编码帧0以及前向预测编码帧2、4、6、8、10以及12，并基于参考依赖关系，完成对这些编码数据的解码以使得录像文件以大于正常播放速度1倍的播放速率进行播放。
在另一些实施例中，在当前的播放倍率大于K时，图像解码器便可以仅接收完整编码帧I并丢弃所有前向预测编码帧，从而以更快的播放速度进行在线回放。
图8为本发明实施例提供的第三码流编码的方法流程图，图9为本发明实施例提供的第三编码数据的示意图。由于远程直播网络流量等的限制，第三码流编码使用的第三输出图像为分辨率较低(如720p)的第三图像帧序列。
如图8所示，该第三码流编码方法可以包括如下步骤：
810、在所述第三图像帧序列中，确定至少一个第三关键帧，两个关键帧之间的图像帧为第三非关键帧。
820、对所述第三关键帧进行帧内压缩编码，以获得完整编码帧。
830、以连续参考的形式，对所述第三非关键帧进行帧间压缩编码，获得前向预测编码帧。
“连续参考”是指每一个第三非关键帧都以前一个图像帧为参考帧，与图6所示的第一码流编码使用相同的参考方式。
840、基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列。
其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成，作为第三编码数据的组成单元。完整编码帧和前向预测编码帧采用了不同的编码方式，其所具有的数据量或者帧的大小存在显著区别。
850、将每段编码序列的完整编码帧的宏块分散到所在编码序列中的一个 或者多个所述前向预测编码帧。
“宏块”是按照特定的数据划分准则，从完整编码帧中划分出来的一部分数据。如图9所示，通过将完整编码帧I的宏块分散到各个前向预测编码帧P中的方式，可以使得每个帧的大小相对平均，传输时码率非常平稳，对网络的冲击较小，适合于远距离的无线通信传输。
综上所述，本发明实施例提供的图像编码方法针对录像文件需要经常被用户读取，并适用于不同的环境中解码播放的特点，采用最普遍的IP模式编码(如HX265/HX264)，并兼顾存储设备的写入能力。
而针对在线回放时的拖拽和加速回放的操作需要，采用跳帧参考的方式确定第二非关键帧的参考帧，以降低图像解码器所需要接收的图像帧数量，减轻无线通信网络的下载负担。
另外，考虑到远程直播时无线网络远程传输时的复杂性，码流不能波动过大的限制，采用宏块分散的方式，将完整编码帧的宏块分散到各个前向预测帧之中，令其码率保持平稳，减少对网络的冲击。
本领域技术人员应该还可以进一步意识到，结合本文中所公开的实施例描述的示例性的图像编码方法的各个步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。
本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。所述的计算机软件可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体或随机存储记忆体等。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；在本发明的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，步骤可以以任意顺序实现，并存在如上所述的本发明的不同方面的许多其它变化，为了简明，它们没有在细节中提供；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (17)

  1. 一种编码方法，用于航拍相机，其特征在于，包括：
    处理原始图像数据，以获得至少两路输出图像；
    对所述至少两路输出图像中每路输出图像进行一种码流编码，以形成至少两路编码数据；其中，不同路的输出图像对应不同的码流；所述至少两路编码数据用于支持至少两种业务需求，所述至少两路编码数据中的每路编码数据用于支持所述至少两种业务需求中的一种业务需求。
  2. 根据权利要求1所述的方法，其特征在于，所述业务需求包括录像、在线回放以及远程直播；所述至少两路输出图像包括：第一输出图像，第二输出图像以及第三输出图像；则，
    所述对所述至少两路输出图像中每路输出图像进行一种码流编码，以形成至少两路编码数据的步骤，具体包括：
    对所述第一输出图像进行第一码流编码，以形成支持录像的第一编码数据；
    对所述第二输出图像进行第二码流编码，以形成支持在线回放的第二编码数据；
    对所述第三输出图像进行第三码流编码，以形成支持远程直播的第三编码数据。
  3. 根据权利要求2所述的方法，其特征在于，所述第一输出图像为具有原始分辨率的第一图像帧序列；则，
    所述对所述第一输出图像进行第一码流编码，以形成支持录像的第一编码数据的步骤，具体包括：
    确定在所述第一图像帧序列中的至少一个图像帧为第一关键帧，其中，两个第一关键帧之间的图像帧为第一非关键帧；
    对所述第一关键帧进行帧内压缩编码，以获得完整编码帧；以及
    参考在所述第一非关键帧之前的图像帧，对所述第一非关键帧进行帧间压缩编码，以获得前向预测编码帧；
    基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成。
  4. 根据权利要求2或3所述的方法，其特征在于，所述方法还包括：
    为所述第一码流编码设置编码质量上限，以使所述第一编码数据的数据流量保持在设定范围内。
  5. 根据权利要求2-4中任一项所述的方法，其特征在于，所述第一码流编码为可变码率编码。
  6. 根据权利要求2-5中任一项所述的方法，其特征在于，所述第二路输出图像为具有第二分辨率的第二图像帧序列，其中，所述第二分辨率低于原始 分辨率；则，
    所述对所述第二路输出图像进行第二码流编码，以形成支持在线回放的第二编码数据的步骤，具体包括：
    确定在所述第二图像帧序列中的至少一个图像帧为第二关键帧，其中，两个第二关键帧之间的图像帧为第二非关键帧；
    对所述第二关键帧进行帧内压缩编码，以获得完整编码帧；并且
    以跳帧参考的形式，对所述第二非关键帧进行帧间压缩编码，以获得前向预测编码帧；
    基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成。
  7. 根据权利要求6所述的方法，其特征在于，所述以跳帧参考的形式，对所述第二非关键帧进行帧间压缩编码，以获得前向预测编码帧的步骤，具体包括：
    选定待编码的第二非关键帧；
    根据在线回放的播放倍率设置，确定所述待编码的第二非关键帧的前一个或者多个图像帧为参考帧；
    计算所述待编码的第二非关键帧与参考帧之间的差别；
    对所述差别，按照预设的压缩算法进行编码，以获得所述待编码的第二非关键帧对应的前向预测编码帧。
  8. 根据权利要求7所述的方法，其特征在于，所述在线回放的播放倍率为1至K之间的正整数；则，
    所述根据在线回放的播放倍率，确定所述待编码的第二非关键帧的前一个或者多个图像帧为参考帧的步骤，具体包括：
    确定所述待编码的第二非关键帧的前2k个图像帧为参考帧，k为0至K的正整数；所述k通过如下算式计算：
    k＝0    mod(n,2)＝1时；
    k＝K    mod(n,2K)＝0时；
    k＝z/2    mod(n,2)＝0时；
    其中，第二关键帧的图像帧序号设置为0，待编码的第二非关键帧的图像帧序号为n，z＝mod(n,2K)。
  9. 根据权利要求2-8任一项所述的方法，其特征在于，所述第一输出图像与第二输出图像的时间轴同步，具有相同的起始时刻和终止时刻。
  10. 根据权利要求2-9中任一项所述的方法，其特征在于，所述第三路输出图像为具有第三分辨率的第三图像帧序列，所述第三分辨率低于原始分辨率；则，
    所述对所述第三路输出图像进行第三码流编码，以形成支持远程直播的编码数据的步骤，具体包括：
    在所述第三图像帧序列中，确定至少一个第三关键帧，其中，两个第三关键帧之间的图像帧为第三非关键帧；
    对所述第三关键帧进行帧内压缩编码，以获得完整编码帧；并且
    以连续参考的形式，对所述第三非关键帧进行帧间压缩编码，以获得前向预测编码帧；
    基于所述完整编码帧和所述前向预测编码帧，形成若干段连续的编码序列，其中，每段编码序列由位于序列首位的完整编码帧和依次排列的前向预测编码帧组成；
    将每段编码序列的完整编码帧的宏块分散到所在编码序列中的一个或者多个所述前向预测编码帧。
  11. 一种图像编码器，其特征在于，包括：处理器以及与所述处理器通信连接的存储器；
    所述存储器存储有计算机程序指令，所述计算机程序指令被所述处理器调用以使所述处理器执行如权利要求1-10任一项所述的编码方法，形成至少两路编码数据以支持至少两种业务需求。
  12. 一种图像传输系统，其特征在于，包括：
    图像编码器，用于执行如权利要求1-10任一项所述的编码方法，形成对应的至少两路编码数据；
    图像解码器，用于根据操作指令，接收部分或者全部所述编码数据并进行解码；
    通信模组，所述通信模组与所述图像编码器和图像解码器连接，用于形成通信信道以传输所述编码数据。
  13. 根据权利要求12所述的图像传输系统，其特征在于，所述图像编码器包括：第一码流编码单元、第二码流编码单元以及第三码流编码单元；
    所述第一码流编码单元用于进行第一码流编码，形成支持录像的第一编码数据；所述第二码流编码单元用于进行第二码流编码，形成支持在线回放的第二编码数据；所述第三码流编码单元用于进行第三码流编码，形成支持远程直播的第三编码数据。
  14. 根据权利要求13所述的图像传输系统，其特征在于，所述图像传输系统还包括：存储设备，所述存储设备与所述图像编码器连接，用于存储所述第一码流编码数据和第二码流编码数据。
  15. 根据权利要求13所述的图像传输系统，其特征在于，在执行在线回放时，所述图像解码器还用于：
    根据当前的播放倍率，接收所述第二编码数据中，关键帧的完整编码帧以及图像帧序号与2k相除余数为0的非关键帧对应的前向预测编码帧，k为当前的播放倍率。
  16. 根据权利要求13所述的图像传输系统，其特征在于，在执行在线回放时，所述图像解码器还用于：在当前的播放倍率大于K时，接收所述第二编 码数据中所有的完整编码帧并丢弃所有的前向预测编码帧。
  17. 根据权利要求13所述的图像传输系统，其特征在于，在执行在线回放时，所述图像解码器还用于：
    在所述第二编码数据中，搜索与指定播放时刻最接近的关键帧并且以所述最接近的关键帧为起点，接收后续的编码序列。
PCT/CN2020/077824 2019-03-06 2020-03-04 编码方法、图像编码器及图像传输系统 WO2020177724A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/467,364 US12003887B2 (en) 2019-03-06 2021-09-06 Image encoding method, image encoder and image transmission system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910168160.X 2019-03-06
CN201910168160.XA CN109819262B (zh) 2019-03-06 2019-03-06 编码方法、图像编码器及图像传输系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/467,364 Continuation US12003887B2 (en) 2019-03-06 2021-09-06 Image encoding method, image encoder and image transmission system

Publications (1)

Publication Number Publication Date
WO2020177724A1 true WO2020177724A1 (zh) 2020-09-10

Family

ID=66608252

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/077824 WO2020177724A1 (zh) 2019-03-06 2020-03-04 编码方法、图像编码器及图像传输系统

Country Status (3)

Country Link
US (1) US12003887B2 (zh)
CN (2) CN109819262B (zh)
WO (1) WO2020177724A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114422822A (zh) * 2021-12-27 2022-04-29 北京长焜科技有限公司 一种支持自适应hdmi编码的无人机数图传输控制方法
CN114827669A (zh) * 2022-03-31 2022-07-29 杭州网易智企科技有限公司 一种视频数据的传输方法、装置、介质及设备

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109819262B (zh) 2019-03-06 2021-06-01 深圳市道通智能航空技术股份有限公司 编码方法、图像编码器及图像传输系统
CN110381331A (zh) * 2019-07-23 2019-10-25 深圳市道通智能航空技术有限公司 一种图像处理方法、装置、航拍设备及存储介质
WO2022000350A1 (zh) * 2020-06-30 2022-01-06 深圳市大疆创新科技有限公司 视频传输方法、可移动平台、终端设备、系统及存储介质
CN114745542A (zh) * 2021-01-08 2022-07-12 华为技术有限公司 编码方法、电子设备、通信系统、存储介质和程序产品
CN113676727A (zh) * 2021-08-18 2021-11-19 深圳市朗强科技有限公司 一种基于wifi的超高清视频的发送、接收方法及设备
CN113891019B (zh) * 2021-09-24 2024-08-20 深圳Tcl新技术有限公司 视频编码方法、装置、拍摄设备和存储介质
CN117294683A (zh) * 2022-06-16 2023-12-26 中兴通讯股份有限公司 视频处理方法、发送端、接收端、存储介质和程序产品
CN115633176B (zh) * 2022-12-08 2023-03-21 北京蔚领时代科技有限公司 动态调整画面群组长度的方法、装置和存储介质
CN115865909B (zh) * 2023-01-13 2023-06-02 中科方德软件有限公司 一种基于spice协议的数据传输方法、装置和可读存储介质
CN117412062A (zh) * 2023-09-28 2024-01-16 协创芯片(上海)有限公司 一种支持h265编码的多媒体芯片

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102724551A (zh) * 2012-06-13 2012-10-10 天脉聚源(北京)传媒科技有限公司 一种视频编码系统和方法
CN103313017A (zh) * 2012-03-16 2013-09-18 安凯(广州)微电子技术有限公司 多路录像方法及系统
CN103428483A (zh) * 2012-05-16 2013-12-04 华为技术有限公司 一种媒体数据处理方法及设备
US20160286231A1 (en) * 2015-03-25 2016-09-29 Hangzhou Hikvision Digital Technology Co., Ltd. Systems and Methods for Controlling Video Display
CN106169998A (zh) * 2016-07-13 2016-11-30 腾讯科技（深圳）有限公司 媒体文件的处理方法和装置
CN108243338A (zh) * 2016-12-23 2018-07-03 杭州海康威视数字技术股份有限公司 视频解码、编码方法、装置及系统
CN109151481A (zh) * 2017-06-28 2019-01-04 腾讯科技（深圳）有限公司 图片的传输和接收的方法、装置及其系统
CN109819262A (zh) * 2019-03-06 2019-05-28 深圳市道通智能航空技术有限公司 编码方法、图像编码器及图像传输系统

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8964830B2 (en) * 2002-12-10 2015-02-24 Ol2, Inc. System and method for multi-stream video compression using multiple encoding formats
US20070081789A1 (en) * 2005-10-06 2007-04-12 Lockheed Martin Corporation Time shifting for video
CN101600097B (zh) * 2009-03-06 2011-06-15 深圳市灵琅科技有限公司 移动多媒体安防监控的方法和系统
CN101668208B (zh) * 2009-09-15 2013-03-27 浙江宇视科技有限公司 帧编码方法及装置
EP2727340A4 (en) * 2011-06-29 2015-05-27 Rockstar Consortium Us Ip METHOD AND DEVICE FOR VIDEO CODING FOR PLAYBACKING MULTIPLE SPEEDS
US10009628B2 (en) * 2013-06-07 2018-06-26 Apple Inc. Tuning video compression for high frame rate and variable frame rate capture
CN105208342B (zh) * 2015-09-25 2018-06-15 中国船舶重工集团公司第七一七研究所 一种双路视频压缩存储及网络切换传输电路
CN105791836B (zh) * 2016-03-07 2019-04-02 中国科学院计算技术研究所 视频编码方法、视频码流自适应传输方法和显示方法
CN106060539B (zh) * 2016-06-16 2019-04-09 深圳风景网络科技有限公司 一种低传输带宽的视频编码方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103313017A (zh) * 2012-03-16 2013-09-18 安凯(广州)微电子技术有限公司 多路录像方法及系统
CN103428483A (zh) * 2012-05-16 2013-12-04 华为技术有限公司 一种媒体数据处理方法及设备
CN102724551A (zh) * 2012-06-13 2012-10-10 天脉聚源(北京)传媒科技有限公司 一种视频编码系统和方法
US20160286231A1 (en) * 2015-03-25 2016-09-29 Hangzhou Hikvision Digital Technology Co., Ltd. Systems and Methods for Controlling Video Display
CN106169998A (zh) * 2016-07-13 2016-11-30 腾讯科技（深圳）有限公司 媒体文件的处理方法和装置
CN108243338A (zh) * 2016-12-23 2018-07-03 杭州海康威视数字技术股份有限公司 视频解码、编码方法、装置及系统
CN109151481A (zh) * 2017-06-28 2019-01-04 腾讯科技（深圳）有限公司 图片的传输和接收的方法、装置及其系统
CN109819262A (zh) * 2019-03-06 2019-05-28 深圳市道通智能航空技术有限公司 编码方法、图像编码器及图像传输系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114422822A (zh) * 2021-12-27 2022-04-29 北京长焜科技有限公司 一种支持自适应hdmi编码的无人机数图传输控制方法
CN114422822B (zh) * 2021-12-27 2023-06-06 北京长焜科技有限公司 一种支持自适应hdmi编码的无人机数图传输控制方法
CN114827669A (zh) * 2022-03-31 2022-07-29 杭州网易智企科技有限公司 一种视频数据的传输方法、装置、介质及设备
CN114827669B (zh) * 2022-03-31 2023-08-18 杭州网易智企科技有限公司 一种视频数据的传输方法、装置、介质及设备

Also Published As

Publication number Publication date
US20210400233A1 (en) 2021-12-23
US12003887B2 (en) 2024-06-04
CN113315973A (zh) 2021-08-27
CN109819262A (zh) 2019-05-28
CN109819262B (zh) 2021-06-01
CN113315973B (zh) 2022-12-20

Similar Documents

Publication Publication Date Title
WO2020177724A1 (zh) 编码方法、图像编码器及图像传输系统
KR101859155B1 (ko) 높은 프레임 레이트 및 가변 프레임 레이트 캡처를 위한 비디오 압축 튜닝
KR102324326B1 (ko) 상이한 인코딩 파라미터를 이용해 인코딩되는 복수의 인코딩 스트리밍
KR101019634B1 (ko) 미디어 전송 시스템 및 방법
US8428423B2 (en) Reproducing apparatus for video data
US8422565B2 (en) Information processing device and method, and information processing system
US20090290645A1 (en) System and Method for Using Coded Data From a Video Source to Compress a Media Signal
JP6463041B2 (ja) 画像処理装置、画像処理方法、及びプログラム
KR20070042119A (ko) 원격 편집 시스템, 주편집 장치, 원격 편집 장치, 편집방법, 편집 프로그램, 및 기억 매체
JPWO2007066709A1 (ja) 符号化装置、符号化方法および符号化プログラム、ならびに、復号装置、復号方法および復号プログラム
JP2008187573A (ja) 情報処理装置および方法、プログラム、並びに記録媒体
CN101291393B (zh) 具高速、高质量图像连续捕捉能力的图像输入设备及方法
KR100678891B1 (ko) Ａｖ데이터 수신시 버퍼량을 컨텐츠 속성에 따라탄력적으로 조절하는 방법 및 장치
JP2020524450A (ja) 多チャネル映像のための伝送システム及びその制御方法、多チャネル映像再生方法及びその装置
CN104735410B (zh) 一种低于4k/s的窄带宽视频传输方法及系统
JP4915208B2 (ja) ストリームデータ再生システム
KR20170101760A (ko) 정보 처리 장치
JP2010011287A (ja) 映像伝送方法および端末装置
JP5072524B2 (ja) 動画像再生装置，およびその処理方法およびそのプログラム
JP2014075688A (ja) 画像処理装置及び画像処理方法
CN117579843B (zh) 视频编码处理方法及电子设备
JP2006033506A (ja) 遠隔編集システム、主編集装置、遠隔編集装置、編集方法、編集プログラム、及び記憶媒体
KR101190753B1 (ko) 디지털 비디오 레코더의 비디오 프레임 전송방법
JP5925020B2 (ja) 再生装置
KR20140074130A (ko) Ｓｄ-ｈｄ 하이브리드 ｄｖｒ 시스템 및 그 구동 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20766765

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20766765

Country of ref document: EP

Kind code of ref document: A1