WO2019134255A1 - Système et procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau - Google Patents

Système et procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau Download PDF

Info

Publication number
WO2019134255A1
WO2019134255A1 PCT/CN2018/078197 CN2018078197W WO2019134255A1 WO 2019134255 A1 WO2019134255 A1 WO 2019134255A1 CN 2018078197 W CN2018078197 W CN 2018078197W WO 2019134255 A1 WO2019134255 A1 WO 2019134255A1
Authority
WO
WIPO (PCT)
Prior art keywords
packet loss
image
module
information
frame
Prior art date
Application number
PCT/CN2018/078197
Other languages
English (en)
Chinese (zh)
Inventor
张联昌
毕永建
孔令波
Original Assignee
厦门亿联网络技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 厦门亿联网络技术股份有限公司 filed Critical 厦门亿联网络技术股份有限公司
Publication of WO2019134255A1 publication Critical patent/WO2019134255A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64784—Data processing by the network
    • H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols

Definitions

  • the present invention relates to the field of real-time encoded image transmission technologies, and in particular, to a real-time image adaptive recovery system and method with network transmission characteristics friendly.
  • this method can reduce the spread of abnormal images, but it will inevitably lead to the 'mosaic' phenomenon (because the actual data has been lost, the missing part of the image can only be recovered by some predictive means, with real There is a certain error in the decoded image, and due to the temporal correlation between the encoded frames, the error image will continue to spread, causing a mosaic phenomenon).
  • the decoding end after detecting the packet loss, the decoding end sends a message request IDR frame to the encoding end device (immediately refreshing the frame), thereby blocking the abnormality caused by the packet loss.
  • the encoding end needs to apply for an IDR frame to truly block the spread of the abnormal image, and then the application for the IDR frame occurs. The more serious the network packet loss, the more frequently the IDR frame is applied.
  • the IDR frame has the following disadvantages: 1) The size of the IDR frame is much larger than the size of the P frame (predictive coded frame) in the case of the same quality, and 2) the image quality of the IDR frame is much worse than that of the P frame under the same size condition. If the IDR frame is frequently applied, the quality of the encoded image will be degraded, and it is not conducive to network transmission. In extreme cases, it will cause network congestion, which will cause more serious packet loss.
  • the inventors have proposed a real-time image adaptive recovery system and method with network transmission characteristics to solve the problem of frequently applying IDR frames in a packet loss environment.
  • the object of the present invention is to provide a real-time image adaptive recovery system and method with friendly network transmission characteristics, so as to solve the problem of frequently applying IDR frames in a packet loss environment, and have certain image quality and network transmission effects of the real-time codec system. Upgrade.
  • a real-time image adaptive recovery system with friendly network transmission characteristics includes two parts: an encoding end and a decoding end, wherein the encoding end includes:
  • a coded image index generating module the module generates a unique image index of the coded sequence, and is used by the encoding end to determine a time position corresponding relationship between the lost image frame and the current frame to be encoded fed back by the encoding end;
  • a coded image macroblock information backup module backing up macroblock type information of the coded image and motion vector information, for calculating a packet loss data propagation path and a region;
  • a packet loss data propagation area calculation module configured to calculate a propagation area of the lost packet data according to the correlation between the inter-frame information, including macroblock type information and motion vector information;
  • the packet loss type coding type control module is configured to control the coded macroblock type of the coding end. If it is determined that the macroblock to be coded is within the range of the packet loss data propagation area, the intra macroblock coding is performed to block the abnormal propagation. Otherwise, the normal encoding process is performed;
  • the decoding end includes:
  • Decoding an image index extraction module extracts an index of the currently decoded frame image, and if a packet loss occurs, the information needs to be transmitted to the encoding end, and the encoding end uses the information to determine the decoded frame between the lost packet and the frame to be encoded. Time correlation
  • Decoding a packet loss information statistics module where the module counts the location of the packet loss data and the coverage of the packet loss data, including a stripe level and a macroblock level, which are used by the encoding end to check the type of information;
  • the module transmits the packet loss packet loss statistics and the packet loss image index information to the encoding end through the network protocol for abnormal image recovery.
  • a real-time image adaptive recovery method with network transmission characteristics including:
  • the encoding end calculates a macroblock correspondence relationship between the decoded frame and the current frame to be encoded according to the packet loss location information fed back by the decoding end;
  • the packet loss location information includes a decoded frame index and a stripe index;
  • the encoding end encodes the intra macroblock mode in the area where the association exists, and blocks the propagation path of the decoded abnormal image;
  • the encoding end sends the encoded frame code stream, and can immediately return to the normal image after receiving the encoded frame data at the decoding end.
  • the present invention has the following advantages:
  • the present invention can accurately control an abnormal image propagation path and an area caused by packet loss
  • the present invention performs precise control of the encoded macroblock type according to the abnormal image propagation range, and blocks abnormal image propagation;
  • the present invention can recover the abnormal image caused by the packet loss without applying for the IDR frame, and the image restoration effect is the same as the application IDR mode, and the image abnormality caused by the data packet loss can be completely recovered, and the network bandwidth occupation is more The IDR method is low.
  • 1 is a schematic diagram of a macroblock dependency relationship between an encoding end and a decoding end
  • FIG. 2 is a flow chart of information transmission and processing of the encoding end and the decoding end of the present invention
  • FIG. 3 is a structural block diagram of a real-time image adaptive recovery system with network transmission characteristics friendly according to the present invention
  • Figure 5 is a flow chart of the decoding process of the present invention.
  • the multi-band coding of AVC (H.264) is taken as an example, but the application range is not limited to AVC, and the macroblock dependency relationship between the encoding end and the decoding end is a simple explanation of related problems, only in the image.
  • the individual macroblocks should be extended to the macroblock dependencies of the entire image.
  • Each image frame of the encoding end and the decoding end is encoded or decoded in time sequence, and the encoding end and the decoding end are in one-to-one correspondence, taking the frame number as an example, wherein the N-1 frame is first encoded or decoded, and the Nth frame is second, Nth +1 again.
  • the blocks with the same color between different image frames represent that there is a dependency relationship between them, and the data of the Nth frame is related to the N-1 frame and N.
  • +1 frame has a relationship, wherein the gray corresponding to yellow, green, blue, orange, and purple in the N frame corresponds to the same gray level of the N-1 frame, and the other gray blocks of the Nth frame and the Nth The +1 frame gray area corresponds.
  • the area divided by the thick line represents the strip area of the AVC, and the square grid inside represents the macroblock information (Macroblock) in the image codec;
  • the N-1 frame of the decoding end has a thinner slash coverage area code.
  • the decoded code stream of this part is lost during transmission. Due to the temporal correlation between the decoded pictures, the N area and the corresponding area of the N+1 frame are decoded.
  • the image is abnormal, which in turn causes the abnormal image to spread (the N-frame at the decoding end and the oblique line in the N+1 frame in Fig. 1 represent the position of the diffusion).
  • the present invention discloses a real-time image adaptive recovery system with network transmission characteristics friendly
  • the encoding end 100 and the decoding end 200 are two parts, wherein the encoding end 100 includes:
  • a coded image index generating module 101 the module generates a unique image index of the coded sequence, and is used by the encoding end to determine a time position corresponding relationship between the lost image frame and the current frame to be encoded fed back by the encoding end;
  • Encoding an image macroblock information backup module 102 the module backing up macroblock type information of the encoded image and motion vector information, for calculating a packet loss data propagation path and a region;
  • a packet loss data propagation area calculation module 103 configured to calculate a propagation area of the packet loss data according to the correlation between the interframe information, including macroblock type information and motion vector information;
  • the packet loss area coding type control module 104 is configured to control the coded macroblock type of the coding end 100. If it is determined that the macroblock to be coded is within the range of the 'dropped data propagation area', the intra macroblock coding is performed. Block the abnormal propagation; otherwise, perform the normal encoding process;
  • the decoding end 200 includes:
  • the decoded image index extraction module 201 is configured to extract an index of the currently decoded frame image. If a packet loss occurs, the information needs to be transmitted to the encoding end, and the encoding end uses the information to determine the decoded frame between the lost packet and the frame to be encoded. Time correlation;
  • Decoding a packet loss information statistic module 202 where the module counts the location of the packet loss data and the coverage of the packet loss data, including a stripe level and a macroblock level, which are used by the encoding end to check the information to prevent misjudgment;
  • the packet loss information feedback module 203 is decoded, and the module transmits the packet loss packet loss statistics and the packet loss image index information to the encoding end through the network protocol for abnormal image recovery.
  • the present invention also discloses a network transmission characteristic friendly real-time image adaptive recovery method implemented by the system. Referring to Figures 4 and 5, the method includes :
  • the encoded image index generating module 101 generates a unique image index of the encoded sequence, and the encoded image macroblock information backup module 102 backs up the macroblock type information and the motion vector information of the encoded image, so that the frame image can be saved in advance at the encoding end 100.
  • the decoded image index extraction module 201 extracts an index of the currently decoded frame image. If a packet loss occurs, the decoding packet loss information statistic module 202 counts the location of the lost packet data and the coverage of the lost packet data (including the strip level and the macro). At the block level, the decoding packet loss information feedback module 203 transmits the packet loss packet loss statistics and the packet loss image index information to the encoding end 100 through the network protocol; if no packet loss occurs, the decoding terminal 200 performs a normal decoding process;
  • the encoding end 100 receives the packet loss location information (including the decoded frame image index extracted by the decoded image index extraction module 201 and the strip index indexed by the decoding packet loss information statistic module 202), the packet loss data propagation area is calculated.
  • the module 103 calculates a macroblock correspondence relationship between the decoded frame and the current frame to be encoded according to the packet loss location information fed back by the decoding end 200;
  • the packet loss type coding type control module 104 of the encoding end 100 performs a macroblock type decision to control the coded macroblock type of the encoding end 100, if it is determined that the macroblock to be encoded is in the range of 'dropped data propagation area' Within the frame, the intra macroblock coding is performed to block the abnormal propagation; otherwise, the normal encoding process is performed, and the intra macroblock (intra) mode is used to encode the associated region, and the propagation path of the decoded abnormal image is blocked. ;
  • the encoding end 100 sends the encoded frame code stream to the decoding end, and after the decoding end receives the encoded frame data, it can be restored to a normal image by macroblock decoding.
  • the coded image adaptive recovery system and method for optimizing network transmission characteristics can accurately estimate the range of diffusion of the 'abnormal image' by using the cooperative cooperation of the encoding end 100 and the decoding end 200 to transmit additional information through the network protocol.
  • the encoding end 100 performs intra macroblock encoding in a specific area according to the calculated 'diffusion range' information, which can actually block the abnormal image diffusion without applying for an IDR frame, and the size of the encoded frame is less than or equal to the IDR frame. Size, the size of the coded frame in the normal network packet loss environment is much smaller than the IDR frame size, thereby avoiding the frequent application of the IDR defect, and has great benefits for the coding quality and the network transmission characteristics.
  • the present invention can be applied to video conferencing systems, video surveillance, and video related APP applications, but is not limited to the above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

L'invention concerne un système et un procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau. Le système comprend deux parties : une extrémité de codage et une extrémité de décodage. L'extrémité de codage comprend un module de génération d'indice d'image de codage, un module de sauvegarde d'informations de macro-bloc d'image de codage, un module de calcul de zone de propagation de données de perte de paquet et un module de commande de type de codage de zone de perte de paquet. L'extrémité de décodage comprend un module d'extraction d'indice d'image de décodage, un module de statistiques d'informations de perte de paquet de décodage et un module de rétroaction d'informations de perte de paquet de décodage. Selon le système et le procédé, le problème d'application fréquente des trames IDR dans un environnement de perte de paquets est résolu, et la qualité d'image et l'effet de transmission de réseau d'un système de codage/décodage en temps réel sont améliorés dans une certaine mesure.
PCT/CN2018/078197 2018-01-03 2018-03-07 Système et procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau WO2019134255A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810005359.6A CN108235150A (zh) 2018-01-03 2018-01-03 一种网络传输特性友好的实时图像自适应恢复系统及方法
CN201810005359.6 2018-01-03

Publications (1)

Publication Number Publication Date
WO2019134255A1 true WO2019134255A1 (fr) 2019-07-11

Family

ID=62645128

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/078197 WO2019134255A1 (fr) 2018-01-03 2018-03-07 Système et procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau

Country Status (2)

Country Link
CN (1) CN108235150A (fr)
WO (1) WO2019134255A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1307430A (zh) * 2000-01-27 2001-08-08 三星电子株式会社 抑制差错扩散的发送/接收数据处理方法及其记录介质
CN1856112A (zh) * 2005-04-20 2006-11-01 华为技术有限公司 基于h.264的压缩视频传输误码消除方法
CN102025993A (zh) * 2010-12-17 2011-04-20 深圳中兴力维技术有限公司 一种基于h.264的视频传输方法及系统
CN103024400A (zh) * 2011-12-19 2013-04-03 北京捷成世纪科技股份有限公司 基于网络的视频压缩容错传输方法及系统
CN106559397A (zh) * 2015-09-30 2017-04-05 联芯科技有限公司 Volte视频电话的图像错误恢复方法、装置和视频引擎

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9661376B2 (en) * 2005-07-13 2017-05-23 Polycom, Inc. Video error concealment method
CN101304538B (zh) * 2008-06-19 2010-07-28 宁波中科集成电路设计中心有限公司 Avs视频解码的错误检测与掩盖方法
CN109936746B (zh) * 2016-12-30 2021-07-16 深圳市大疆创新科技有限公司 图像处理方法与设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1307430A (zh) * 2000-01-27 2001-08-08 三星电子株式会社 抑制差错扩散的发送/接收数据处理方法及其记录介质
CN1856112A (zh) * 2005-04-20 2006-11-01 华为技术有限公司 基于h.264的压缩视频传输误码消除方法
CN102025993A (zh) * 2010-12-17 2011-04-20 深圳中兴力维技术有限公司 一种基于h.264的视频传输方法及系统
CN103024400A (zh) * 2011-12-19 2013-04-03 北京捷成世纪科技股份有限公司 基于网络的视频压缩容错传输方法及系统
CN106559397A (zh) * 2015-09-30 2017-04-05 联芯科技有限公司 Volte视频电话的图像错误恢复方法、装置和视频引擎

Also Published As

Publication number Publication date
CN108235150A (zh) 2018-06-29

Similar Documents

Publication Publication Date Title
CN100459717C (zh) 基于h.264的压缩视频传输误码消除方法
EP1157562B1 (fr) Codage video
CN103152576B (zh) 一种适用于组播抗丢包的视频编码及解码装置
CN105704580B (zh) 一种视频传输方法
WO2011038694A1 (fr) Procédé, dispositif et système réseau destinés au traitement de transmission et d'envoi de données vidéo
US20110249127A1 (en) Estimating Video Quality Corruption in Lossy Networks
KR20140098248A (ko) 비디오 스트림 내의 아티팩트를 최소화하는 방법, 비디오 스트림의 속성을 변경하는 시스템 및 컴퓨터 판독가능 매체
CN101355707B (zh) 一种立体视频分层编码差错控制方法
CN110392284A (zh) 视频编码、视频数据处理方法、装置、计算机设备和存储介质
CN101312536B (zh) 一种用于视频传输差错控制的视频编码方法及装置
WO2020237466A1 (fr) Procédé et appareil de vidéotransmission, et aéronef, dispositif de lecture et support de stockage
WO2012003808A1 (fr) Procédé, dispositif et système pour rafraîchissement vidéo
WO2019134255A1 (fr) Système et procédé de récupération adaptative d'image en temps réel respectueux de la caractéristique de transmission de réseau
CN104363461A (zh) 视频帧的错误隐藏方法及应用其的视频解码方法
CN104967871B (zh) 一种视频编码码流的统计复用系统及方法
CN101394568A (zh) 视频数据的更新方法及其装置和系统
CN112929667B (zh) 一种编解码方法、装置、设备及可读存储介质
WO2006054206A2 (fr) Procede de determination d'une indication de corruption d'une sequence de trames de donnees codees
CN106559397A (zh) Volte视频电话的图像错误恢复方法、装置和视频引擎
CN101188771A (zh) 视频解码错误检测与消除的方法及装置
CA2528475A1 (fr) Systeme de communication de donnees et procede de communication de donnees d'images
JPH04115792A (ja) 画像信号符号化方式
Ganguly et al. A-REaLiSTIQ-ViBe: Entangling Encoding and Transport to Improve Live Video Experience
CN115277508A (zh) 一种远程摄像头注入云主机的网络带宽预测方法
CN104661032A (zh) 一种采用动态ip解析技术可移动性高速车载视频监控系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18898098

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18898098

Country of ref document: EP

Kind code of ref document: A1