CN111212286B - Distributed multi-view video transmission method and system based on layer repair - Google Patents
Distributed multi-view video transmission method and system based on layer repair Download PDFInfo
- Publication number
- CN111212286B CN111212286B CN202010038328.8A CN202010038328A CN111212286B CN 111212286 B CN111212286 B CN 111212286B CN 202010038328 A CN202010038328 A CN 202010038328A CN 111212286 B CN111212286 B CN 111212286B
- Authority
- CN
- China
- Prior art keywords
- frame
- code stream
- decoding
- intra
- coding mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/395—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving distributed video coding [DVC], e.g. Wyner-Ziv video coding or Slepian-Wolf video coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/65—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a distributed multi-view video transmission method and a distributed multi-view video transmission system based on layer restoration, which relate to the field of video transmission, and comprise the following steps: the video acquisition equipment performs WZ encoding on K frames and WZ frames in parallel in the acquired video data and encodes the K frames in an intra-frame encoding mode; storing the K frame code stream after the WZ coding in a cache region; sending the K frame code stream and the WZ frame code stream to decoding equipment; the method comprises the steps that when a decoding device detects that packet loss or errors occur in K frame code streams, packet information is sent to a video acquisition device, and the K frame code streams corresponding to identification information in the K frame code streams after WZ coding are received; correcting errors according to the K frame code stream and the K frame code stream corresponding to the identification information to obtain high-quality side information; and then decoded to construct video data. The scheme of the invention improves the transmission quality of the K frame, realizes the robust transmission of the DMVC system, compresses the code rate of the WZ frame, and improves the RD performance of the DMVC system, so that the scheme of the invention has stronger expansibility and adaptability.
Description
Technical Field
The invention relates to the field of video transmission, in particular to a distributed multi-view video transmission method and system based on layer repair.
Background
Distributed Multi-view Video Coding (DMVC) brings new eosin for Multi-view Video Coding of dense low-power consumption terminals in a wireless environment, and the DMVC adopts an independent Coding-joint decoding mode, and has the characteristics of low Coding complexity, high error code robustness, and the like. However, the problems of large data volume after coding, limited wireless bandwidth, large fluctuation, high wireless channel transmission error rate and the like bring new challenges to the wireless transmission of the DMVC data, so that the development of research on the wireless transmission technology of the DMVC video data has very important academic value and practical significance.
The DMVC system is independent coding-joint decoding, the correct decoding of K frames in video data is beneficial to generating high-quality side information, and the side information is directly related to the data compression rate of the whole DMVC system and the recovery effect of video. Based on a multi-view model, under the environment of a wireless network which is easy to lose, how to generate a stable-quality and reliable K frame is a problem to be solved urgently in DMVC system transmission.
Disclosure of Invention
In view of the above problems, the present invention provides a method and a system for distributed multi-view video transmission based on layer repair, which generate more stable and reliable K frames in a wireless network environment with easy loss, thereby solving the above problems.
The embodiment of the invention provides a distributed multi-view video transmission method based on layer repair, which comprises the following steps:
the video acquisition equipment performs WZ encoding on a K frame and a WZ frame in the acquired video data in parallel and encodes the K frame in an intra-frame encoding mode;
the video acquisition equipment stores the K frame code stream after the WZ coding in a cache region;
the video acquisition equipment sends a K frame code stream coded in an intra-frame coding mode and a WZ frame code stream coded in a WZ mode to decoding equipment;
the decoding equipment receives a K frame code stream and a WZ frame code stream which are coded in an intra-frame coding mode;
the decoding equipment detects whether the K frame code stream coded in the intra-frame coding mode has packet loss or errors;
when the K frame code stream coded in the intra-frame coding mode has a packet loss or an error, the decoding device sends packet information to the video acquisition device, where the packet information includes: identification information of lost packets or error packets in the K frame code stream coded in the intra-frame coding mode;
the decoding equipment receives the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding;
the decoding equipment corrects the error according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information;
and the decoding equipment decodes and constructs the acquired video data according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
Optionally, the video capture device WZ-encodes the K frames, including:
the video acquisition equipment carries out DWT (discrete wavelet transform) on the K frame so as to quantize each transformed sub-band coefficient;
and the video acquisition equipment extracts a bit plane from the quantized K frames and performs channel coding on the bit plane so as to realize WZ coding on the K frames.
Optionally, after the decoding device sends packet information to the video capture device, the method further includes:
the video acquisition equipment receives the packet information;
and the video acquisition equipment sends a packet corresponding to the identification information in the K frame code stream after the WZ coding to the decoding equipment according to the identification information.
Optionally, the performing, by the decoding device, error correction according to the K frame code stream encoded in the intra-frame encoding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information includes:
the decoding equipment decodes the K frames of code streams coded in the intra-frame coding mode to obtain damaged side information corresponding to damaged K frames;
the decoding equipment carries out DWT transformation on the damaged side information so as to quantize each transformed sub-band coefficient;
the decoding device extracts a bit plane from the quantized damaged side information to obtain side information bit plane data corresponding to the damaged side information;
the decoding device performs channel decoding on the K frames of code streams corresponding to the identification information to obtain bit plane data of the K frames corresponding to the identification information;
and the decoding equipment corrects the damaged side information bit plane data by using the bit plane data of the K frames corresponding to the identification information to obtain high-quality side information.
Optionally, the decoding device decodes the K frame code stream encoded in the intra-frame encoding mode to obtain damaged side information corresponding to the damaged K frame, including:
the decoding equipment adopts an error concealment technology to decode the K frame code stream coded in the intra-frame coding mode, and rebuilds a damaged K frame image;
and the decoding equipment adopts an error concealment technology to obtain the damaged K frame image with better quality, and determines the damaged K frame image as the damaged side information corresponding to the damaged K frame image.
Optionally, the video capture device sends the K frame code stream and the WZ frame code stream coded in the intra-frame coding mode to a decoding device, and the method further includes:
the decoding equipment receives the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode;
the decoding equipment detects that no packet loss or error occurs in the K frame code stream coded in the intra-frame coding mode;
and the decoding equipment decodes and constructs the collected video data according to the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
The invention also provides a system for distributed multi-view video transmission based on layer repair, which comprises: video capture device and decoding equipment, video capture device includes: the device comprises a parallel coding module, a cache storage module and a code stream sending module; the decoding apparatus includes: the device comprises a code stream receiving module, a detection module, a packet information sending module, a corresponding code stream receiving module, an error correction module and a decoding construction module;
the parallel coding module is used for carrying out WZ coding on a K frame and a WZ frame in the collected video data in parallel and coding the K frame in an intra-frame coding mode;
the cache storage module is used for storing the K frame code stream after the WZ coding in a cache region;
the code stream sending module is used for sending the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode to the decoding equipment;
the code stream receiving module is used for receiving the K frame code stream and the WZ frame code stream which are coded in an intra-frame coding mode;
the detection module is used for detecting whether the K frame code stream coded in the intra-frame coding mode has packet loss or errors;
the packet information sending module is configured to send packet information to the video acquisition device when the K frame code stream encoded in the intra-frame encoding mode is lost or has an error, where the packet information includes: identification information of lost packets or error packets in the K frame code stream coded in the intra-frame coding mode;
the code stream receiving corresponding module is used for receiving the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding;
the error correction module is used for correcting errors according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information;
and the decoding construction module is used for decoding and constructing the acquired video data according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
Optionally, the parallel encoding module includes:
the change quantization sub-module is used for carrying out DWT (discrete wavelet transform) on the K frame so as to quantize each transformed sub-band coefficient;
and the extraction coding sub-module is used for extracting bit planes of the quantized K frames and carrying out channel coding on the bit planes so as to realize WZ coding on the K frames.
Optionally, the video capture device further comprises: a packet information receiving module and a corresponding code stream sending module;
the packet information receiving module is used for receiving the packet information;
and the corresponding code stream sending module is used for sending a packet corresponding to the identification information in the K frame code stream after the WZ coding to the decoding equipment according to the identification information.
Optionally, the error correction module includes:
the damaged side information decoding submodule is used for decoding according to the K frame code stream coded in the intra-frame coding mode to obtain damaged side information corresponding to the damaged K frame;
a damaged side information variation submodule for performing DWT conversion on the damaged side information to quantize each transformed sub-band coefficient;
the damaged side information extraction submodule is used for extracting a bit plane from the quantized damaged side information to obtain side information bit plane data corresponding to the damaged side information;
the corresponding code stream decoding submodule is used for carrying out channel decoding on the K frame code stream corresponding to the identification information to obtain bit plane data of the K frame corresponding to the identification information;
and the error correction submodule is used for correcting the damaged side information bit plane data by using the bit plane data of the K frames corresponding to the identification information to obtain high-quality side information.
Optionally, the decoding damaged side information sub-module is specifically configured to:
decoding the K frame code stream coded in the intra-frame coding mode by adopting an error concealment technology to reconstruct a damaged K frame image;
and obtaining the damaged K frame image with better quality by adopting an error concealment technology, and determining the damaged K frame image as the damaged side information corresponding to the damaged K frame image.
The invention provides a distributed multi-view video transmission method based on layer repair.A video acquisition device carries out WZ encoding on a K frame and a WZ frame in parallel in acquired video data and encodes the K frame in an intra-frame encoding mode; then storing the K frame code stream after the WZ coding in a cache region; the video acquisition equipment sends the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode to the decoding equipment; the method comprises the steps that when packet loss or errors occur in K frame code streams coded in an intra-frame coding mode, a decoding device sends packet information to a video collecting device, and the decoding device receives the K frame code streams corresponding to identification information in the K frame code streams coded in the WZ mode; the decoding equipment corrects the error according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information; and decoding and constructing the acquired video data by the decoding equipment according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
According to the scheme, under the condition that the K frame is lost or wrong, the error of the K frame is corrected by utilizing the WZ coded K frame in the cache to obtain high-quality side information, correct decoding blocks of the K frame without loss are fully utilized, the transmission quality of the K frame is obviously improved, the robust transmission of a DMVC system is realized, and the advantage that the compressibility of the K frame of the DMVC system is not changed is achieved. The scheme of the invention greatly improves the decoding quality of the K frame, compresses the code rate of the WZ frame, further improves the overall RD performance of the DMVC system and ensures that the scheme of the invention has stronger expansibility and adaptability.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic diagram of a current distributed multi-view video transmission model based on a DJSCC framework;
fig. 2 is a flowchart of a method for distributed multi-view video transmission based on layer repair according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a transmission model of distributed multi-view video transmission based on layer repair according to an embodiment of the present invention;
FIG. 4 is a graph comparing code rate values for three code rates according to an embodiment of the present invention;
FIG. 5 is a graph of PSNR values of decoded video quality comparing three cases in an embodiment of the present invention;
FIG. 6 is a graph illustrating rate-distortion curves comparing the decoded video quality of K frames under three conditions in an embodiment of the present invention;
fig. 7 is a graph illustrating rate-distortion curves comparing the decoded video quality of WZ frames under three situations in the embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention, but do not limit the invention to only some, but not all embodiments.
The inventors have found that the correct decoding of K frames in video data of a DMVC system helps to generate high quality side information, which is directly related to the data compression rate of the entire DMVC system and the recovery effect of the video. Based on the above findings, the inventors further study how to improve the quality of the K frame, and through careful study, a large number of experimental tests, and by combining with the DMVC system layer repair DJSCC theoretical framework, the great improvement of the quality of the K frame is achieved, and the scheme of the present invention is explained and explained in detail below.
In order to better describe the scheme of the present invention, first, a brief description is made of the encoding and decoding method adopted by the current DMVC system. Referring to fig. 1, a schematic diagram of the current distributed multi-view video transmission model based on the DJSCC framework is shown.
As in fig. 1, each view in the distributed multi-view video coding framework is divided into corresponding K-frames and WZ-frames according to a preset GOP, i.e., the video series is divided into corresponding K-frames and WZ-frames, where the K-frames employ the intra-coding mode (I-encoder) of the current compression coding technology (e.g., JPEG2000, h.264intra, HEVC, etc.), and the WZ-frames employ Wyner-Ziv coding (WZ-encoder).
And (3) directly carrying out independent coding between all viewpoints of a coding end without any information interaction, and then sending the coded code stream into a wireless channel (packet loss channel) for transmission.
At a decoding end, firstly, adopting intra-frame decoding for K frames; decoding of the WZ frame firstly needs to generate time domain side information in a view point and space domain side information between the view points, namely side information formed on the basis of the K frame, then optimal side information is obtained through a certain fusion algorithm, and finally the WZ frame is decoded and reconstructed.
And finally recombining the obtained K frames and WZ frames into a decoded video sequence according to the initial GOP setting sequence, and outputting the decoded video sequence to a playing device for playing.
The above is the process of the distributed multi-view video transmission based on the DJSCC framework at present. Referring to fig. 2, a flowchart of a method for distributed multi-view video transmission based on layer repair according to an embodiment of the present invention is shown, where the method includes:
step 101: the video capture device performs WZ encoding on K frames and WZ frames in parallel in the captured video data, and encodes the K frames in an intra-frame encoding mode.
In the embodiment of the present invention, the video capture device also divides the captured video data into corresponding K frames and WZ frames according to the preset GOP, and then performs WZ encoding on the K frames and the WZ frames in parallel, and certainly, the K frames also need to be encoded in an intra-frame encoding mode, for example: encode K frames with HEVC intra coding, etc.
Optionally, the performing, by the video capture device, WZ encoding on the K frame specifically includes:
step s 1: the video acquisition equipment carries out DWT (discrete wavelet transform) on the K frame so as to quantize each transformed sub-band coefficient;
step s 2: and the video acquisition equipment extracts a bit plane from the quantized K frames and performs channel coding on the bit plane so as to realize WZ coding on the K frames.
In the embodiment of the invention, the video acquisition equipment firstly carries out DWT (Discrete Wavelet transform) on the K frame to quantize each transformed sub-band coefficient, then extracts a bit plane from the quantized K frame data, carries out channel coding on the bit plane, and realizes WZ coding on the K frame after the coding is finished.
Step 102: and the video acquisition equipment stores the K frame code stream after the WZ coding in a cache region.
Step 103: and the video acquisition equipment sends the K frame code stream and the WZ frame code stream which are coded in the intra-frame coding mode to the decoding equipment.
In the embodiment of the invention, after the coding is finished, the video acquisition equipment stores the K frame code stream after the WZ coding in the cache region, does not transmit the K frame code stream, and sends the K frame code stream after the coding in the intra-frame coding mode and the WZ frame code stream after the WZ coding to the decoding equipment through a packet loss channel.
It should be noted that, the buffer may also temporarily store the K frame code stream and the WZ frame code stream coded in the intra-frame coding mode, which is beneficial to: the transmission code rate of the whole video data code stream can be reduced, and the congestion rate of a wireless transmission network is reduced.
Step 104: the decoding device receives a K frame code stream and a WZ frame code stream which are coded in an intra-frame coding mode.
Step 105: the decoding device detects whether packet loss or errors occur in the K frame code stream coded in the intra-frame coding mode.
In the embodiment of the invention, the decoding equipment receives the K frame code stream and the WZ frame code stream which are coded in the intra-frame coding mode, and needs to detect and decode the K frame code stream and the WZ frame code stream. In the detection process, it needs to be determined whether a packet loss or an error occurs in the K frame code stream encoded in the intra-frame encoding mode.
Step 106: under the condition that a K frame code stream coded in an intra-frame coding mode has packet loss or errors, the decoding equipment sends packet information to the video acquisition equipment, wherein the packet information comprises: and identification information of lost packets or error packets in the K frame code stream coded in the intra-frame coding mode.
In the embodiment of the present invention, a decoding device detects that a packet loss or an error occurs in a K frame code stream encoded in an intra-frame encoding mode, and in this case, the decoding device sends packet information to a video acquisition device, where the packet information includes: and identification information of lost packets or error packets in the K frame code stream coded in the intra-frame coding mode. It should be noted that, in general, an encoded K frame code stream is transmitted in the form of a data packet, a whole section of the K frame code stream is divided into a plurality of packets for transmission, each packet has identification information representing the uniqueness of the packet, so that when a packet is lost or has an error, a decoding device can know which packet is lost or has an error according to the identification information, and then the identification information is sent to a video acquisition device as packet information.
Step 107: and the decoding equipment receives the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding.
In the embodiment of the invention, after the decoding equipment sends the packet information, the decoding equipment receives the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding in the cache region of the video acquisition equipment.
Optionally, after the decoding device sends the packet information, the video capture device needs to perform the following steps:
step t 1: the video acquisition equipment receives the packet information;
step 2: and the video acquisition equipment sends a packet corresponding to the identification information in the K frame code stream after the WZ coding to the decoding equipment according to the identification information.
In the embodiment of the invention, the WZ coded K frame code stream stored in the cache region of the video acquisition equipment is also stored in a packet form and corresponds to each piece of identification information, and after the video acquisition equipment receives the packet information, the packet corresponding to the identification information in the WZ coded K frame code stream can be sent to the decoding equipment according to the identification information.
Step 108: and the decoding equipment corrects the error according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information.
In the embodiment of the invention, after the decoding equipment receives the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding in the cache region of the video acquisition equipment, the decoding equipment corrects the error according to the code stream and the previously received K frame code stream which is coded in the intra-frame coding mode, thereby obtaining the high-quality side information. Compared with the prior art, the high-quality side information is more excellent in quality and clearer in image, and the side information is formed only on the basis of K frames.
Optionally, step 108 specifically includes:
step v 1: decoding by the decoding equipment according to the K frame code stream coded in the intra-frame coding mode to obtain damaged side information corresponding to the damaged K frame;
step v 2: the decoding equipment carries out DWT transformation on the damaged side information so as to quantize each transformed sub-band coefficient;
step v 3: the decoding equipment extracts a bit plane from the quantized damaged side information to obtain side information bit plane data corresponding to the damaged side information;
step v 4: the decoding equipment performs channel decoding on the K frames of code streams corresponding to the identification information to obtain bit plane data of the K frames corresponding to the identification information;
step v 5: and the decoding equipment corrects the damaged side information bit plane data by using the bit plane data of the K frames corresponding to the identification information to obtain high-quality side information.
In the embodiment of the invention, after obtaining the K frame code stream corresponding to the identification information, the decoding device firstly decodes the K frame code stream encoded in the intra-frame encoding mode to obtain the damaged side information corresponding to the damaged K frame; and then, the decoding equipment adopts an error concealment technology to obtain a damaged K frame image with better quality, and determines the damaged side information corresponding to the damaged K frame image.
Then, the decoding device performs DWT conversion on the damaged side information to quantize each transformed sub-band coefficient, extracts a bit plane from the quantized damaged side information to obtain bit plane data of the damaged side information, performs channel decoding on the received K frame code stream corresponding to the identification information to obtain bit plane data of the K frame corresponding to the identification information, and finally, performs error correction on the damaged side information bit plane data by using the bit plane data of the K frame corresponding to the identification information, namely, performs error correction on the image of the damaged K frame, so that the obtained side information has higher quality and is clearer compared with the side information formed only based on the K frame at present.
Step 109: and decoding and constructing the acquired video data by the decoding equipment according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
In the embodiment of the invention, after the high-quality side information is obtained, the decoding equipment can decode and construct the collected video data according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
Optionally, in the foregoing process, if the decoding device receives a K frame code stream encoded in an intra-frame coding mode and a WZ frame code stream encoded in a WZ coding mode; detecting that no packet loss or error occurs in a K frame code stream coded in an intra-frame coding mode; the decoding device decodes and constructs the collected video data according to the K frame code stream and the WZ frame code stream coded in the intra-frame coding mode in the same manner as the prior art.
The above entire process can be more intuitively shown in fig. 3, and with reference to fig. 3, a schematic diagram of a transmission model of distributed multi-view video transmission based on layer repair according to an embodiment of the present invention is shown, where the WZ coding of a WZ frame is the same as that in the prior art, and is not shown, and is only shown for the case of a K frame, when the K frame adopts HEVC intra-frame coding, a WZ encoder is also used to perform DWT change, quantization and channel coding on the K frame, and then the K frame is stored in a buffer area, the K frame of HEVC intra-frame coding is transmitted to a decoding end through a packet loss channel, when the decoding end detects that the K frame of HEVC intra-frame coding has packet loss or error, the decoding end transmits packet information to the encoding end, and after the encoding end receives the K frame of WZ coding corresponding to the packet information in the buffer area, the decoding end transmits the K frame of HEVC intra-frame to the decoding end; in addition, the encoding end can also estimate the code rate according to the code rate sent by the decoding end to obtain the WZ encoded K frame corresponding to the lost or error K frame; a decoding end adopts an error concealment technology to decode and reconstruct a quality-damaged image for the received HEVC intra-frame coded K frame; then, error concealment is carried out to obtain an image with better quality, and the image with better quality is used as the side information of the current K frame; meanwhile, after the same DWT conversion, quantization and bit plane extraction are carried out on the side information of the current K frame by the WZ decoder at the decoding end, side information bit plane data corresponding to the bit plane of the current K frame are obtained, and finally, the damaged side information bit plane data are corrected by the WZ decoder according to the received bit plane data of the WZ coded K frame corresponding to the identification information, so that side information with higher quality is obtained, and the technical scheme of the invention is realized.
In order to verify the validity of the method of the invention, the following simulation tests were carried out:
referring to fig. 4, a graph of code rate values under the condition of comparing three code rates in the embodiment of the present invention is shown, specifically:
the first scheme is as follows: the lowest code rate;
scheme II: actual code rate;
the third scheme is as follows: the method of the invention.
In the figure, the vertical axis represents the Rate (Rate, in bps), and the horizontal axis represents the frame number (No. of Kendo) of the video sequence, and rates of the 1 st frame to the 50 th frame are obtained. The curve composed of the plus sign and the dash-dot line is the lowest code rate value of the scheme 1, the curve composed of the square and the dash-dot line is the actual code rate value of the scheme 2, and the curve composed of the star and the dash-dot line is the code rate estimation value of the scheme 3. Therefore, the code rate value of the method is lower and better.
Referring to fig. 5, a graph of decoded PSNR values comparing decoded video quality under three conditions in the embodiment of the present invention is shown, specifically:
the first scheme is as follows: k frames are not lost;
scheme II: k frames are lost but not repaired;
the third scheme is as follows: the method of the invention.
In the figure, the vertical axis represents the PSNR value of the image quality level in dB, and the horizontal axis represents the frame number of the video sequence, and the decoding quality PSNR values of the 1 st frame to the 20 th frame are obtained. The curve consisting of the star and the dash-and-dot line is the decoded PSNR value without loss for case 1, the curve consisting of the triangle and the dash-and-dot line is the decoded PSNR value without repair for case 2, and the curve consisting of the circle and the dash-and-dot line is the decoded PSNR value for case 3 using the method of the present invention. Therefore, the decoded video quality of the method of the invention is higher and better.
Referring to fig. 6, a schematic diagram of a rate-distortion curve comparing the K frame decoded video quality under three conditions in the embodiment of the present invention is shown, specifically:
the first scheme is as follows: k frames are not lost;
scheme II: k frames are lost but not repaired;
the third scheme is as follows: the layer repair method of the present invention.
In the figure, the vertical axis represents the PSNR value of the image quality in dB, and the horizontal axis represents the code rate value required for video sequence transmission. Wherein, the dotted line is a rate-distortion curve when the K frame in the scheme 1 is not lost, the curve formed by the circle and the dotted line is a rate-distortion curve when the K frame in the scheme 2 is lost (loss rate 5%) but not repaired, and the curve formed by the box and the dotted line is a rate-distortion curve when the K frame in the scheme 2 is lost (loss rate 10%) but not repaired. The curve consisting of circles and realizations is the rate-distortion curve for case 3 when K frames are lost (loss rate 5%) and with repair, and the curve consisting of boxes and realizations is the rate-distortion curve for case 3 when K frames are lost (loss rate 10%). Therefore, the K frame decoding video quality of the method is higher and better.
Referring to fig. 7, a schematic diagram of a rate-distortion curve comparing the decoded video quality of WZ frames under three conditions in the embodiment of the present invention is shown, specifically:
the first scheme is as follows: k frames are not lost;
scheme II: k frames are lost but not repaired;
the third scheme is as follows: the layer repair method of the present invention.
In the figure, the vertical axis represents the PSNR value of the image quality in dB, and the horizontal axis represents the code rate value required for video sequence transmission. Wherein, the dotted line is a rate-distortion curve when the K frame in the scheme 1 is not lost, the curve formed by the circle and the dotted line is a rate-distortion curve when the K frame in the scheme 2 is lost (loss rate 5%) but not repaired, and the curve formed by the box and the dotted line is a rate-distortion curve when the K frame in the scheme 2 is lost (loss rate 10%) but not repaired. The curve consisting of circles and realizations is the rate-distortion curve for case 3 when K frames are lost (loss rate 5%) and with repair, and the curve consisting of boxes and realizations is the rate-distortion curve for case 3 when K frames are lost (loss rate 10%). Therefore, the WZ frame decoding video quality of the method is higher and better.
By the embodiment, the video acquisition equipment performs WZ coding on the K frame and the WZ frame in the acquired video data in parallel and codes the K frame in an intra-frame coding mode; then storing the K frame code stream after the WZ coding in a cache region; the video acquisition equipment sends the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode to the decoding equipment; the method comprises the steps that when packet loss or errors occur in K frame code streams coded in an intra-frame coding mode, a decoding device sends packet information to a video collecting device, and the decoding device receives the K frame code streams corresponding to identification information in the K frame code streams coded in the WZ mode; the decoding equipment corrects the error according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information; and decoding and constructing the acquired video data by the decoding equipment according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode. According to the scheme, under the condition that the K frame is lost or wrong, the error of the K frame is corrected by utilizing the WZ coded K frame in the cache to obtain high-quality side information, correct decoding blocks of the K frame without loss are fully utilized, the transmission quality of the K frame is obviously improved, the robust transmission of a DMVC system is realized, and the advantage that the compressibility of the K frame of the DMVC system is not changed is achieved. The scheme of the invention greatly improves the decoding quality of the K frame, compresses the code rate of the WZ frame, further improves the overall RD performance of the DMVC system and ensures that the scheme of the invention has stronger expansibility and adaptability.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The embodiments of the present invention have been described in connection with the accompanying drawings, and the principles and embodiments of the present invention are described herein using specific examples, which are provided only to help understand the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (8)
1. A method for distributed multi-view video transmission based on layer repair, the method comprising:
the video acquisition equipment performs WZ encoding on a K frame and a WZ frame in parallel in acquired video data and encodes the K frame in an intra-frame encoding mode;
the video acquisition equipment stores the K frame code stream after the WZ coding in a cache region;
the video acquisition equipment sends the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode to decoding equipment through a packet loss channel;
the decoding equipment receives a K frame code stream and a WZ frame code stream which are coded in an intra-frame coding mode;
the decoding equipment detects whether the K frame code stream coded in the intra-frame coding mode has packet loss or not;
under the condition that packet loss occurs in the K frame code stream coded in the intra-frame coding mode, the decoding device sends packet information to the video acquisition device, wherein the packet information comprises: the identification information of the lost packet in the K frame code stream coded in the intra-frame coding mode;
the video acquisition equipment receives the packet information;
the video acquisition equipment sends a packet corresponding to the identification information in the WZ-coded K frame code stream to the decoding equipment according to the identification information;
the decoding equipment receives the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding;
the decoding equipment corrects the error according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information;
and the decoding equipment decodes and constructs the acquired video data according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
2. The method of claim 1, wherein the video capture device WZ encodes the K frames, comprising:
the video acquisition equipment carries out DWT (discrete wavelet transform) on the K frame and then quantizes each transformed sub-band coefficient;
and the video acquisition equipment extracts a bit plane from the quantized K frames and performs channel coding on the bit plane so as to realize WZ coding on the K frames.
3. The method according to claim 2, wherein the decoding device corrects the error according to the K frame code stream encoded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain the high-quality side information, and the method comprises:
the decoding equipment decodes the K frames of code streams coded in the intra-frame coding mode to obtain damaged side information corresponding to damaged K frames;
the decoding equipment carries out DWT transformation on the damaged side information and then quantizes each transformed sub-band coefficient;
the decoding device extracts a bit plane from the quantized damaged side information to obtain side information bit plane data corresponding to the damaged side information;
the decoding device performs channel decoding on the K frames of code streams corresponding to the identification information to obtain bit plane data of the K frames corresponding to the identification information;
and the decoding equipment corrects the bit plane data of the side information corresponding to the damaged side information by using the bit plane data of the K frames corresponding to the identification information to obtain the side information with high quality.
4. The method according to claim 3, wherein the decoding device performs decoding according to the K frame code stream encoded in the intra-frame coding mode to obtain damaged side information corresponding to a damaged K frame, and the method comprises:
the decoding equipment adopts an error concealment technology to decode the K frame code stream coded in the intra-frame coding mode, and rebuilds a damaged K frame image;
and the decoding equipment adopts an error concealment technology to obtain the damaged K frame image with better quality, and determines the damaged K frame image as the damaged side information corresponding to the damaged K frame image.
5. The method of claim 1, wherein the video capture device transmits the K frame code stream and the WZ frame code stream encoded in the intra-frame encoding mode to a decoding device, and wherein the method further comprises:
the decoding equipment receives the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode;
the decoding equipment detects that no packet loss occurs in the K frame code stream coded in the intra-frame coding mode;
and the decoding equipment decodes and constructs the collected video data according to the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
6. A system for distributed multi-view video transmission based on layer repair, the system comprising: video capture device and decoding equipment, video capture device includes: the device comprises a parallel coding module, a cache storage module, a code stream sending module, a packet information receiving module and a corresponding code stream sending module;
the decoding apparatus includes: the device comprises a code stream receiving module, a detection module, a packet information sending module, a corresponding code stream receiving module, an error correction module and a decoding construction module;
the parallel coding module is used for carrying out WZ coding on a K frame and a WZ frame in the collected video data in parallel and coding the K frame in an intra-frame coding mode;
the cache storage module is used for storing the K frame code stream after the WZ coding in a cache region;
the code stream sending module is used for sending the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode to decoding equipment through a packet loss channel;
the packet information receiving module is used for receiving packet information;
the code stream sending corresponding module is used for sending a packet corresponding to the identification information in the WZ coded K frame code stream to the decoding equipment according to the identification information;
the code stream receiving module is used for receiving the K frame code stream and the WZ frame code stream which are coded in an intra-frame coding mode;
the detection module is used for detecting whether the K frame code stream coded in the intra-frame coding mode has packet loss or not;
the packet information sending module is configured to send packet information to the video acquisition device when the K frame code stream encoded in the intra-frame encoding mode is lost, where the packet information includes: the identification information of the lost packet in the K frame code stream coded in the intra-frame coding mode;
the code stream receiving corresponding module is used for receiving the K frame code stream corresponding to the identification information in the K frame code stream after the WZ coding;
the error correction module is used for correcting errors according to the K frame code stream coded in the intra-frame coding mode and the K frame code stream corresponding to the identification information to obtain high-quality side information;
and the decoding construction module is used for decoding and constructing the acquired video data according to the high-quality side information, the K frame code stream coded in the intra-frame coding mode and the WZ frame code stream coded in the WZ coding mode.
7. The system of claim 6, wherein the parallel encoding module comprises:
the variable quantization submodule is used for carrying out DWT (discrete wavelet transform) on the K frame and then quantizing each transformed sub-band coefficient;
and the extraction coding sub-module is used for extracting bit planes of the quantized K frames and carrying out channel coding on the bit planes so as to realize WZ coding on the K frames.
8. The system of claim 7, wherein the error correction module comprises:
the damaged side information decoding submodule is used for decoding according to the K frame code stream coded in the intra-frame coding mode to obtain damaged side information corresponding to the damaged K frame;
the damaged side information variation submodule is used for carrying out DWT (discrete wavelet transform) on the damaged side information and then quantizing each transformed sub-band coefficient;
the damaged side information extraction submodule is used for extracting a bit plane from the quantized damaged side information to obtain side information bit plane data corresponding to the damaged side information;
the corresponding code stream decoding submodule is used for carrying out channel decoding on the K frame code stream corresponding to the identification information to obtain bit plane data of the K frame corresponding to the identification information;
and the error correction submodule is used for correcting the bit plane data of the side information corresponding to the damaged side information by using the bit plane data of the K frame corresponding to the identification information to obtain the side information with high quality.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010038328.8A CN111212286B (en) | 2020-01-14 | 2020-01-14 | Distributed multi-view video transmission method and system based on layer repair |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010038328.8A CN111212286B (en) | 2020-01-14 | 2020-01-14 | Distributed multi-view video transmission method and system based on layer repair |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111212286A CN111212286A (en) | 2020-05-29 |
| CN111212286B true CN111212286B (en) | 2022-02-01 |
Family
ID=70789051
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010038328.8A Active CN111212286B (en) | 2020-01-14 | 2020-01-14 | Distributed multi-view video transmission method and system based on layer repair |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111212286B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112565765B (en) * | 2020-12-11 | 2021-11-09 | 四川大学 | Multi-point video hierarchical coding and decoding method and transmission device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740081A (en) * | 2012-06-21 | 2012-10-17 | 西南交通大学 | Method for controlling transmission errors of multiview video based on distributed coding technology |
| CN103533387A (en) * | 2013-10-21 | 2014-01-22 | 腾讯科技（深圳）有限公司 | Live video control method, equipment and system |
| CN106060567A (en) * | 2016-06-16 | 2016-10-26 | 四川大学 | Wavelet domain distributed multi-view video coding based on layered WZ frame |
| CN106534873A (en) * | 2016-11-29 | 2017-03-22 | 四川大学 | Distributed video coding key frame loss error protection scheme based on wavelet domain |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130117796A1 (en) * | 2011-11-07 | 2013-05-09 | Empire Technology Development Llc | Redundant key frame transmission |
-
2020
- 2020-01-14 CN CN202010038328.8A patent/CN111212286B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740081A (en) * | 2012-06-21 | 2012-10-17 | 西南交通大学 | Method for controlling transmission errors of multiview video based on distributed coding technology |
| CN103533387A (en) * | 2013-10-21 | 2014-01-22 | 腾讯科技（深圳）有限公司 | Live video control method, equipment and system |
| CN106060567A (en) * | 2016-06-16 | 2016-10-26 | 四川大学 | Wavelet domain distributed multi-view video coding based on layered WZ frame |
| CN106534873A (en) * | 2016-11-29 | 2017-03-22 | 四川大学 | Distributed video coding key frame loss error protection scheme based on wavelet domain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111212286A (en) | 2020-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10701362B2 (en) | High throughput significance map processing for CABAC in HEVC | |
| WO2018103568A1 (en) | Methods of encoding and decoding cloud desktop content, device, and system | |
| CN102045557B (en) | Video encoding and decoding method and video encoding device and decoding device thereof | |
| CN100512443C (en) | Distributive vide frequency coding method based on self adaptive Hashenhege type vector quantization | |
| KR101631280B1 (en) | Method and apparatus for decoding image based on skip mode | |
| KR20110025444A (en) | Method and apparatus for generating bitstream based on syntax element | |
| US20120230422A1 (en) | Method and System Using Prediction and Error Correction for the Compact Representation of Quantization Matrices In Video Compression | |
| CN111212286B (en) | Distributed multi-view video transmission method and system based on layer repair | |
| CN104767997B (en) | A kind of visual signature coding method and device towards video | |
| CN116527909A (en) | Method, apparatus, device, storage medium and program product for transmitting coding parameters | |
| JP2007166618A (en) | Video data encoder, video data decoder, video data encoding method, and video data decoding method | |
| US9307257B2 (en) | Method for improving compression efficiency of distributed source coding using intra-band information | |
| CN102710943A (en) | Real-time video transmission method based on forward error correction encoding window expanding | |
| WO2022110870A1 (en) | Image encoding and decoding method, encoding and decoding apparatus, encoder, and decoder | |
| Peng et al. | An optimized algorithm based on generalized difference expansion method used for HEVC reversible video information hiding | |
| KR101230856B1 (en) | Method for deciding intra-coded frame mode and dvc codec using the same | |
| KR101710622B1 (en) | Method and apparatus for encoding/decoding image based on skip mode | |
| KR102020953B1 (en) | Image Reencoding Method based on Decoding Data of Image of Camera and System thereof | |
| CN112218086A (en) | Encoding method, decoding method, transmission method, encoding device, decoding device, and system | |
| Huchet et al. | Adaptive source representation for distributed video coding | |
| CN109510978B (en) | Data processing performance detection method and device | |
| Li et al. | Distributed video coding based on conditional entropy hash | |
| CN112004095B (en) | Storage-medium-free still image interframe lossless recovery method | |
| Xu et al. | Sub-sampling framework of distributed video coding | |
| CN107835422B (en) | HEVC multi-description image coding algorithm based on significance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |