CN103916714A - Packet-loss resistance preprocessing and recovery method for wireless network video stream - Google Patents
Packet-loss resistance preprocessing and recovery method for wireless network video stream Download PDFInfo
- Publication number
- CN103916714A CN103916714A CN201410091095.2A CN201410091095A CN103916714A CN 103916714 A CN103916714 A CN 103916714A CN 201410091095 A CN201410091095 A CN 201410091095A CN 103916714 A CN103916714 A CN 103916714A
- Authority
- CN
- China
- Prior art keywords
- pixel
- packet
- loss
- video
- sprite
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a packet-loss resistance preprocessing and recovery method for a wireless network video stream. Before encoding is carried out, a video is preprocessed through the packet-loss resistance preprocessing and recovery method, and the packet-loss resistance capacity of the video in the network transmission process is improved. The packet-loss resistance preprocessing and recovery method comprises the steps that adjacent pixels in a video image are separated to be distributed to different sub-images so as to be encoded; when packet loss is detected at a video stream receiving end, image distortion caused by the packet loss is restored to the maximum extent through interpolation carried out on information of pixels around lost pixels according to the redundancy characteristic of information of the adjacent pixels, and video stream media service quality is improved.
Description
Technical field
The invention belongs to wireless network streaming media transmission field, relate to a kind of badly at network, under the serious environment of packet loss, solve video source preliminary treatment and the loss recovery method of video quality degradation.
Background technology
Along with the development of the universal and multimedia technology of broadband networks, the application of stream media technology is also more and more extensive, as digital broadcasting service, video conference business, video-on-demand service, IPTV, mobile flow medium service etc.The common feature of these business, is all that multi-medium data is cut apart after encapsulation packing, by the communication network of bottom, carries out the distribution of data.Due to the transmission unreliability of bottom-layer network, will inevitably there is the situations such as packet generation error code, loss.Now, can propose the feedback request that packet loss retransmits by receiving terminal, transmitting terminal resends loss or wrong packet to receiving terminal, solves this problem.
Do packet loss and retransmit, just need receiving terminal, within the judgement time, unreceived packet is carried out to request repeat, will increase like this time delay of streaming media service.Under the severe wireless environment of some network environments, adopt packet loss to retransmit and still can not recover whole packet losses completely, and same bag repeatedly re-transmission can cause time delay significantly to increase, the user who has a strong impact on such as video conference of real time flow medium business etc. experiences.Therefore needing has a kind of method, can under the environment that has certain packet loss, recover as much as possible original image, has both guaranteed the ageing of video traffic, has guaranteed again the content quality of business.
Summary of the invention
technical problem:the present invention is directed to the video quality degradation that the streaming media service under severe wireless network environment seriously causes due to packet loss, provide a kind of and still there is higher video quality under higher packet loss environment, improve the anti-dropout preliminary treatment of wireless network video stream and restoration methods that user video is experienced.
technical scheme:the anti-dropout preliminary treatment of wireless network video stream of the present invention and restoration methods, comprise the following steps:
1) transmitting terminal is divided into pixel in video pictures in different sprites according to certain rule, makes in original image neighbor all in different sprites;
2) each sprite obtaining in described step 1) is carried out to Video coding and obtain video data stream, described data flow is divided into packet, and by recording the head of sprite partitioning scheme and location of pixels field in packet marking, be then transferred to receiving terminal;
3) receiving terminal is to the decoded packet data receiving, and then obtained the video pictures pixel coverage of the packet that success has received according to packet head, thereby draws the video pictures pixel coverage in the packet of loss;
4) the synthetic final picture of each sprite that the sprite partitioning scheme that receiving terminal is recorded according to packet head obtains decoding, then according to the pixel value being correctly decoded, adopt the mode of interpolation to recover to lose the data of pixel, the video pictures being restored.
In the preferred version of the inventive method, the idiographic flow of step 1) is:
Former video pictures is divided into PxQ little rectangular block, then by the pixel (x of original picture video, y) be mapped to (x mod P, y mod Q) pixel (floor (x/P) in individual little rectangular block, floor (y/Q)) on, the little rectangular block finally obtaining is sprite, wherein, P is the columns of the little rectangular block cut apart, Q is the line number of the little rectangular block cut apart, and x is the abscissa of original picture pixel, and y is the ordinate of original picture pixel, mod is complementation, floor(*) be downward rounding operation.
The step 2 of the above-mentioned preferred version of the inventive method) in, on packet, the head of mark comprises following information: former video pictures is divided into the value of columns P and the line number Q of little rectangular block, the location of pixels scope that sprite comprises.
In the preferred version of the inventive method, step 2) in based on RTP transmission by packet to receiving terminal.
In the preferred version of the inventive method, in step 3), receiving terminal is set up the video pictures pixel coverage of a tag slot record packet that success has received for every two field picture, in the time that this frame will be play, using unwritten pixel coverage in tag slot as the pixel data scope of losing.
In the preferred version of the inventive method, in step 4), by after synthetic sprite final picture, check the pixel coverage of losing, for the pixel of each loss, according to its neighbor, take the mode of interpolation to recover the data of losing pixel.
In the step 4) of the above-mentioned preferred version of the inventive method, the flow process of taking the mode of interpolation to recover the data of losing pixel is:
A) calculate respectively each loss pixel and its nearest distance dx1 that receives pixel M, N, P, Q on four direction up and down, dx2, dy1, dy2, wherein pixel M, N are at the left and right sides of this loss pixel, pixel P, Q, at the upper and lower sides of this loss pixel, define horizontal distortion distance dx=dx1+dx2, and longitudinal distortion is apart from dy=dy1+dy2, distortion distance d=min (dx, dy), wherein min (dx, dy) represents to get smaller in dx and dy;
B) all loss pixels are sorted by the size of distortion distance d, get the loss pixel of d minimum, carry out interpolation according to linear interpolation method, if dx≤dy, lose color-values Y=YM+dx1* (the YN-YM)/dx of pixel, wherein YM is the color-values of pixel M, and YN is the color-values of pixel N; If dx>dy, loses color-values Y=YP+dy1* (the YQ-YP)/dy of pixel, wherein YP is the color-values of pixel P, and YQ is the color-values of pixel Q;
C) pixel interpolation being completed joins and receives in pixel set, judges whether that the data of losing in addition pixel do not recover, and returns in this way step a), otherwise completes the demosaicing of losing pixel.
In the inventive method, by neighbor in video original image is separated in different sprites, carry out respectively video compression coding and cut apart transmission respectively cutting apart sprite, utilize the redundancy of information between each sprite, in the time there is packet loss in transmission, video receiver can utilize the data in other sprite to carry out the sub-image data that demosaicing is lost.Because neighbor is in difference coding transmission in different sprites, receiving terminal can utilize the adjacent pixel of not losing to utilize the method for interpolation to recover in the time packet loss being detected.
beneficial effect:the present invention compared with prior art, has the following advantages:
The present invention has introduced video pre-filtering mechanism, before Video coding, first neighbor in image is separated to code division transmission respectively in different sprites, utilize the correlation between neighbor, when after the coded data packet loss of certain block in a sprite, can utilize the neighbor of preliminary treatment in other sprites to recover the pixel value of loss by the method for interpolation.The video qualities such as a large amount of mosaics that bring with the general streaming media video system packet loss serious phenomenon that declines, the present invention can make Web Video Service under higher packet loss environment, still have higher video quality, improves user's video tastes.
Accompanying drawing explanation
Fig. 1 is the structure in packet preliminary treatment packet header.
Fig. 2 is that 2x2 is cut apart sprite and original picture mapping scheme under pattern.
Fig. 3 is the flow chart of the inventive method.
Embodiment
Below in conjunction with embodiment and Figure of description, technical solution of the present invention is described in detail.
As shown in Figure 3, the anti-dropout preliminary treatment of wireless network video stream of the present invention and restoration methods comprise the following steps:
1) transmitting terminal collects after a frame video original image, and original video picture segmentation is become to several sprites, and neighbor in original picture will be separated in different sprites.
2) transmitting terminal adopts H.264/AVC standard to carry out Video coding to each sprite obtaining in step 1), data flow after coding is divided into a certain size and is adapted at the packet of transmission over networks, and by recording the head of sprite partitioning scheme, sprite numbering and location of pixels field in packet marking, for receiving terminal carries out packet splicing, packet loss demosaicing provides necessary reference information.By the packet of accomplishing fluently head with the transmission of packing of the streaming media communication standard of RFC3984 regulation.
3) receiving terminal receiving end/sending end data the streaming media communication standard according to RFC3984 regulation are isolated packet, extract the information of packet head, utilize H.264/AVC decoder to carry out video decode to packet inclusion, comprise the position of sprite pixel in original picture according to this packet of information reverting in packet head, deposit in the lump view data after decoding and positional information in buffer queue medium to be played.
4), in the time that video frame will be play, receiving terminal takes out and allly stabs consistent picture data with this frame time from buffer queue, according to acquired sprite partitioning scheme, sprite positional information, each sprite is reverted back to original image.For the Pixel Information lacking due to packet loss in image, adopt the mode of interpolation to recover loss Pixel Information, the video pictures after finally being recovered according to losing the pixel Pixel Information that is correctly decoded acquisition around.
In an embodiment of the inventive method, in step 1), the mode of transmitting terminal divided video original image is: former video pictures is divided into PxQ little rectangular block, then by the pixel (x of original picture video, y) be mapped to (x mod P, y mod Q) pixel (floor (x/P) in individual little rectangular block, floor (y/Q)) upper, the little rectangular block finally obtaining is sprite.Figure 2 shows that the pixel segmentation method that picture segmentation is become to 2x2 little rectangular block, in figure, each square represents a pixel.
In above-described embodiment of the inventive method, transmitting terminal adopts H.264/AVC standard to carry out Video coding to each sprite, the Data Segmentation obtaining after coding is become to be adapted to the packet of transmission over networks, for each packet adds packet head, the length of head is 48bit, as shown in Figure 1, everybody is defined as follows section structure:
Bit 0 ~ 3: Rect_P, the columns P of the little rectangular block being divided into
Bit 4 ~ 7: Rect_Q, the line number Q of the little rectangular block being divided into
Bit 8 ~ 15: subimg_no, the sequence number of sprite
Bit 16 ~ 31: startpos, comprises the original position of sprite pixel fragment in packet
Bit 32 ~ 47: endpos, comprises the end position of sprite pixel fragment in packet
In an embodiment of the inventive method, transmitting terminal is stamped packet after head, packet is carried out to transport layer encapsulation according to the H.264 Streaming Media load form stipulating in RFC3984 agreement, network packet transmission adopts realtime transmission protocol RTP, and Signalling exchange adopts RTCP Real-time Transport Control Protocol RTCP.
In an embodiment of the inventive method, receiving terminal is set up the video pictures pixel coverage of a corresponding tag slot record packet that success has received to each frame.When receiving terminal receives after packet, extract and analyze packet head, obtain the pixel coverage of video pictures in this packet, then video content in packet decoded and deposit buffer queue in, upgrade afterwards the tag slot of the corresponding frame of this video pictures.
In an embodiment of the inventive method, in step 4), in the time that video frame will be play, receiving terminal takes out and allly stabs consistent sub-image data with this frame time from buffer queue, and each sprite is reverted back to original image.From tag slot corresponding to this frame, obtain all location of pixels of successful receipt decoding of this frame, thereby draw all location of pixels of losing due to packet loss.Utilize and lose pixel acquired Pixel Information around, adopt the method for interpolation to recover to lose pixel data.
In step 4), the demosaicing method of picture loss pixel is:
A) calculate respectively each loss pixel and its nearest distance dx1 that receives pixel M, N, P, Q on four direction up and down, dx2, dy1, dy2, wherein pixel M, N are at the left and right sides of this loss pixel, pixel P, Q, at the upper and lower sides of this loss pixel, define horizontal distortion distance dx=dx1+dx2, and longitudinal distortion is apart from dy=dy1+dy2, distortion distance d=min (dx, dy), wherein min (dx, dy) represents to get smaller in dx and dy;
B) all loss pixels are sorted by the size of distortion distance d, get the loss pixel of d minimum, carry out interpolation according to linear interpolation method, if dx≤dy, lose color-values Y=YM+dx1* (the YN-YM)/dx of pixel, wherein YM is the color-values of pixel M, and YN is the color-values of pixel N; If dx>dy, loses color-values Y=YP+dy1* (the YQ-YP)/dy of pixel, wherein YP is the color-values of pixel P, and YQ is the color-values of pixel Q;
C) pixel interpolation being completed joins and receives in pixel set, judges whether that the data of losing in addition pixel do not recover, and returns in this way step a), otherwise completes the demosaicing of losing pixel.
Claims (7)
1. the anti-dropout preliminary treatment of wireless network video stream and a restoration methods, is characterized in that, the method comprises the following steps:
1) transmitting terminal is divided into pixel in video pictures in different sprites according to certain rule, makes in original image neighbor all in different sprites;
2) each sprite obtaining in described step 1) is carried out to Video coding and obtain video data stream, described data flow is divided into packet, and by recording the head of sprite partitioning scheme and location of pixels field in packet marking, be then transferred to receiving terminal;
3) receiving terminal is to the decoded packet data receiving, and then obtained the video pictures pixel coverage of the packet that success has received according to packet head, thereby draws the video pictures pixel coverage in the packet of loss;
4) the synthetic final picture of each sprite that the sprite partitioning scheme that receiving terminal is recorded according to packet head obtains decoding, then according to the pixel value being correctly decoded, adopt the mode of interpolation to recover to lose the data of pixel, the video pictures being restored.
2. the anti-dropout preliminary treatment of wireless network video stream according to claim 1 and restoration methods, is characterized in that, the idiographic flow of described step 1) is:
Former video pictures is divided into PxQ little rectangular block, then by the pixel (x of original picture video, y) be mapped to (x mod P, y mod Q) pixel (floor (x/P) in individual little rectangular block, floor (y/Q)) on, the little rectangular block finally obtaining is sprite, wherein, P is the columns of the little rectangular block cut apart, Q is the line number of the little rectangular block cut apart, and x is the abscissa of original picture pixel, and y is the ordinate of original picture pixel, mod is complementation, floor(*) be downward rounding operation.
3. the anti-dropout preliminary treatment of wireless network video stream according to claim 2 and restoration methods, it is characterized in that, described step 2) in, on packet, the head of mark comprises following information: former video pictures is divided into the value of columns P and the line number Q of little rectangular block, the location of pixels scope that sprite comprises.
4. according to the anti-dropout preliminary treatment of wireless network video stream and restoration methods described in claim 1,2 or 3, it is characterized in that described step 2) in based on RTP transmission by packet to receiving terminal.
5. according to the anti-dropout preliminary treatment of wireless network video stream and restoration methods described in claim 1,2 or 3, it is characterized in that, in described step 3), receiving terminal is set up the video pictures pixel coverage of a tag slot record packet that success has received for every two field picture, in the time that this frame will be play, using unwritten pixel coverage in tag slot as the pixel data scope of losing.
6. according to the anti-dropout preliminary treatment of wireless network video stream and restoration methods described in claim 1,2 or 3, it is characterized in that, in described step 4), sprite is being synthesized after final picture, check the pixel coverage of losing, for the pixel of each loss, according to its neighbor, take the mode of interpolation to recover the data of losing pixel.
7. the anti-dropout preliminary treatment of wireless network video stream according to claim 6 and restoration methods, is characterized in that, in described step 4), the flow process of taking the mode of interpolation to recover the data of losing pixel is:
A) calculate respectively each loss pixel and its nearest distance dx1 that receives pixel M, N, P, Q on four direction up and down, dx2, dy1, dy2, wherein pixel M, N are at the left and right sides of this loss pixel, pixel P, Q, at the upper and lower sides of this loss pixel, define horizontal distortion distance dx=dx1+dx2, and longitudinal distortion is apart from dy=dy1+dy2, distortion distance d=min (dx, dy), wherein min (dx, dy) represents to get smaller in dx and dy;
B) all loss pixels are sorted by the size of distortion distance d, get the loss pixel of d minimum, carry out interpolation according to linear interpolation method, if dx≤dy, lose color-values Y=YM+dx1* (the YN-YM)/dx of pixel, wherein YM is the color-values of pixel M, and YN is the color-values of pixel N; If dx>dy, loses color-values Y=YP+dy1* (the YQ-YP)/dy of pixel, wherein YP is the color-values of pixel P, and YQ is the color-values of pixel Q;
C) pixel interpolation being completed joins and receives in pixel set, judges whether that the data of losing in addition pixel do not recover, and returns in this way step a), otherwise completes the demosaicing of losing pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410091095.2A CN103916714B (en) | 2014-03-13 | 2014-03-13 | Packet-loss resistance preprocessing and recovery method for wireless network video stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410091095.2A CN103916714B (en) | 2014-03-13 | 2014-03-13 | Packet-loss resistance preprocessing and recovery method for wireless network video stream |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103916714A true CN103916714A (en) | 2014-07-09 |
| CN103916714B CN103916714B (en) | 2017-02-15 |
Family
ID=51042045
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410091095.2A Withdrawn - After Issue CN103916714B (en) | 2014-03-13 | 2014-03-13 | Packet-loss resistance preprocessing and recovery method for wireless network video stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103916714B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105025347A (en) * | 2015-07-21 | 2015-11-04 | 北京暴风科技股份有限公司 | Sending and receiving methods of GOP image groups |
| CN105791827A (en) * | 2016-02-26 | 2016-07-20 | 北京计算机技术及应用研究所 | Video coding method of wireless channel |
| CN108616328A (en) * | 2016-12-12 | 2018-10-02 | 北京视联动力国际信息技术有限公司 | A kind of audio/video flow loss recovery method and audio/video flow transmitting terminal and receiving terminal |
| CN111404650A (en) * | 2020-03-17 | 2020-07-10 | 许昌泛网信通科技有限公司 | Vehicle control message processing method |
| CN115665467A (en) * | 2022-10-02 | 2023-01-31 | 刘晓敏 | Ultra-high-definition video signal transmission system and method based on 5G |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101137060A (en) * | 2007-09-30 | 2008-03-05 | 腾讯科技（深圳）有限公司 | Video transmission method, system and equipment |
| US20110096836A1 (en) * | 2008-06-13 | 2011-04-28 | Einarsson Torbjoern | Packet loss analysis |
| CN102378067A (en) * | 2011-11-21 | 2012-03-14 | 武汉大学 | Robustness mobile video decoding method |
-
2014
- 2014-03-13 CN CN201410091095.2A patent/CN103916714B/en not_active Withdrawn - After Issue
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101137060A (en) * | 2007-09-30 | 2008-03-05 | 腾讯科技（深圳）有限公司 | Video transmission method, system and equipment |
| US20110096836A1 (en) * | 2008-06-13 | 2011-04-28 | Einarsson Torbjoern | Packet loss analysis |
| CN102378067A (en) * | 2011-11-21 | 2012-03-14 | 武汉大学 | Robustness mobile video decoding method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105025347A (en) * | 2015-07-21 | 2015-11-04 | 北京暴风科技股份有限公司 | Sending and receiving methods of GOP image groups |
| CN105025347B (en) * | 2015-07-21 | 2018-10-09 | 暴风集团股份有限公司 | A kind of method of sending and receiving of GOP images group |
| CN105791827A (en) * | 2016-02-26 | 2016-07-20 | 北京计算机技术及应用研究所 | Video coding method of wireless channel |
| CN105791827B (en) * | 2016-02-26 | 2018-07-24 | 北京计算机技术及应用研究所 | A kind of method for video coding of wireless channel |
| CN108616328A (en) * | 2016-12-12 | 2018-10-02 | 北京视联动力国际信息技术有限公司 | A kind of audio/video flow loss recovery method and audio/video flow transmitting terminal and receiving terminal |
| CN111404650A (en) * | 2020-03-17 | 2020-07-10 | 许昌泛网信通科技有限公司 | Vehicle control message processing method |
| CN115665467A (en) * | 2022-10-02 | 2023-01-31 | 刘晓敏 | Ultra-high-definition video signal transmission system and method based on 5G |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103916714B (en) | 2017-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7436454B2 (en) | Method and apparatus for transmitting encoded information based upon piority data in the encoded information | |
| CN101453639B (en) | Encoding, decoding method and system for supporting multi-path video stream of ROI region | |
| US8358704B2 (en) | Frame level multimedia decoding with frame information table | |
| KR100612003B1 (en) | Apparatus for transmitting/receiving bitstream in network and method thereof | |
| CN105704580B (en) | A kind of video transmission method | |
| CN103916714A (en) | Packet-loss resistance preprocessing and recovery method for wireless network video stream | |
| CN101742271B (en) | Method, system and device for transmitting stream media data | |
| CN105874794A (en) | System and method for video multicasting | |
| CN106911699B (en) | Method for realizing I frame retransmission based on RTP protocol | |
| CN107995493A (en) | A kind of multi-description video coding method of panoramic video | |
| CN110740334A (en) | frame-level application layer dynamic FEC encoding method | |
| CN111093083A (en) | Data transmission method and device | |
| CN102158696B (en) | Three-dimensional video transmission method based on expanding window fountain code | |
| CN101867801A (en) | A kind of communication means and communication system | |
| US7856585B2 (en) | Content distribution method, encoding method, reception/reproduction method and apparatus, and program | |
| CN104796793B (en) | Opportunistic Multimedia Dynamic cloud platform and the transmission method of many relaying classification cooperations | |
| CN102378012A (en) | Data hiding-based H.264 video transmission error code recovery method | |
| CN101515934B (en) | Method, device and communication system for forwarding scalable video coded data message | |
| CN101986713B (en) | View synthesis-based multi-viewpoint error-resilient encoding frame | |
| CN103378955A (en) | Data retransmission method, data retransmission system, data retransmission device and data acquisition device | |
| CN101296166B (en) | Method for measuring multimedia data based on index | |
| Belda et al. | Hybrid FLUTE/DASH video delivery over mobile wireless networks | |
| CN1210962C (en) | Active error-preventing method for video image transmission | |
| CN107835422B (en) | HEVC multi-description image coding algorithm based on significance | |
| KR101725345B1 (en) | System and method for retransmitting packet mixing unicasting and broadcasting/multicasting in wireless lan |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| AV01 | Patent right actively abandoned | ||
| AV01 | Patent right actively abandoned | ||
| AV01 | Patent right actively abandoned |
Granted publication date: 20170215 Effective date of abandoning: 20230829 |
|
| AV01 | Patent right actively abandoned |
Granted publication date: 20170215 Effective date of abandoning: 20230829 |