Network video compression fault-tolerant transmission method and system
Technical field
The present invention relates to network multi-media communication technical field, particularly relate to a kind of network video compression fault-tolerant transmission method and system.
Background technology
Along with the progressively foundation of third generation digital radio mobile communications network and rear 3 g mobile communication system, the multimedia data service of transmitted over wireless networks based on video information obtains fast development, people thirst for exploitation and use multimedia on radio channel, and reason is huge market potential and operates easily.But due to the limited bandwidth resources of wireless channel, and exist channel conditions instability cause high error code and network congestion problem, bring serious impact to reconstruction video quality.
Current, the radio mobile communication network bandwidth not only needs processed voice information, also needs the media forms such as process image, music, video, provides web page browsing, videoconference, electronic commerce information service.Wherein, voice and video communication is one of wireless basic service.But wireless channel transmission of video mainly faces 2 points: network congestion and unreliable transmission.To this, domestic and international researcher proposes many schemes.In web impact factor, adopt the buffer queue in management router, processing mode includes by Active queue management, active queue management etc.In Joint Source/channel Coding Design control, existing many schemes, wherein great majority are the codings coming dynamic conditioning information source and passage according to channel and network state.But above-mentioned various method all also exists respective deficiency, to congestion control, do not consider source properties and carry out abandoning of packet; For message sink coding, do not consider that the loss of packet causes because of congested, also have error code to cause.At present, based on wireless transmission of video being also generally faced with video image is fuzzy, mosaic, video playback are very discontinuous, the many difficulties such as large, the video-losing of video playback time delay, the video data volume, these defects are all had higher requirement to Video coding and fault-toleranr technique.At present, employing be all some fault tolerant mechanisms of H264 self, but still can cause the problem occurring that mosaic or image are made mistakes in the event of mistake, especially after there is mistake, mistake can the problem of continuous diffusion.
There is the shortcomings such as bandwidth fluctuation, error code and packet loss due to wireless channel, can cause and occur in decoding end the phenomenon that mosaic or image are made mistakes, if there is an image to make mistakes in a video sequence, those data below just all can be made mistakes, and the problem that mistake can constantly expand.Now transmission over networks video is occurred that the shortcomings such as bandwidth fluctuation, error code and packet loss generally all do not do fault-tolerant processing.If having done also is only the process doing some Error Compensation in decoding end, make current image can obtain complete representing.But like this can not solving error diffusion problem.
In prior art, general video streaming adopts Real-time Transport Protocol transmission.
S4: coding side to the data after decoding end transfer encoding, decoding end to decoding data, according to decoded data to by mistake generating decoded information and feeding back to coding side by described data transmission interface channel;
Error detection and error concealment technology is introduced in decoding end.
Described error detection is in decode procedure, detect the grammer of this each data segment of code stream and the correctness of content, as shown in Figure 3, in the present embodiment, and specifically comprising of described error detection:
S41: the detection carrying out video sequence parameter collection from video sequence level to network abstract layer and video image parameter set;
S42: grammer and the content detection of carrying out video chip level, for judging key message in current video sheet and whether syntactic element meets standard and whether be the parameter that this decoder is supported;
S43: the detection carrying out macro-block level content, for detecting one by one the texture information in code stream, motion vector information and macro block type information, and according to the information errors in the code stream found and information dropout situation, generates corresponding decoded information.In the present embodiment, to the macro block of mistake, generating corresponding decoded information is that the macro block information value on correspondence position is set to 1.
Error concealment is estimation and the recovery of the drop-out caused for error of transmission.Adopt corresponding error concealment algorithm according to concrete type of error, improve the quality of decipher reestablishment image.
In the present embodiment, the corresponding bit of each macro block, namely the corresponding byte of every 8 macro blocks, which offers a saving code stream.In the present embodiment, during macroblock decode success, the decoded information of feedback is 0, when decoding unsuccessfully, and the decoded information of feedback is 1.The size of this wherein each macro block is the block of 16*16, if sub-block has problem, so whole macro block is 1.The information whether decoding of each macro block is correct is determined with a bit, and be correctly 0, mistake is 1.The data volume of transmission can be reduced so further.
S5: the mode that coding side is encoded by the described decoded information adjustment received, is specially: when the decoded information that coding side receives represents that decoding end is correctly decoded, in the present embodiment namely when decoded information is 0, then encodes according to normal mode; When the decoded information that coding side receives represents decoding end decoding error, in the present embodiment namely when decoded information is 1, then the macro block of correspondence is encoded according to intra-frame macro block coded system.
Described step S1 also comprises the step of prediction of information of taking exercises to the image of down-sampling, and according to image motion prediction information, macro block updating frequency is controlled, for the macro block comprising motion vector or motion compensation information, makes it have higher macro block updating frequency;
For region static in image, as the background of image, do not carry out macro block updating.
Summary of the invention
(1) technical problem that will solve
The technical problem to be solved in the present invention is: provide a kind of network video compression fault-tolerant transmission method and system, has an image to make mistakes to solve in a video sequence, and those data below just all can be made mistakes, and the problem that mistake can constantly expand.
(2) technical scheme
For solving the problem, on the one hand, the invention provides a kind of network video compression fault-tolerant transmission method, comprise the following steps:
S1: down-sampling is carried out to the video data gathered, and code information analysis is carried out to the data after down-sampling;
S2: the result according to described code information analysis is encoded to data;
S3: set up the data transmission interface channel between coding side and decoding end;
S4: coding side to the data after decoding end transfer encoding, decoding end to decoding data, according to decoded data to by mistake generating decoded information and feeding back to coding side by described data transmission interface channel;
S5: the mode that coding side is encoded by the described decoded information adjustment received.
Preferably, in described step S1, the concrete steps that the data after down-sampling carry out code information analysis are comprised:
S11: make stress and strain model respectively to the video image obtained after data down-sampling, obtains the subgraph after splitting;
S12: haar feature is extracted to described subgraph;
S13: if the image of process is piece image, then forward step S11 to; If the image of process is the second width and later image thereof, then forward step S14 to;
S14: the similarity calculating two continuous frames video image;
S15: differentiate two continuous frames video image, if the similarity difference of two continuous frames video image is greater than setting threshold value, then judges that the rear two field picture in described two continuous frames image there occurs scene switching.
Preferably, the similarity employing formula below of two continuous frames video image is calculated in described step S14:
Wherein, S (f
n-1, f
n) be the similarity of two continuous frames video image, f
nfor the data after down-sampling, to the data f after down-sampling
nmake k*k stress and strain model respectively,
for the subgraph after segmentation,
for the subgraph after segmentation
the characteristic vector of corresponding sub-block, wherein n represents the n-th two field picture, k representative segmentation size, and k gets 1,2,4,8, i and represents mesh row coordinate, and j represents mesh column coordinate; Every frame image features vector total length (2 × 2+4 × 4+8 × 8) × 8=672.
Preferably, described step S2 is specially:
Switch if it is determined that current frame image there occurs scene compared with previous frame image, then adopt I frame to encode to this two field picture, the bit size of every frame after recalculating;
If it is determined that current frame image does not have occurrence scene to switch compared with previous frame image, then this two field picture is encoded according to normal order.
Preferably, switch if it is determined that current frame image there occurs scene compared with previous frame image, then between current frame image and next frame image, insert the frame that motion vector and residual error are all 0, and code check is recalculated to next frame image.
Preferably, in step S4, decoding end also detects the grammer of this each data segment of code stream and the correctness of content to during decoding data in decode procedure, specifically comprises:
S41: the detection carrying out video sequence parameter collection from video sequence level to network abstract layer and video image parameter set;
S42: grammer and the content detection of carrying out video chip level, for judging key message in current video sheet and whether syntactic element meets standard and whether be the parameter that this decoder is supported;
S43: the detection carrying out macro-block level content, for detecting one by one the texture information in code stream, motion vector information and macro block type information, and according to the information errors in the code stream found and information dropout situation, generates corresponding decoded information.
Preferably, the drop-out that step S4 also comprises error of transmission causes is estimated and recovers.
Preferably, described step S5 coding side is specially by the mode of the described decoded information adjustment coding received:
When the decoded information that coding side receives represents that decoding end is correctly decoded, then encode according to normal mode;
When the decoded information that coding side receives represents decoding end decoding error, then the macro block of correspondence is encoded according to intra-frame macro block coded system.
Preferably, described step S1 also comprises the step of prediction of information of taking exercises to the image of down-sampling, and according to image motion prediction information, controls macro block updating frequency.
Preferably, described according to image motion prediction information, control is carried out to macro block updating frequency and is specially: for the macro block comprising motion vector or motion compensation information, make it have higher macro block updating frequency; For region static in image, do not carry out macro block updating.
On the other hand, present invention also offers a kind of network video compression fault tolerant transmission system, comprising:
Sampling analysis module, for carrying out down-sampling to the video data gathered, and carries out code information analysis to the data after down-sampling;
Coding module, comprises coding side, for encoding to data according to the result of described code information analysis, and is adjusted the mode of coding by the described decoded information received;
Connection establishment module, for setting up the data transmission interface channel between coding side and decoding end;
Decoding and feedback module, comprise decoding end, comprise the decoding end to the decoding data that coding side transmits, for generating decoded information to missing and feeding back to coding side by described data transmission interface channel according to decoded data.
(3) beneficial effect
The invention enables energy transmitting video data reliably under wireless channel transmission, strengthen the fault-tolerance of image, carried out again upgrading to the macro block of mistake, mistake can not be spread, and then made image still can see image clearly after there is obliterated data.
The present invention is when current frame image there occurs scene switching compared with previous frame image, and adopt I frame to encode to current frame image, it is larger that the interval due to I frame can be arranged, and therefore decreases the code stream of transmission to a certain extent, makes transmission more reliable.
Because occurrence scene switches, the correlation between continuous print two frame data is lowered, and complexity will depart from actual conditions, cause target bit allocation unreasonable, finally cause deterioration of image quality.The present invention has introduced the scene handoff algorithms more optimized, and the judgement that scene is switched is more accurate, solves because former frame mean activity accurately can not reflect present frame activity situation, makes actual effect in rate control process, cause the problem of judder.
The present invention in an encoding process, adjusts coding dynamically by feedback.Decoder easily to the position and scope etc. of encoder reporting errors macro block, can be passed to coding side decoded informations such as the location numbers of the macro block of makeing mistakes, knows which macro block has gone out mistake after coding side is obtained.The information that coding side is given for decoding end is reconstructed present frame, namely carries out intra-frame macro block (Intra) coding for decoding end error macro block, thus can stop the diffusion of error code.
Accompanying drawing explanation
Fig. 1 is the flow chart according to embodiment of the present invention video compression fault-tolerant transmission method;
Fig. 2 is the flow chart according to embodiment of the present invention video compression fault-tolerant transmission method step 1;
Fig. 3 is the flow chart according to embodiment of the present invention video compression fault-tolerant transmission method error-detecting method.
Detailed description of the invention
Below in conjunction with drawings and Examples, that the present invention is described in detail is as follows.
Embodiment one:
As shown in Figure 1, the present embodiment describes a kind of network video compression fault-tolerant transmission method, comprises the following steps:
S1: down-sampling is carried out to the video data gathered, and code information analysis is carried out to the data after down-sampling;
As shown in Figure 2, described step S1 specifically comprises the following steps:
S11: to the data f after down-sampling
nmake k*k stress and strain model respectively, definition
for the subgraph after segmentation, wherein n represents the n-th two field picture, k representative segmentation size, and k gets 1,2,4,8, i and represents mesh row coordinate, and j represents mesh column coordinate;
S12: to described subgraph
extract haar feature, definition
represent
the characteristic vector of corresponding sub-block, every frame image features vector total length (2 × 2+4 × 4+8 × 8) × 8=672;
S13: if n=1, preserves
forward step S11 to; If n > 1, then forward step S14 to;
S14: similarity S (f is calculated to two continuous frames video image
n-1, f
n):
S15: differentiate two continuous frames video image, if S is (f
n-1, f
n) > S
0, then judge that the n-th two field picture there occurs scene and switches.
S2: the result according to described code information analysis is encoded to data;
S0: be a setting value, can get between 0.5 ~ 0.8;
Be specially: switch if it is determined that current frame image there occurs scene compared with previous frame image, then adopt I frame to encode to this two field picture, the bit size of every frame after recalculating; If it is determined that current frame image does not have occurrence scene to switch compared with previous frame image, then this two field picture is encoded according to normal order.
Owing to switching in the event of scene, the occupation rate of buffering area can raise suddenly, by the calculating of buffering area, if buffering area is near 70% of upper benefit restriction, in order to reduce the occupation rate of buffering area, current frame image switches if it is determined that there occurs scene compared with previous frame image, between current frame image and next frame image, then insert the frame that motion vector and residual error are all 0, the occupation rate of buffering area will be made like this to have dropped, thus meet the needs of decoder to buffering area, and code check is recalculated to next frame image.
S3: set up a reliable data transmission interface channel between coding side and decoding end, the present embodiment adopts TCP to connect; Coding side is first transferred to decoding end the header of coding, after obtaining the confirmation feedback that decoding end receives, just starts transmitting video data stream.
Embodiment two:
For realizing a network video compression fault tolerant transmission system for method described in embodiment one, comprising:
Sampling analysis module, for carrying out down-sampling to the video data gathered, and carries out code information analysis to the data after down-sampling;
Coding module, comprises coding side, for encoding to data according to the result of described code information analysis, and is adjusted the mode of coding by the described decoded information received;
Connection establishment module, for setting up the data transmission interface channel between coding side and decoding end;
Decoding and feedback module, comprise the decoding end to the decoding data that coding side transmits, for generating decoded information to missing and feeding back to coding side by described data transmission interface channel according to decoded data.
The present invention in an encoding process, adjusts coding dynamically by feedback.Decoder easily to the position and scope etc. of encoder reporting errors macro block, can be passed to coding side decoded informations such as the location numbers of the macro block of makeing mistakes, knows which macro block has gone out mistake after coding side is obtained.The information that coding side is given for decoding end is reconstructed present frame, thus can stop the diffusion of error code.
Above embodiment is only for illustration of the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all equivalent technical schemes also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.