CN102447910A - H.264 coding video data wireless transmission method and wireless video monitoring system - Google Patents

Abstract

The invention discloses an H.264 coding video data wireless transmission method. According to the method, an error control method based on video content characteristics is utilized in a process of wirelessly transmitting data and error control is carried out on video contents belonging to different characteristics and having different visual attentions by a different-H.264 error control method according to human eye characteristics. According to the H.264 coding video data wireless transmission method, conditions of channel transmission are further considered, and the sequence of data blocks waiting to be retransmitted is decided according to pre-calculated transmission priority of each data block in a video frame so that the total frame loss rate is reduced; and channel self-adaptive mixed FEC/ARQ (Forward Error Correction/Automatic Repeat Request) error control is carried out on data packets in a data link layer. The invention further discloses a wireless video monitoring system. According to the invention, the error control is carried out on video information respectively from an application layer and the data link layer, thus the vision information requirements of a client at a receiving terminal are met, simultaneously the higher transmission efficiency can be ensured and the watching waiting time of the client is shortened.

Description

H.264 coding video frequency data radio transmitting method and wireless video monitoring system

Technical field

The present invention relates to a kind of video data radio transmitting method, relate in particular to a kind of H.264 coding video frequency data radio transmitting method, and the wireless video monitoring system that uses this method, data encoding technique field in the Network Transmission belonged to.

Background technology

Along with greatly developing of Internet of Things and 3G network, for the realization of Smart Home, " the wisdom earth " provide unlimited maybe.Video monitoring of mobile phone system under the support of existing technology of Internet of things and 3G network then is a kind of typical application, and it can be applicable to the Smart Home field and realizes functions such as household security protection, has important practical application meaning.

The transmission of the video information in the video monitoring of mobile phone system all is based on wireless channel, and is well-known, faces the challenge of three aspects in the wireless video transmission: compression efficiency, power division and error control.Wherein,, thereby cause the serious decline of reconstruction video quality, particularly essence so error control just seems because wireless channel extremely is easy to generate the error code diffusion phenomena.

The video error control technology is meant that encoder is through increasing the structure that the respective coding strategy perhaps improves video code flow; Make Video Decoder be convenient to detect mistake; And utilize space and time correlation in the image to recover the data of losing or damaging because of mistake, reduce the technology of channel error to the influence of image transmission quality.To this situation; Position according to the controlling mechanism effect is different with working method in the video monitoring of mobile phone system at present; Many error-control techniques have been proposed and have used; But many source encodings still are relatively independent of Channel Transmission, and the overall permanence that the consideration video content of not complete, system and channel error control combine also reckons without and in whole process, merges the characteristic that content-based video analysis treatment technology adapts to wireless channel.

The situation of the employed error-control technique of existing video monitoring of mobile phone system:

(1) error control of video content: existing most video monitoring of mobile phone system seldom expands to video content with the error control grain graininess; In decoding end object video is adopted error conceal method at most; And, can not adopt the error-control technique that H.264 standard provided that video content is carried out error control in encoder-side, promptly use; Also just video content is done as a wholely, used a certain error-control technique that H.264 standard provided to carry out error control.Can reach the purpose that reduces transmission delay, increases the workload of server although it is so, but lose the assurance that receiving terminal image high-quality, low distortion recover.Even adopt error concealment technology at receiving terminal this moment, also there are some unavoidable problem, for example, the spatial concealment technology is to have strong correlation in the hypothesis adjacent image piece, therefore this method is inappropriate for object edge or the violent part of moving; In addition, be the content that can obtain video image in advance based on the prerequisite of the error conceal method of object video, but this is inapplicable for many application scenarios.

(2) error control of packet: finally will being encapsulated in one by one through the frame of video that forms after H.264 encoding, packet transmits in physical layer.These packets also the problem of packet loss, error code, loss of data can occur in transmission course, thereby have influenced the recovery of frame data.So reliable video monitoring system can data link layer to this one by one packet carry out error control.

The error-control schemes that the video monitoring system of at present existing transmission based on wireless channel is used in data link layer has:

Scheme one: forward error correction technique (FEC)

Forward error correction technique (FEC) promptly adds the redundant correcting sign indicating number in the transmission code row, under certain condition, when receiving terminal found that packet is made mistakes, the redundant correcting sign indicating number in the transmission code row capable of using carried out error correction.

Advantage: processing expenditure is low, and it is little to delay time, and real-time is good;

Shortcoming: reliability is lower, coding and decoding complex equipments and costliness.

Scheme two: automatic retransmission technique (ARQ)

The message that the data message that automatic retransmission technique (ARQ) promptly asks the transmit leg re-transmission to make mistakes through the recipient recovers to make mistakes.

Advantage: simple, system reliability is high;

Shortcoming: need feedback channel, time-delay is big, and real-time is bad.

Scheme three: mix automatic request retransmission mechanism (mixing FEC/ARQ)

Mixing FEC/ARQ technology has promptly been introduced a sub-entry in ARQ, be used for correcting number of times that the error of transmission of appearance reduces re-transmission to satisfy the requirement of time delay, i.e. error correction automatically in the error correcting capability scope exceeds this scope and just retransmits.

Advantage: be a kind of compromise of FEC and ARQ technology, through limiting the number of times that retransmits to satisfy the demand of time delay;

Shortcoming: receive the restriction of FEC code efficiency, fixing automatic retransmission time can not adapt to the condition of changeable wireless channel.

(3) based on the mutual error control method of codec: the mutual error control method of coder/decoder requires to have backward channel, and only is applicable to the situation of point-to-point usually.Answer-mode has the ability of stronger inhibition mistake diffusion, but when round trip delay time was bigger, code efficiency obviously reduced.Unacknowledged mode is applicable to the situation that channel error rate is not high, but mistake can spread in round trip delay time and spreads.How the situation according to channel automaticallyes switch, and still has certain degree of difficulty at present.

In sum, the application of existing error-control technique in the video monitoring of mobile phone system will make the quality of cell-phone customer terminal image have a greatly reduced quality, and problem in this respect will inevitably become one of factor of the application that influences Smart Home.

Summary of the invention

Technical problem to be solved by this invention is to overcome the deficiency that the existing existing error control method of video Radio Transmission Technology can not satisfy actual needs; A kind of H.264 coding video frequency data radio transmitting method is provided; And adopt this method to carry out the wireless video monitoring system of video data wireless transmission; Can improve the quality that video content is rebuild, also reach the optimal compromise between controlling video transmission error ability and the efficiency of transmission simultaneously.。

The present invention is concrete to adopt following technical scheme to solve the problems of the technologies described above:

A kind of H.264 coding video frequency data radio transmitting method, H.264 transmitting terminal encodes to original video, and the data packet transmission after will encoding through wireless mode is to receiving terminal; Receiving terminal is decoded to the packet that receives, and obtains original video; Adopted error control method in the data wireless transmission course, be specially based on the video content characteristic:

Transmitting terminal carries out feature detection to original video content, and discerns corresponding video unit according to testing result, is specially: at first carry out face characteristic and detect, the video unit that will comprise face characteristic is identified as the high-level semantic feature unit; Then all the other video units are carried out motion feature and detect, the video unit that will comprise motion feature is identified as semantic feature unit, middle level; At last remaining video unit is carried out color, density or direction character and detect, and it is identified as low layer semantic feature unit;

To high level, middle level, low layer semantic feature unit, adopt corresponding error control method respectively, be specially: for the high-level semantic feature unit, the error control method that adopts inter-coded macroblocks to refresh; For semantic feature unit, middle level, adopt reprocessing error recovery approach based on time domain; For low layer semantic feature unit, adopt reprocessing error recovery approach based on the spatial domain.

Further, also adopt channel self-adapting to mix automatic request retransmission mechanism in the said data wireless transmission course and carried out error control.

Further, transmitting terminal before the data behind the coding are sent, earlier according to computes and record the priority of each data block in the Frame that will send,

${\mathrm{\η}}_i=\frac{1}{t_n-t_{i,n}},$

In the formula, η _iExpression the priority of i data block of the n frame that will send, t _nRepresent the moment that the n frame is play, the delivery time of i data block of n frame is t _{I, n}

When retransmission events takes place in the send window when, confirm the sending order of each data block according to the priority of need re-transmitting data block, the data block that priority is high is preferentially sent.

A kind of wireless video monitoring system comprises:

At least one video acquisition unit is used for the monitor video of real-time collection site;

One server, it is connected with said video acquisition unit wire signal, be used for H.264 the monitor video that video acquisition unit transmits is encoded, and the coding video frequency data after will encoding sends through wireless mode;

At least one wireless terminal, it is connected with said server wireless signal, receives the H.264 coding video frequency data that said server transmits;

Server is incited somebody to action when H.264 coding video frequency data sends to wireless terminal, the said H.264 coding video frequency data of arbitrary technical scheme radio transmitting method more than the employing.

One aspect of the present invention has adopted diversified error protection scheme in the angle of application layer from video content to video content; On the other hand; In the angle of data link layer from data block; The priority of calculating each data block needs the re-transmission order of re-transmitting data block with decision; Reach the purpose of the reorganization time delay that reduces total frame miss rate and receiving terminal frame, simultaneously the packet after the encapsulation is also carried out error control, finally reach the minimum distortion degree of video information.The present invention can thoroughly change the low situation of wireless video images reconstruction quality in the past; From application layer and data link layer video information is carried out error control respectively; Satisfy receiving terminal client's visual information demand; Simultaneously can guarantee higher efficiency of transmission again, reduce the stand-by period that the client watches.

Description of drawings

Fig. 1 is the Organization Chart of video monitoring of mobile phone system;

Fig. 2 is the model framework figure of the error protection scheme of content-based characteristic;

Fig. 3 is a video content low-level feature calculation flow chart;

Fig. 4 is the realization flow figure of channel self-adapting hybrid ARQ/FEC error-control technique.

Embodiment

Be that example is elaborated to technical scheme of the present invention with video monitoring of mobile phone system shown in Figure 1 below:

As shown in Figure 1, this video monitoring of mobile phone system comprises camera, server and mobile phone end; Wherein camera is used for the video of real-time collection site and the monitor video that collects is transferred to server through wired mode; Server is according to the requirement of mobile phone end, processing such as the resolution of converting video, code check and frame per second, and H.264 encode, the data behind the coding are transferred to mobile phone end through wireless channel; Mobile phone end received code data are also carried out decoding and displaying.Wherein, the error control method that has adopted kinds of schemes to combine in the wireless data transmission process, detailed process is following:

(1) server adopts video content feature detection method to extract different video content characteristics, on based on the basis of different content characteristic, adopts different error-control schemes H.264 to encode to guarantee video image quality substantially, to satisfy visual demand;

(2) server cuts into a plurality of data blocks in data link layer to frame data, and calculates the priority of each data block, and the data block that priority is high is retransmitted by preferential when taking place to retransmit;

(3) server encapsulates packing in data link layer to each data block, and adopts channel self-adapting to mix automatic request retransmission mechanism (FEC/ARQ) to each packet and carry out error control;

(4) cell-phone customer terminal logon server request video information is concurrent publishes picture as information such as resolution sizes, and video information is transferred to mobile phone end through wireless channel;

(5) error detection occurs of the advanced line data bag of mobile phone end decoder level is if corresponding packet level mistake Restoration Mechanism is adopted in the verification failure; If verification succeeds, based on standard H.264, extracts different video content characteristics, and to different video content characteristics have as or as the corresponding error recovery techniques of ground employing, make the video recovery quality reach optimization.

Can see from above wireless data transmission process; The present invention has adopted the error control method of content-based characteristic; And combining adaptive mixes automatic request retransmission mechanism, and the different pieces of information piece is calculated the reorganization time delay that respective transmissions priority reduces the miss rate and the receiving terminal frame of frame.Further describe in detail in the face of technical scheme of the present invention down.

(1) based on the error control of video content characteristic

Up to the present, error control algorithm great majority in wireless mobile multimedia service, the especially video transmission also rest on " bit " level, promptly regard video as binary bit stream, and have ignored information that video content itself the comprised perceptibility to the people.In fact, video communication is different from common digital communication, because the last stay of two nights of video is people's perception, people's vision often can be tolerated some distortions under certain conditions, and therefore, the fidelity that surpasses visual resolving power requires there is no need.The human visual system has a lot of apperceive characteristics, such as the brightness sheltering, and spatial concealment property and temporal masking property or the like.If can make full use of human visual system's physiological property, suitably adjust some parameter, just can obtain considerable visual effect with less cost.Utilize vision physiological---psychological characteristic is carried out error control; Just require computer can " understand " content of video; Know that which type of error control method which type of video content needs protect; Therefore, the present invention combines the theory and the method for numerous areas such as computer vision, compression of digital video, video content analysis and wireless video communication, is applied to the video error control technology; Vision perception characteristic according to content adopts corresponding error-control technique, makes the reconstruction of image satisfy people's visual demand.Fig. 2 has shown the error control model that the present invention is based on content characteristic, and its particular content is following:

I) set up the Three-tier Architecture Model of video content characteristic

Video content comprises abundant semantic information, and simultaneously, the research of psychology of vision finds that also for video information, people's face that human eye can be paid special attention to occur in the video perhaps is included in the fritter marking area in large stretch of homogeneous region.According to these characteristics, native system at first carries out structuring with non-structured video information, and video data is set up model.

The content of video comprises aspects such as color, density, direction, motion and people's face.Color characteristic is mainly considered color space, dominant hue and color layout etc.; Density is meant catoptrical amount under the specified conditions, has reflected the density degree of homogeneous region to a certain extent; Direction character is mainly described the direction of the local crestal line of salient region; Main feeling the pulse with the finger-tip mark movement locus moves; Face characteristic is described the human face region size that occurs in the video unit and is distributed.Different video content characteristics is different for the percentage contribution of human eye perceived quality, and what cause really that the human visual system notes is not the intensity of these characteristics itself, but the contrast intensity between these characteristics.

Native system is the video monitoring of mobile phone system under the wireless channel; Finally will be applied to the household security protection in Smart Home field or the like; So, no matter be that the video information that server or cell-phone customer terminal obtain comprises human face region mostly, from the psychology of vision angle; The size of human face region is not only the high-level semantic characteristic, and can arouse attention especially.At first can be divided into several independently scene unit to video sequence, in each scene motion of objects arranged, be divided into several camera lenses to scene according to these motion features according to whether people's face occurring.Each camera lens further is divided into several frames according to vision significance characteristics such as color, density and directions.Therefore, this three-layer weave structure of---camera lens---unit according to scene can be divided into the high-level semantic characteristic to content characteristic, for example people's face accordingly---middle level semantic feature, for example motion---low layer visual signatures such as color, density.

II) video content feature extraction

No matter be after server or cell-phone customer terminal obtain video information in the native system; All to carry out feature extraction to video; Form people's face---motion---color, three level characteristics of density; Thereby make server can adopt different error control and error recovery approach according to these three levels, arrive the effect that increases the protection of effective information with cell-phone customer terminal.So system at first will consider how to realize feature extraction the server and client side.

● face characteristic extracts

For under the situation that does not increase resource, the quality of face image is greatly improved, just need detect people's face target, want effective track human faces simultaneously.Detection method to people's face all is the basis with the vision, and existing at present edge detection method, colour of skin method, shape method, texturing method and modelling or the like can supply to adopt.The Face Detection method is adopted in detection to people's face in the native system.Step is following:

The first step: use the human face region that possibly exist in the colour of skin chrominance information positioning video frame.Discover that people's the skin tone and the tone of other natural forms have evident difference.Although the color of skin varies with each individual, under different light, also change to some extent, it always occupies a specific zone in color spectrum.Utilize this characteristic, can split area of skin color.

Second step: cut apart area of skin color, form the candidate face zone.Merge and get rid of some candidate face zones, finally confirm the concrete scope of human face region.If C ₀Be colour of skin homogeneous region R _i, i ∈ [1 ... N] set, at first confirm a regional connectivity figure, then according to maximum allowable range D with connective decision rule _r(R _j, R _j) confirm two adjacent region R _iAnd R _jWhether merge.In the HSV space, people's face is determined by colourity (H component) that mainly the variation of saturation and brightness (S component and V component) is mainly caused by light, so, select color distortion tolerance to be:

D _r(R _i，R _j)＝α|H _i-H _j|+β|S _i-S _j|+γ|V _i-V _j| (1-1)

Wherein, (H _i, S _i, V _i) be region R _iIn average colour of skin vector.α is much larger than β and γ.

Iterate repeatedly, become a human face region up to merging all less than the area of skin color that allows distance.

● motion feature extracts

Video is not only tieed up in the space, and on time dimension, also has logical construction.Therefore motion feature becomes a very important aspect of video content.The peak value of the pattern of the motion object in the video is less; And the direction transformation frequency is higher; And this motion motion of objects field is temporal and spatial correlations, and instantaneous motion of objects can be regarded linear movement as, the occupied macro block of each object in present image; The size and Orientation of its motion vector is consistent, so can detect the motion object through the similarity of analyzing motion vector.

Owing in the video bit stream motion vector has been arranged,, has utilized motion vector to describe motion feature in order to reduce amount of calculation.Native system has adopted a kind of motion attention model, supposes that motion field has three transducers: density sensor, space correlation transducer and association in time transducer.To the frame of each input, at first based on the piece estimation of taking exercises, the two-dimensional histogram of calculating kinematical vector finds the maximum motion vector of occurrence number, is made as V _g, its occurrence number is C _gIf, C _gSurpass the half the of macroblock number, just global motion vector is made as V _g, otherwise establishing global motion vector is zero motion.In having removed the later motion field of global motion, native system is with two characteristics: motion density and motion consistency come modelling motion conspicuousness.

For the motion vector among the macro block i, the density of definition vector is:

$I\left(i\right)=\sqrt{d_x^2+d_y^2}---(1-2)$

Calculate the motion vector density of all macro blocks in the frame, just can obtain density profile.

Consider time, spatial window then around macro block i.If W _x(i) be the motion vector collection of these all macro blocks of window, the phase place of definition motion vector is:

$\mathrm{Phase}=\arctan \left(\frac{d_y}{d_x}\right)---(1-3)$

The scope of phase is [0,2 π], is calculating W _x(i) after the phase place histogram, through the space-time consistency of computing macro block, can obtain motion field consistency distribution map, last integral density distribution map and consistency distribution map obtain motion attentiveness distribution map.Thereby extract the motion feature of video content.

● low-level feature (color, density and direction) extracts

The calculation process of low-level features is as shown in Figure 3.

For each color component R, G and B in the video image, according to 1: 1 to 1: 2 of original image ⁿRatio generate gaussian pyramid.

For example, to the input picture of 640*480 resolution, be divided into nine spatial level in the native system according to two fens gaussian pyramids, generated 1: 1 (zoom factor is 0) to 1: 256 a plurality of subgraphs such as (zoom factor are 8).Center pixel be exactly progression c ∈ 2,3, the pixel of 4}, surrounding pixel be exactly progression at s=c+ δ, δ ∈ 3, the pixel of 4}.The method that grade difference is striden in calculating is the meticulous level of first interpolation, subtracts each other by pixel then, representes with Θ.Between center and peripheral region, on the many levels of δ=s-c, generated the multilayer characteristic like this at c.

is to the calculating of density feature

Use r, g, b represent the red, green, blue passage of frame of video, and frame density is defined as: I=(r+g+b)/3, I are used for producing gaussian pyramid I (δ), and wherein δ ∈ [0,8] is a convergent-divergent progression.Because tone variations feels not obvious in the zone of low-light level, so normalizing operation only is applied to The zone, I _MaxThe maximum of I in the expression entire image.Frame density with described neuron for the variation in brightness of center and peripheral region relatively the sensitivity.Center and density feature difference are on every side calculated with following formula:

I(c，s)＝|I(c)ΘI(s)| (1-4)

Wherein, c ∈ 2,3, and 4}, s=c+ δ, δ ∈ 3,4}

is to the calculating of color characteristic

Specific reaction can take place to the visual stimulus of two kinds of primary colors in gangliocyte: red and green; Blue and yellow.Therefore define the Color Channel of broad sense: R=r-(g+b)/2 representes red; G=g-(r+b)/2 representes green, and B=b-(r+g)/2 representes blue, and Y=(r+g)/2-|r-g|/2-b representes yellow (negative value is made as zero); Can produce four gaussian pyramid R (δ) from these several Color Channels; G (δ), B (δ) and Y (δ), center and color characteristic figure on every side calculate by following formula:

RG(c，s)＝|(R(c)-G(c))Θ(G(s)-R(s))| (1-5)

BY(c，s)＝|(B(c)-Y(c))Θ(Y(s)-B(s))| (1-6)

(c s) explains red-green pairing effect to RG, and (c s) explains the blue-yellow pairing effect to BY.

is to directional characteristic calculating

(δ θ) calculates directional information, δ ∈ [0,8] expression convergent-divergent progression wherein, θ ∈ { 0 °, 45 °, 90 °, 135 ° } expression direction through density I being used oriented Gabor pyramid O.The Gabor filter is the product of cosine and 2 dimension gaussian envelope, corresponding to the pulse in visually-perceptible territory---and response sensitiveness.Direction character distribution map O (c, s θ) calculate with following formula:

O(c，s，θ)＝|O(c，θ)ΘO(s，θ)| (1-7)

Wherein, θ ∈ 0 °, and 45 °, 90 °, 135 ° } the expression direction.

III) content-based characteristic adopts the concrete realization of different error-control schemes

Video is different from common binary bit stream, and it contains abundant semantic information.The different piece of code stream is different for the contribution of video-aware; But because the information of the various piece representative of video code flow has strong relevance in terms of content; So can not cut apart simply; Must instruct with content characteristic and select suitable error control method, just can reach best visual effect.

The error control of high-level semantic characteristic---human face region

It is to stop error propagation and recover one of effective method of image that inter-coded macroblocks refreshes.Though adopting the method for frame refreshing can propagate by control error; But the method for intraframe coding has increased data volume, therefore, and under the limited condition of wireless signal-path band width; This method only is fit to the video content of protection particular importance, and the redundancy that produces can not be too many.Have the zone of high-level semantic characteristic in the video, for example human face region has continuity, and the height consistency is arranged between the adjacent macroblocks, and the method data volume that therefore adopts intraframe coding to refresh is little, can reach an equilibrium on error control and the code efficiency.

Server end detects after the video human face zone at pretreatment stage, confirms to comprise the number and the position of the macro block of human face region, and defining these macro blocks is the frame refreshing macro block.Like this, in transmission course, when error code appearred in channel, the mobile phone decoding end was only predicted from the macro block of correctly receiving the data of the human face region in the video and is obtained, and can not receive the influence of error propagation, has guaranteed the basic vision effect.

The video area of native system to causing especially that human eye is paid close attention to, human face region for example, the mode that adopts inter-coded macroblocks to refresh specifically is divided into three layers of processing.Ground floor: the macroblock number of confirming the needs intraframe coding according to the packet loss of channel.The error propagation effect degree of frame one-level is EP in the setting video _n(n is a time index), the macroblock number that needs intraframe coding to refresh is so calculated as follows:

$N_{\mathrm{intra}}^{\mathrm{GOP}}=\frac{\frac{1}{N_{\mathrm{GOP}}}\underset{n=1}{\overset{N_{\mathrm{GOP}}}{\mathrm{\Σ}}}{\mathrm{EP}}_n\×\mathrm{PLR}}{{\mathrm{TH}}_{\mathrm{intra}}}---(1-13)$

Wherein, is all number of macroblocks that need intraframe coding to refresh in the video image.N _GOPIt is the frame number of video image.PLR is the channel packet loss.TH _IntraIt is flexible parameter.

The second layer: give macro blocks all in the camera lens according to macro-block level error propagation effect degree EP _MBOrdering is selected top

Individual macro block carries out frame refreshing.Can protect so most possibly by the macro block of error propagation effect.

The 3rd layer: the processing of the second layer has brought two problems, and the one, the frame refreshing macro block possibly too concentrate on the first half of camera lens; The 2nd, a plurality of macro blocks possibly concentrate on same frame.In order to overcome this two problems, carried out the 3rd layer of processing, promptly be evenly distributed the macro block that needs intraframe coding to refresh of frame one-level, concrete handling process is following:

If n=2 (such as first P frame in the camera lens)

$N_{\mathrm{intra}}^n=\frac{{\mathrm{EP}}_n}{\underset{i=n}{\overset{N_{\mathrm{GOP}}}{\mathrm{\Σ}}}{\mathrm{EP}}_i}\×N_{\mathrm{intra}}^{\mathrm{GOP}}$

else?if?3≤n≤N _GOP (1-14)

$N_{\mathrm{intra}}^n=\frac{{\mathrm{EP}}_n}{\underset{i=n}{\overset{N_{\mathrm{GOP}}}{\mathrm{\Σ}}}{\mathrm{EP}}_i}\×(N_{\mathrm{intra}}^{\mathrm{GOP}}-\underset{i=2}{\overset{n-1}{\mathrm{\Σ}}}{N}_{\mathrm{intra}}^i)$

endif

if $N_{\mathrm{intra}}^n>k_{\mathrm{MB}}{N}_{\mathrm{MB}}^F$ then $N_{\mathrm{intra}}^n=k_{\mathrm{MB}}{N}_{\mathrm{MB}}^F$

Wherein, n is the frame index in the camera lens; It is the macroblock number of intraframe coding among the frame n;

It is macroblock number all in the frame; K _MB(0≤K _MB≤1) is the Control Parameter of constraint inter-coded macroblocks.Through the 3rd layer processing, the macro block that intraframe coding is refreshed is evenly distributed in camera lens, avoids too concentrated.

Intraframe coding refreshes that to be operated in coding side be that server end in the native system is accomplished, and increases any load can not for the decoder of cell-phone customer terminal, and is easy to and other error-control schemes combine.

Middle level, low layer semantic feature---the error control of motion feature, density, color, direction

Video content comprise movable information the zone, have density, color and directional characteristic salient region or the like; In having for these, the error control of the video area of low layer semantic feature, native system mainly is in the decoder of cell-phone customer terminal, to adopt the mode of the reprocessing error concealment in time domain and spatial domain to realize.

The scope that error recovery techniques can limit and specified data is made mistakes, but also must estimate the information of losing and recover by the error concealment technology.Because human eye has fault-tolerance; Not that fairly obvious wrong people does not generally have obvious sensation; And since image sequence itself just by on the very strong space with temporal correlation, this goes to predict that for using the video data correctly receive the data of losing provide possibility.Through obliterated data is filled up, make video quality return to human eye can accept stage.

Mistake to the data of the moving region in the video is recovered; Native system adopts the motion vector according to the time domain adjacent macroblocks that H.264 standard provided to use prediction algorithm " conjecture " to abandon the method for the motion vector of macro block after cell-phone customer terminal utilizes above-mentioned motion feature detection method to extract motion feature; The motion vector that utilizes " conjecture " to go out carries out the motion compensation of reference frame, recovers the macro block that abandons.If motion vector mean value thinks then that less than a threshold values (such as 1/4 pixel) the image motion amplitude is very little, the macro block that abandons is realized error concealment through the macro block data that directly duplicates relevant position in the reference frame; If motion vector mean value greater than threshold values, is just predicted the motion vector that abandons macro block, from adjacent candidate motion vector, choose an only motion vector as predicting the outcome.

Often spatial position change is bigger for low-level feature salient region in the video lens, unlike human face region continuity is arranged equally.If the method that therefore adopts inter-coded macroblocks to refresh can increase data volume, add the burden of macroreticular.Native system is based on the computational methods of above-mentioned low-level feature salient region; To give mobile phone end to the Task Distribution that the mistake of the data in density, color and direction zone in the video is recovered; And adopt spatial domain error concealment technology that the salient region with low-level feature is carried out mistake and recover, reach minimum thereby the salient region of the low characteristic of assurance video receives the influence of mistake.

Specifically, system adopts the level and smooth restoration methods in the Error concealment techniques of spatial domain to carry out error protection.This method is based on and has possible strong correlation in the adjacent image piece, if therefore damaged block is in identical zones such as color, density or direction with adjacent block, then can produces and cover effect preferably.

2) each data block is transmitted according to different transmission priorities

In native system; H.264 coded format is adopted in the transmission of wireless video, and code stream is made up of the frame of I, P and category-B type respectively, and the I frame is the frame of independent decoding; And P and B frame need could be decoded with reference to the I frame, so the different time dependences that just are it with general data of video stream data.

Each frame of video must be carried out decoding processing at decoder and arrived for the previous period, and video could be play continuously and do not produce disappearance and cause mistake and influence play quality like this.If use t _nRepresent the moment that the n frame is play, the n frame must be at t so _n-T _pArrive constantly, here T _pBe the time of decoding cost, also need consider property association in time of different frame certainly.Suppose that each frame is divided into m data block, each packet i decoding that belongs to the n frame equally all is t constantly _n-T _pIf, lack wherein any data block, will produce the disappearance mistake so.To producing the wrong data block of disappearance, system must be able to retransmit it.Which does that at first transmit actually in all data blocks that need retransmit, could reduce total frame miss rate?

From physical layer, each data block does not have the difference on the time delay, therefore, just asks in sequence according to receiving the NACK that comes during re-transmission.But, cause with respect to the upper strata very big delay variation being arranged in the adjacent data block of physical layer because the various network differences in channel conditions is very big.This just a kind of situation might occur; Time in the time limit weak point that the re-transmission stand-by period looks and reaches decoding; If we still retransmit according to receiving the NACK request that comes in proper order, may increase the time delay of receiving terminal to this video requency frame data piece reorganization, even be not beneficial to the recovery of frame of video.

In order to address this problem; Native system at first writes down the transmission priority of each data block in each frame of video, so that when repeat requests takes place, can be according to this priority; Select the high data block of priority to retransmit; Rather than retransmit in proper order according to the NACK request that receives, thereby reduce the time delay of receiving terminal as far as possible to this video requency frame data piece reorganization, reduce total frame miss rate.

Introduce the computational methods of the priority that each data block transmits in the frame of video below:

The data block delivery time of supposing each frame is t _{I, n}, be Δ t then to the time difference of playing _{I, n}=t _n-t _{I, n}, the priority that defines each data block is η _i=Δ t _{I, n} ^-1

Concrete execution mode is that transmitting terminal is provided with a priority query, when data block of every transmission, just in priority query, writes down this data block sequence number and the pairing priority value of this data block.In when, in send window retransmission events taking place when; According to the record in this priority query; In the data block that needs retransmit, select the high data block of priority to carry out prioritised transmission; Avoid occurring retransmitting long Frame of the stand-by period short situation of time limit that requires to decode, reduce as far as possible, obtain the peak efficiency of frame data piece reorganization thereby make at the total frame miss rate of receiving terminal.

3) based on the error-control technique of packet

Server after data block packed, adopts the mixing FEC/ARQ error-control technique based on channel self-adapting to carry out error control to packet after data link layer has been calculated priority to frame data divided data piece.

Fig. 4 has showed the flow chart of the error-control technique of channel self-adapting mixing FEC/ARQ that system adopts.

In mixing FEC/ARQ technology, key is to confirm the length of the redundant information of each packet.Make N represent the total length of packet, r representes the length of the redundant information of a packet.When the mobile phone end decoder carries out error checking and correction to a packet that receives; When surpassing r/2 as if the quantity that error bit occurs; Think that then mistake has appearred in this packet in transmission, when packet makes a mistake, if number of retransmissions does not surpass the restriction of maximum retransmission; Wait for that then transmitting terminal retransmits this packet, otherwise decoder utilizes the redundant information in the packet to carry out the mistake recovery.

For the optimal value of redundant information length in the packet, system adopts following computational methods:

Make ber and per represent the error rate and the packet loss of wireless channel respectively, when ber gives regularly, per can use following relational expression to obtain:

$\mathrm{per}=1-\underset{i=0}{\overset{[r/2]}{\mathrm{\Σ}}}{C}_N^i{\left(\mathrm{ber}\right)}^i\·{(1-\mathrm{ber})}^{N-1}\·{(1-\mathrm{ber})}^{N-i}---(3-1)$

The optimal value of redundant information can obtain through following equation like this:

r _opt＝min{r|per≤perobj} (3-2)

Wherein, perobj representes the desired value of wireless channel packet loss.

Encoder is according to the assembly average of the feedback information calculating channel packet loss of receiving terminal, and we represent with perobs.In order to obtain packet loss more accurately, need wireless channel be divided into two different state, i.e. stable state and transient state.So-called stable state is meant that wireless channel, portable terminal and external environment do not have violent variation, and the error rate of wireless channel does not have violent fluctuation.So-called transient state be meant stage of wireless channel state transition between two kinds of stable states, or motion of mobile terminals changes violent; Or external environment significantly changes; The error rate time variation of wireless channel is very strong, and usually, this state can last very long.Obviously under the two states, the packet loss of wireless channel also is not quite similar.

How that judges the residing state of current wireless channel, and how between this two states, switches freely and do not cause the vibration of system, then is again an important problem.Native system adopts a kind of early warning mechanism, uses the early warning value of a local value of packet loss as state variation.Add up the feedback information of several packets of nearest very small amount; Whether detect it has gone out significantly unusual; When the Where topical statistical value is obviously unusual, explain that significant variation has taken place the external environment of wireless video communication system, system is in state variation period; Then change current channel status into transient state, to change the method for current error control mechanism and statistics packet loss.

For stable situation, the computing formula of perobs is following:

$\mathrm{perobs}(n-\mathrm{rttn})=1/S\underset{i=0}{\overset{S-1}{\mathrm{\Σ}}}m(n-\mathrm{rttn}-i)---(3-3)$

Wherein, n representes current time; Rttn representes data packets for transmission number in the loop time; S representes to be used for the quantity of the packet sample of statistical computation; I the moment of m (i) expression data packets for transmission feedback information, value are 1 to represent NACK, show that mistake has appearred in packet in transmission course; Value is 0 to represent ACK, shows that packet is received by correct.

For the transient state situation, adopt the iteration forgetting factor method to calculate packet loss, that is:

$\mathrm{perobs}(n-\mathrm{rttn})=(\mathrm{perobs}(n-\mathrm{rttn}-1)\×\mathrm{\β}+\frac{1}{s}m(n-\mathrm{rttn}))\×\frac{1}{1+\mathrm{\β}}---(3-4)$

Under initial situation, be as the computing formula of the packet loss of the iterative initial value of state variation early warning value:

$\mathrm{perbos}(n-\mathrm{rttn})=(\mathrm{perbos}(n-\mathrm{rttn}-s)\×{\mathrm{\β}}^s+\frac{1}{s}\underset{i-0}{\overset{s-1}{\mathrm{\Σ}}}m(n-\mathrm{rttn}-i)\·{\mathrm{\β}}^i)\×\frac{1-\mathrm{\β}}{1-{\mathrm{\β}}^{s+1}}---(3-5)$

Wherein, β representes forgetting factor, the tracing step that the value decision systems of value between 0.95～1, and β usually changes channel bit error rate, and β is big more, and the step-length of tracking is also big more.And s replaces S with less relatively packet sample value s.

Optimal redundancy value of information r according to relational expression (3-2) etching system when obtaining n-rttn _Opt(n-rttn).

With r (n) expression n redundant information value constantly, then:

Therefore under the stable situation, the variation of the error rate is less, is stabilized near the numerical value, can think r _Opt(n-rttn) be still the current optimal value of wanting the data packets for transmission redundant information, that is:

r(n)＝r _opt(n-rttn) (3-6)

Under the transient state situation, the error rate of wireless channel is acute variation in time, and the factor of this delay must be taken into account, need be to r _Opt(n-rttn) carry out certain adjustment to obtain r (n), that is:

r(n)＝r _opt(n-rttn)+(r _opt(n-rttn)-r(n-rttn))/2 (3-7)

So, whether the value of the maximum retransmission that packet allowed and the length of redundant information is reasonable, and we can weigh through efficiency of transmission γ:

$\mathrm{\γ}=\frac{\underset{i=1,t_i\≤t_{\max }}{\overset{n_p}{\mathrm{\Σ}}}(N-R_i)}{{\mathrm{Nn}}_p+N\underset{i=1}{\overset{n_p}{\mathrm{\Σ}}}{t}_i}---\left(\text{3-4}\right)$

Wherein, n _pExpression needs the sum of data packets for transmission, t _iThe number of retransmissions of representing i packet, t _MaxThe maximum retransmission that the expression wireless video communication system allows, N representes the length of a packet, R _iThe length of representing the redundant information of i packet.

Packet is adopted the error-control technique of the mixing FEC/ARQ of channel self-adapting, and its advantage is:

(1) can guarantee the coherent and smooth and easy of wireless video service, can not interrupt owing to emergent environment change causes connecting;

(2) under the situation that network condition is known nothing, through the optimization of study realization efficiency of transmission, conserve network resources.

Claims (6)

1. coding video frequency data radio transmitting method H.264, H.264 transmitting terminal encodes to original video, and the data packet transmission after will encoding through wireless mode is to receiving terminal; Receiving terminal is decoded to the packet that receives, and obtains original video; It is characterized in that, adopted error control method in the data wireless transmission course, be specially based on the video content characteristic:

Transmitting terminal carries out feature detection to original video content, and discerns corresponding video unit according to testing result, is specially: at first carry out face characteristic and detect, the video unit that will comprise face characteristic is identified as the high-level semantic feature unit; Then all the other video units are carried out motion feature and detect, the video unit that will comprise motion feature is identified as semantic feature unit, middle level; At last remaining video unit is carried out color, density or direction character and detect, and it is identified as low layer semantic feature unit;

To high level, middle level, low layer semantic feature unit, adopt corresponding error control method respectively, be specially: for the high-level semantic feature unit, the error control method that adopts inter-coded macroblocks to refresh; For semantic feature unit, middle level, adopt reprocessing error recovery approach based on time domain; For low layer semantic feature unit, adopt reprocessing error recovery approach based on the spatial domain.

2. coding video frequency data radio transmitting method H.264 according to claim 1 is characterized in that, has also adopted channel self-adapting to mix automatic request retransmission mechanism in the said data wireless transmission course and has carried out error control.

3. coding video frequency data radio transmitting method H.264 according to claim 1 is characterized in that, transmitting terminal before the data behind the coding are sent, earlier according to computes and record the priority of each data block in the Frame that will send,

?，

In the formula;

expression the priority of

individual data block of

frame that will send; In the moment that expression

frame is play, the delivery time of

individual data block of

frame is

;

When retransmission events takes place in the send window when, confirm the sending order of each data block according to the priority of need re-transmitting data block, the data block that priority is high is preferentially sent.

4. coding video frequency data radio transmitting method H.264 according to claim 1 is characterized in that said face characteristic detects the Face Detection method that adopts.

5. state H.264 coding video frequency data radio transmitting method according to claim 1, it is characterized in that, when motion feature detects, adopt the motion attention model.

6. wireless video monitoring system comprises:

At least one video acquisition unit is used for the monitor video of real-time collection site;

One server, it is connected with said video acquisition unit wire signal, be used for H.264 the monitor video that video acquisition unit transmits is encoded, and the coding video frequency data after will encoding sends through wireless mode;

At least one wireless terminal, it is connected with said server wireless signal, receives the H.264 coding video frequency data that said server transmits;

It is characterized in that server is incited somebody to action when H.264 coding video frequency data sends to wireless terminal, adopt each said H.264 coding video frequency data radio transmitting method of claim 1-5.

Images