CN102905200A - Video interesting region double-stream encoding and transmitting method and system - Google Patents
Video interesting region double-stream encoding and transmitting method and system Download PDFInfo
- Publication number
- CN102905200A CN102905200A CN2012102790053A CN201210279005A CN102905200A CN 102905200 A CN102905200 A CN 102905200A CN 2012102790053 A CN2012102790053 A CN 2012102790053A CN 201210279005 A CN201210279005 A CN 201210279005A CN 102905200 A CN102905200 A CN 102905200A
- Authority
- CN
- China
- Prior art keywords
- roi
- area
- video
- real
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a video interesting region double-stream encoding and transmitting method and a video interesting region double-stream encoding and transmitting system. By the method, an interesting region is subjected to enhanced encoding by adopting a relatively high code rate, and a non-interesting region is encoded by adopting a common code rate; a generated basic stream and a generated enhanced stream can be transmitted under a condition of finite bandwidth according to distribution in a time domain. The information determination degree is determined according to the probability of a specific semantic event, and user-oriented interesting encoding is executed, so that information redundancy the user is not interested in can be effectively removed from a monitored video, and the overall encoding efficiency of the monitored video can be greatly improved. By the method, an extension-type encoding model with adaptive determination degree and an optimal channel space domain-time domain resource distribution double-stream packaging method are constructed by researching the information determination degree of the monitored video, so that adaptive encoding of interesting information of the monitored video is realized; and the monitored video encoding efficiency is improved.
Description
Technical field
The present invention relates to a kind of method and system of Video coding transmission technique field, specifically a kind of method of again code stream being carried out stream transmission for area-of-interest reinforcement coding.
Background technology
Society politics, rapid economic development, video monitoring is as a kind of effective precautionary measures and post-survey means, and it is concerned degree and also improves constantly.The user is different from traditional entertainment video to the video monitoring system demand, mainly contains the demand of two aspects: real-time demand and high-quality require (also claim have access to demand) afterwards.
H.264 be MPEG-4 the tenth part, it is the high compression digital video coding-coding device standard that is proposed by the joint video team (JVT, Joint Video Team) that ITU-T Video coding expert group (VCEG) and ISO/IEC dynamic image expert group (MPEG) constitute jointly.H264 has many good characteristics: higher code efficiency, high-quality video draw, higher network-adaptive ability, adopt the hybrid coding structure, H.24 can be applied in different occasions, mistake restore funcitons, higher complexity.In addition, the H264 standard is also supported flexible macro-block order (FMO) and scalable coding (SVC) function.
At present, based on telescopic nature in H.264, can be applicable to the different demands of different terminals, but less for will H.264 being applied to field of video monitoring and carrying out the research of adaptive coding.At paper " the multithread coding of monitor video and transmission research " (Shanghai Communications University's Master's thesis, 2012) the middle scheme that proposes the reasonable distribution bandwidth on a kind of time-domain, Effective Raise code efficiency, reject the information redundancy on the time-domain, but do not strengthen coding according to the interested spatial domain of user in the video in the paper, thereby it is redundant to exist certain spatial information.Paper " Scalable ROI Algorithm for is Video Streaming H.264/SVC-Based " (IEEE International Conference on Consumer Electronics (ICCE), 2011) in ROI is carried out in the central area of video and strengthen coding, but its area-of-interest is divided and is based on human-eye model, be not to propose for the monitor video sequence, the time-domain redundant information do not rejected simultaneously yet.
The problem of the mass data transfers storage that runs into for developing rapidly of current monitor video industry, the characteristics that the monitor video demand are different from traditional entertainment video according to the user, this paper has proposed to be applied to the double-deck ROI of the video zone space-time unite coding and transmission method of video monitoring system, to anomalous event in the time period area-of-interest carry out high-quality coding and the time domain bandwidth of reasonable distribution bit stream.Considering simultaneously in the video sequence comes evaluates video quality can not reflect the difference in ROI zone and non-ROI zone with the PSNR value of single view picture figure, thereby a utility function is proposed as objective indicator, video in this specific area of supervisory control system is carried out objective evaluation, in conjunction with subjective index video is carried out comprehensive consideration simultaneously.
Summary of the invention
The object of the invention is to overcome the weak point that technology is arranged, a kind of method and system of again code stream being carried out stream transmission for area-of-interest reinforcement coding have been proposed, can be according to user's interested area information in the video, user's area-of-interest is strengthened coding, reject time redundancy and spatial redundancy in the video, realize the efficient utilization of bandwidth.
The present invention is achieved by the following technical solutions:
The invention provides a kind of area-of-interest double-stream coding transmission method, described area-of-interest double-stream coding transmission, refer to: area-of-interest adopts higher code check to strengthen coding, but not area-of-interest then adopts general code check to encode, the Basic Flow that produces and enhanced flow are by the distribution on the time-domain, so that it can finish transmission in the finite bandwidth situation; Described method comprises the steps:
Described utility function, refer to: the difference that fully takes into account in the video sequence the contained information significance level of the information of different period zoness of different, wherein the abnormal period area-of-interest has the highest significance level, in utility function, also has the highest weight, but not the information significance level of abnormal period is lower, and the weight of corresponding distribution is also less.
This method mainly is for field of video monitoring to be encoded in the ROI zone, and more attention is zebra stripes zone etc. in car plate, the traffic in the field of video monitoring, and to pay close attention to model different with human eye.This paper does not lie in the extraction in ROI zone because focusing on ROI regional code algorithm realizes, thus the ROI zone be retrieved as known conditions.
Adopt the ROI extraction algorithm to obtain the area information of ROI, adopt the second pattern among the FMO, namely prospect and background mode according to FMO characteristic and ROI area information, are divided into ROI sheet group and background plate group to image.Because the ROI zone of different sheet groups is absolute codings, can realize that by forbidding the prediction between the sheet group ROI zone code stream and the non-ROI zone code stream that generate are separate.The ROI area information that basis obtains in advance when encoding like this is as the foreground information of the 2nd kind of mode among the FMO.Basic layer is arranged the real-time streams code check and enhancement layer arranges the QP value, just can obtain two parts code stream at enhancement layer like this, i.e. the enhancing code stream in the enhancing code stream in ROI zone and non-ROI zone.By deleting the enhancing code stream in non-ROI zone, just remaining basic layer bit stream and ROI zone strengthen code stream, and two-part code stream decoding namely can obtain the ROI zone the more video sequence of high image quality.
Again because the ROI area information is time dependent, be that the ROI zone is dynamic, so when the ROI area information changes, need to remodify the parameter that MBA is set, video in the new time is encoded, and the code stream of removing the non-ROI of enhancement layer zone behind the coding gets final product.The video sequence that is stitched together behind a plurality of code stream decodings that obtain has at last namely been realized the effect that dynamic ROI zone strengthens.
Step 3, the band-width tactics according to step 1 is distributed carries out stream transmission;
Described stream transmission, concrete grammar is: when anomalous event occurs, only transmit basic layer bit stream; After anomalous event finishes, reduce the code check of basic layer bit stream, make unnecessary bandwidth be used for the enhancement layer bitstream of transmission abnormality event, its basic enhancement layer bitstream only comprises the enhancing information of area-of-interest.When event checking module is judged the generation of anomalous event, and obtain the abnormal area information in the abnormal time section.Event checking module is issued the enhanced flow encoder with anomalous event and area information.Simultaneously, have corresponding algorithm and obtain adaptively best Data Rate Distribution scheme according to the ROI area size, and real-time streams code check and the QP value of encoding in the scheme passed to real-time encoder and enhanced encoder.Enhanced encoder can be strengthened coding to the anomalous event abnormal area, and code stream is stored.When the anomalous event end, system enters without the anomalous event process.Encode according to resulting real-time coding code check in the scheme just this moment, and the encoder bit rate of real-time streams video is reduced, and reserves the transmission that bandwidth is used for enhanced flow.After high-quality stream transmission of video was finished, the encoder bit rate with the real-time streams encoder raised again, and all available network transmission bandwidths are used for the transmission of real-time streams video.
The described Data Rate Distribution scheme that obtains adaptively the best according to the ROI area size, refer to: front-end detection be whether a zone is the probability of user's area-of-interest, probability determines whether it is area-of-interest by double-current ROI utility function thus, and the basic layer of transmission bandwidth with coding parameter, Basic Flow and the enhanced flow of enhancement layer distributes after adjusting adaptively in this utility function.
The present invention also provides a kind of system that realizes above-mentioned area-of-interest double-stream coding transmission method, and this system comprises: monitor video transmitting terminal, network channel and monitor video receiving terminal.Wherein:
Transmitting terminal comprises: event and regional detection module, real-time streams encoder, enhanced flow encoder, real-time Transmission module and non real-time transport module; Real-time streams encoder, enhanced flow encoder produce respectively above-mentioned real-time streams and enhanced flow, the real-time Transmission module, and non real-time transport module carries out encapsulated delivery to real-time streams and enhanced flow respectively.Event and regional detection module be for detection of time of anomalous event and zone, and encode with real-time streams with gained information control enhanced flow coding.
Receiving terminal comprises: real-time streams decoder, memory module and display module; Real-time streams decoder and display module provide the service of real-time demonstration for the user; Memory module can be deposited real-time streams and thereby enhanced flow forms high-quality stream, provides the service of afterwards having access to for the user with display device.
Further, described real-time streams encoder produces real-time streams according to detection module information: be used for displaying live view, and less demanding for picture quality, higher to requirement of real-time, need to and be transferred to immediately the rear end to the video real-time coding.
Further, described enhanced flow encoder is according to detection module information, produce enhanced flow: transmitting terminal utilizes event checking module to detect the zone (being area-of-interest) that event occurs in time period that a certain event occurs and this time period, and coding side can be strengthened encoding to this period specific region.Because enhanced flow mainly is the effect of collecting evidence in order to satisfy post-survey, thereby not high to requirement of real-time, but the picture quality of specific region is had relatively high expectations.Therefore enhanced flow is arranged in to transmit in a period of time after anomalous event occurs and gets final product.The code stream and period and the area information that comprise the specific time period specific region in the enhanced flow.
Compared with prior art, the present invention has following beneficial effect: the inventive method is except rejecting temporal information redundant in the video sequence, can also reject spatial information redundant in the video sequence, thereby realize the reinforcement coding of area-of-interest in the anomalous event, improve bandwidth availability ratio.Use this programme, so that the user interest information that originally can't see clearly in some video gets and can see clearly under certain bandwidth.
Description of drawings
Fig. 1 area-of-interest stream transmission system construction drawing;
Image segmentation when Fig. 2 does not use FMO;
The sheet group mapping mode FMO of Fig. 3 prospect and background;
Original Tcross sequence the 192nd frame of Fig. 4;
Double-deck ROI the 192nd frame of Fig. 5 Tcross;
Double-deck non-ROI the 192nd frame of Fig. 6 Tcross;
Fig. 7 Tcross single current the 192nd frame.
Embodiment
Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment is implemented under take technical solution of the present invention as prerequisite, provided detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to following embodiment.
Present embodiment is based on the JSVM platform, and area-of-interest in the abnormal period is strengthened coding, again the code stream that obtains is divided into basic layer bit stream and enhancement layer bitstream carries out stream transmission.Under limited bandwidth, in the video that recover the rear end, can more clearly see the content of area-of-interest in the abnormal period.
The area-of-interest double-stream coding transmission method that present embodiment relates to comprises following concrete steps:
The research of the adaptive monitor video high efficient coding of information degree of certainty, its main thought is: according to contrary Shannon information definition: information is deterministic increase; The tolerance of amount of information becomes degree of certainty by negentropy.In monitor video, the probability that user-defined high-level semantic (unusual invasion waits particular event) occurs, namely the degree of certainty of information becomes the valid metric of weighing this section video information significance level, has directly reflected the interest level of user to this information.The information degree of certainty is introduced Video coding, according to information degree of certainty size in the video, carry out adaptively the differentiation coding, reach the purpose of effective compressed information redundancy.Based on the information degree of certainty, instruct the scalable coding of monitor video on the dimensions such as time, space and quality, i.e. the adaptive scalable coding of information degree of certainty.Information is deterministic increase; The tolerance of amount of information becomes degree of certainty by negentropy.
Utility function is a concept on the economics the earliest.Two key issues are arranged in the Analysis of Policy Making: one is that uncertainty to the state of research phenomenon quantizes; The 2nd, to the various consequence assignment that may occur.In general, the uncertainty of state is described with the probability that various states occur, and the value of consequence appears in research then will use utility theory.Utility function is introduced video research, by utility function video is estimated.
By the restrictive condition of utility function optimal solution and bandwidth, find the solution specific coding parameter and the allocated bandwidth ratio of obtaining at last.
The system configuration that adopts in the present embodiment comprises the monitor video transmitting terminal as shown in Figure 1, network channel and monitor video receiving terminal.
Monitor video is carried out after the double fluid encapsulation, need after the encapsulation in the bit stream process network bandwidth resources to distribute.Specifically be allocated as follows:
When event checking module is judged the generation of anomalous event, and obtain the abnormal area information in the abnormal time section.Event checking module is issued the enhanced flow encoder with anomalous event and area information.Simultaneously, have corresponding algorithm and obtain adaptively best Data Rate Distribution scheme according to the ROI area size, and real-time streams code check and the QP value of encoding in the scheme passed to real-time encoder and enhanced encoder.Enhanced encoder can be strengthened coding to the anomalous event abnormal area, and code stream is stored.
When the anomalous event end, system enters without the anomalous event process.Encode according to resulting real-time coding code check in the scheme just this moment, and the encoder bit rate of real-time streams video is reduced, and reserves the transmission that bandwidth is used for enhanced flow.After high-quality stream transmission of video was finished, the encoder bit rate with the real-time streams encoder raised again, and all available network transmission bandwidths are used for the transmission of real-time streams video.
The derivation of " multithread of monitor video coding and transmission research " (Shanghai Communications University's Master's thesis, 2012), present embodiment provide with document difference code stream implication under mathematic(al) representation.
In the anomalous event process: the code check of real-time streams and high-quality stream and transmission occupied bandwidth parameter can be summarized by following formula:
R in the following formula
rBe real-time streams video frequency coding rate, B
rBe the transmission bandwidth that real-time streams takies, B is the total bandwidth that video monitoring system can be used.B
hThe transmission bandwidth that takies for high-quality stream.
After anomalous event finished, the code check of real-time streams coding reduced, and the transmission bandwidth that real-time streams takies reduces, and the bandwidth that has more is for the transmission of the high-quality stream of depositing.Can be represented by the formula.
B
r+B
h=B
Other defined variable K=T
d/ T
a, be the time ratio of non-anomalous event process and anomalous event process.Then the code check of each time period and respective bandwidth need satisfy following formula.B wherein
R, AEThe bandwidth of expression real-time streams after anomalous event, R
H, errorThe code check that high-rise ROI zone produces during the expression anomalous event, R
R, errorReal-time streams code check during the expression anomalous event, B
R, errorReal-time streams bandwidth during the expression anomalous event.
R
r,AE=B
r,AE<B
R
r，error=B
r，error=B
According to the code check that has proposed in the existing document for the SVC standard code of QP value, image fault is with the following formula of the relation of quantization step value.Wherein subscript 0 correspondence is basic layer, and subscript 1 corresponding enhancement layer, D are distortion (MSE), and R is the code stream code check, and Q is the coded quantization step-length.
D
1(Q
0,Q
1)≈D
0(Q
1)
R
1(Q
0,Q
1)=R
0(Q
1)-cR
0(Q
0)
In conjunction with following formula, according to the algorithm of following introduction, just can obtain the R in the non-anomalous event process again
rR with the anomalous event process
rAnd Q
hThe optimization of rate allocative decision.
Information is deterministic increase; The tolerance of amount of information becomes degree of certainty I by negentropy
r, shown in following formula:
I
r=logP
Evaluates video quality generally is divided into two aspects, evaluating objective quality and subjective quality assessments.Present embodiment proposes a comparatively reasonably objective indicator, is used for estimating the video quality under this special context of monitor video, and this objective indicator is ROI double fluid utility function.
The relation of PNSR and distortion is shown below:
In order to make it second dervative occur less than zero (quality that is video improves the embodiment that can occur edge effect at human eye) in the mathematics performance, be constructed as follows two functions the initial PSNR of video be correspondingly processed:
u
q，r=U
q，r(psnr
r)
u
q，h=U
q,h(psnr
h)
Wherein, u
Q, rThe video quality value of utility of real-time streams, psn
rBe the PSNR value of real-time streams, u
Q, rThe video quality value of utility of high-quality stream, psnr
hBe the PSNR value of high-quality stream, U
Q, r() be the video quality utility function of real-time streams, U
Q, h() be the video quality utility function of high-quality stream.
Use y=x
(1/n)Function construct first derivative greater than the zero minus function of second dervative, wherein the n value is larger, edge effect is more obvious.Therefore can get (n in the example
r, n
hGet respectively 3 and 2):
The final single frames total utility function of double fluid ROI transmission method can be expressed as following formula: u
i=w
Q, r* u
Q, r, i+ w
Q, h, roi* u
Q, h, roi, i* s
Roi, i+ w
Q, h, nroi* u
Q, h, nroi, i* S
Nroi, i
W wherein
Q, rThe expression actual monitored without in the anomalous event in real time code stream produce the weight of effectiveness, represented when occuring without anomalous event the degree of concern to video, w
Q, h, roi, w
Q, h, nroiRepresent respectively high-quality code stream ROI zone and the regional weight that produces effectiveness of non-ROI in the actual monitored anomalous event, representative can provide in the concrete value formula below different to area-of-interest and non-area-of-interest degree of concern in the actual monitored anomalous event.That introduces in every u value such as the front formula is identical, and the picture that representative obtains with certain coding parameter coding can bring user's corresponding (u
Q, rCorresponding to anomalous event coded portion, u
Q, h, roiCorresponding anomalous event encoding region of interest part, u
Q, h, nroiThe non-encoding region of interest part of corresponding anomalous event) full figure value of utility.S wherein
Roi, iAnd s
Nroi, iRepresent that respectively i frame roi zone and non-roi zone account for the percentage of gross area size, reacting is exactly video i frame area-of-interest shared space proportion in video integral body in the actual monitored system.And judge that by event checking module whether a zone needs really decide as the information degree of certainty in ROI zone according to this zone during ROI.Be shown below, wherein Squre
Roi, iRepresent i frame roi region area, Squre
WholeRepresent the i frame video gross area, wherein C is the thresholding that takes a decision as to whether the information degree of certainty in ROI zone, this thresholding means to only have when certain regional information degree of certainty during greater than certain thresholding, the user just with it as area-of-interest, need in the practical application to determine according to the different demands of user, wherein I is the information degree of certainty in zone, and this patent is judged based on the information degree of certainty, but is not this patent research emphasis.It is relevant that the occurrence of C is also chosen the algorithm that adopts with the I value.
At last, each frame of one section video flowing is weighted obtains last video utility function, be shown below.Wherein N represents actual one section totalframes that monitor video comprises, and other several variable connotations are identical with the before description to single frames total utility function.The i.e. average utility value of one section monitor video of E wherein, be one to the reference evaluation index of one section monitor video resulting video quality under certain bandwidth condition restriction.
Can know that by subjective experience area-of-interest is less, people are larger to the attention rate in zone, when area-of-interest expands whole figure to, reduce its attention rate is corresponding.In the method, for w
Q, h, roiValue be shown below, be a positive definite value for a certain video sequence.
T in the formula
ErrorThe frame number of anomalous event is got an average weighted effect in the expression actual monitored video in formula, obtains an average of area-of-interest area occupied ratio.W wherein
Q, h, roiWith S
Nroi, iThe implication of parameter provides in single frames total utility function representation.The weight that is area-of-interest changes to 1 by 4, and its value is to become the inverse correlation relation with the value of average area-of-interest area occupied ratio.In addition, w in the present embodiment
Q, r, w
Q, h, nroiAll get 1.
To sum up get final product-the final expression formula of E (u), carry out linear search with the Fibonacci method again and find the solution R
R, AE(the code check value of real-time code stream after the anomalous event) gets final product.
For the burst of image, macro block need to be arranged to the map information of burst.Generally, a two field picture divides one or more, the grand some macro blocks of bag in each sheet, and macro block is pressed raster scan order and is arranged.As shown in Figure 2.
In addition, H.264 provide the pattern of another kind of macroblock map, i.e. flexible macro-block ordering (FMO, Flexible Macro-block Ordering).The concept of this pattern using sheet group (Slice Group), the macro block that each sheet group comprises is determined by macro block division group mapping table in the parameter set.In the macro block division group mapping table each macro block there is sheet group ownership number, indicates which sheet group macro block belongs to.Each sheet group can have one or more bursts, and macro block presses in corresponding burst that raster scan order arranges in each burst.In addition, when not using the FMO mode, also can regard a kind of special circumstances of FMO as, namely whole figure only comprises the situation of a sheet group.
Coding for the ROI zone can be realized based on the FMO characteristic.Choosing adopts certain ROI extraction algorithm to obtain the area information of ROI, adopts the second pattern (seeing accompanying drawing 3) among the FMO, and namely prospect and background mode according to FMO characteristic and ROI area information, are divided into ROI sheet group and background plate group to image.Because the ROI zone of different sheet groups is absolute codings, can realize that by forbidding the prediction between the sheet group ROI zone code stream and the non-ROI zone code stream that generate are separate.The ROI area information that basis obtains in advance when encoding like this is as the foreground information of the 2nd kind of mode among the FMO.Basic layer is arranged the real-time streams code check and enhancement layer arranges the QP value, just can obtain two parts code stream at enhancement layer like this, i.e. the enhancing code stream in the enhancing code stream in ROI zone and non-ROI zone.By deleting the enhancing code stream in non-ROI zone, just remaining basic layer bit stream and ROI zone strengthen code stream, and two-part code stream decoding namely can obtain the ROI zone the more video sequence of high image quality.
Step 3 to same monitor video source, is carried out the two-way coding and is produced real-time streams video and high-quality stream video.
Real-time streams: be used for displaying live view, less demanding for picture quality, higher to requirement of real-time, need to and be transferred to immediately the rear end to the video real-time coding.
Enhanced flow: transmitting terminal utilizes event checking module to detect the zone (being area-of-interest) that event occurs in time period that a certain event occurs and this time period, and coding side can be strengthened coding to this period specific region.Because enhanced flow mainly is the effect of collecting evidence in order to satisfy post-survey, thereby not high to requirement of real-time, but the picture quality of specific region is had relatively high expectations.Therefore enhanced flow is arranged in to transmit in a period of time after anomalous event occurs and gets final product.The code stream and period and the area information that comprise the specific time period specific region in the enhanced flow.
As shown in Figure 1, this stream transmission system configuration comprises the monitor video transmitting terminal, network channel and monitor video receiving terminal.
Transmitting terminal comprises: event and regional detection module, real-time streams encoder, enhanced flow encoder, real-time Transmission module, non real-time transport module.Two encoders produce respectively above-mentioned real-time streams and enhanced flow, and two transport modules carry out encapsulated delivery to real-time streams and enhanced flow respectively.Event and regional detection module be for detection of time of anomalous event and zone, and encode with real-time streams with gained information control enhanced flow coding, the information source code check adjusted to rapidly consistent with the effective transmission bandwidth of channel.
Receiving terminal comprises: real-time streams decoder, memory module and display module.Real-time streams decoder and display module provide the service of real-time demonstration for the user.Memory module can be deposited real-time streams and thereby enhanced flow forms high-quality stream, provides the service of afterwards having access to for the user with display device.
Describedly the information source code check is adjusted to rapidly consistent with the effective transmission bandwidth of channel, referred to: at first be the cleaning to the transmitting terminal buffer memory, abandon being subject to the bag that lost package pollutes in the buffer memory, avoid the transmission of invalid data and waste bandwidth; Next is that the encoder bit rate of coding module is controlled, and unites realization by revising the measures such as quantization parameter, image frame per second and image resolution ratio, and coding module encoding bit rate and the target bit rate that is estimated to are complementary.
Monitor video is carried out after the double fluid encapsulation, need after the encapsulation in the bit stream process network bandwidth resources to distribute.Concrete allocation strategy is as follows:
When event checking module is judged the generation of anomalous event, and obtain the abnormal area information in the abnormal time section.Event checking module is issued the enhanced flow encoder with anomalous event and area information.Simultaneously, have corresponding algorithm and obtain adaptively best Data Rate Distribution scheme according to the ROI area size, and real-time streams code check and the QP value of encoding in the scheme passed to real-time encoder and enhanced encoder.Enhanced encoder can be strengthened coding to the anomalous event abnormal area, and code stream is stored.
When the anomalous event end, system enters without the anomalous event process.Encode according to resulting real-time coding code check in the scheme just this moment, and the encoder bit rate of real-time streams video is reduced, and reserves the transmission that bandwidth is used for enhanced flow.After high-quality stream transmission of video was finished, the encoder bit rate with the real-time streams encoder raised again, and all available network transmission bandwidths are used for the transmission of real-time streams video.
Concrete bandwidth parameter is determined according to step 1 in the distribution.
Use the test result of this programme in this enforcement of table 1 in the T-cross sequence variation event
The contrast of the concrete visible accompanying drawing 4,5,6,7 of image.
Can find out by contrast, adopt method of the present invention, can realize that code check more reasonably distributes on time-domain and the spatial domain, the picture quality of picture when improving anomalous event, make area-of-interest that higher picture quality be arranged, satisfy better and satisfy the afterwards demand of evidence obtaining under the finite bandwidth condition.
Although content of the present invention has been done detailed introduction by above preferred embodiment, will be appreciated that above-mentioned description should not be considered to limitation of the present invention.After those skilled in the art have read foregoing, for multiple modification of the present invention with to substitute all will be apparent.Therefore, protection scope of the present invention should be limited to the appended claims.
Claims (10)
1. area-of-interest double-stream coding transmission method, it is characterized in that, described area-of-interest double-stream coding transmission, refer to: area-of-interest adopts higher code check to strengthen coding, but not area-of-interest then adopts general code check to encode, the Basic Flow that produces and enhanced flow are by the distribution on the time-domain, so that it can finish transmission in the finite bandwidth situation; Described method comprises the steps:
Step 1 on the basis that obtains interested area information and abnormal information, obtains the specific coding parameter of video-frequency basic flow and enhanced flow by utility function, obtains simultaneously best Bandwidth Allocation Policy;
Step 2 on the basis of step 1, is strengthened coding to the video content of area-of-interest in the abnormal period;
Step 3, the band-width tactics according to step 1 is distributed carries out stream transmission.
2. area-of-interest double-stream coding transmission method according to claim 1, it is characterized in that, described utility function, refer to: the difference that fully takes into account in the video sequence the contained information significance level of the information of different period zoness of different, wherein the abnormal period area-of-interest has the highest significance level, also have the highest weight in utility function, but not the information significance level of abnormal period is lower, the weight of corresponding distribution is also less.
3. area-of-interest double-stream coding transmission method according to claim 1 is characterized in that, described stream transmission, and concrete grammar is: when anomalous event occurs, only transmit basic layer bit stream; After anomalous event finishes, reduce the code check of basic layer bit stream, make unnecessary bandwidth be used for the enhancement layer bitstream of transmission abnormality event, its basic enhancement layer bitstream only comprises the enhancing information of area-of-interest; Judge the generation of anomalous event when event checking module, and obtain abnormal area information in the abnormal time section, event checking module is issued the enhanced flow encoder with anomalous event and area information, simultaneously, obtain adaptively best Data Rate Distribution scheme according to the ROI area size, and real-time streams code check and the QP value of encoding in the scheme passed to real-time encoder and enhanced encoder, enhanced encoder can be strengthened coding to the anomalous event abnormal area, and code stream is stored; When anomalous event finishes, system enters without the anomalous event process, and encode according to resulting real-time coding code check in the scheme just this moment, and the encoder bit rate of real-time streams video is reduced, and reserves the transmission that bandwidth is used for enhanced flow; After high-quality stream transmission of video was finished, the encoder bit rate with the real-time streams encoder raised again, and all available network transmission bandwidths are used for the transmission of real-time streams video.
4. area-of-interest double-stream coding transmission method according to claim 3, it is characterized in that, the described Data Rate Distribution scheme that obtains adaptively the best according to the ROI area size, refer to: front-end detection be whether a zone is the probability of user's area-of-interest, probability determines whether it is area-of-interest by double-current ROI utility function thus, and the basic layer of transmission bandwidth with coding parameter, Basic Flow and the enhanced flow of enhancement layer distributes after adjusting adaptively in this utility function.
5. area-of-interest double-stream coding transmission method according to claim 4 is characterized in that, described double-current ROI utility function is specially:
The final single frames total utility function representation of double fluid ROI transmission method is following formula:
u
i=w
q,r*u
q，r，i+w
q,h,roi*u
q,h,roi，i*s
roi，i+w
q,h，nroi*u
q,h，nroi，i*S
nroi，i
W wherein
Q, rThe expression actual monitored without in the anomalous event in real time code stream produce the weight of effectiveness, represented when occuring without anomalous event the degree of concern to video, w
Q, h, roi, w
Q, h, nroiRepresent that respectively high-quality code stream ROI zone and non-ROI zone in the actual monitored anomalous event produce the weight of effectiveness, representative is to different to area-of-interest and non-area-of-interest degree of concern in the actual monitored anomalous event; The picture that every u value representative obtains with certain coding parameter coding can bring user's corresponding full figure value of utility, u
Q, rCorresponding to anomalous event coded portion, u
Q, h, roiCorresponding anomalous event encoding region of interest part, u
Q, h, nroiThe non-encoding region of interest part of corresponding anomalous event;
s
Roi, iAnd s
Nroi, iRepresent that respectively i frame roi zone and non-roi zone account for the percentage of gross area size, reacting is exactly video i frame area-of-interest shared space proportion in video integral body in the actual monitored system; And judge that by event checking module whether a zone needs according to really deciding as the information degree of certainty in ROI zone in this zone during ROI; Squ
Reroi, iRepresent i frame roi region area, Squre
WholeRepresent the i frame video gross area; C is the thresholding that takes a decision as to whether the information degree of certainty in ROI zone, and I i.e. the information degree of certainty in zone;
At last, each frame of one section video flowing is weighted obtains last video utility function, be shown below, wherein N represents actual one section totalframes that monitor video comprises, other several variable connotations are identical with the before description to single frames total utility function, E i.e. the average utility value of one section monitor video, be one to the reference evaluation index of one section monitor video resulting video quality under certain bandwidth condition restriction;
For W
Q, h, roiValue be shown below, be a positive definite value for a certain video sequence,
T in the formula
ErrorThe frame number of anomalous event in the expression actual monitored video, in formula, get an average weighted effect, obtain an average of area-of-interest area occupied ratio, namely the weight of area-of-interest changes to 1 by 4, and its value is to become the inverse correlation relation with the value of average area-of-interest area occupied ratio;
To sum up get final product-the final expression formula of E (u), carry out linear search with the Fibonacci method again and find the solution R
R, AE, R
R, AECode check value for real-time code stream after the anomalous event.
6. each described area-of-interest double-stream coding transmission method according to claim 1-5, it is characterized in that, described video content to area-of-interest in the abnormal period is strengthened coding, be specially: the area information that obtains ROI, adopt the second pattern among the FMO, be prospect and background mode, according to FMO characteristic and ROI area information, image be divided into ROI sheet group and background plate group; The ROI zone of different sheet groups is absolute codings, by forbidding that the prediction between the sheet group can realize that the ROI zone code stream and the non-ROI zone code stream that generate are separate, the ROI area information that basis obtains in advance when encoding like this is as the foreground information of the second pattern among the FMO; Basic layer is arranged the real-time streams code check and enhancement layer arranges the QP value, just can obtain two parts code stream at enhancement layer, be the enhancing code stream in ROI zone and the enhancing code stream in non-ROI zone, by deleting the enhancing code stream in non-ROI zone, just remaining basic layer bit stream and ROI zone strengthen code stream, and two-part code stream decoding namely obtains the ROI zone the more video sequence of high image quality.
7. area-of-interest double-stream coding transmission method according to claim 6, it is characterized in that, described ROI area information is time dependent, be that the ROI zone is dynamic, when the ROI area information changes, need to remodify the parameter that MBA is set, the video in the new time is encoded, remove the code stream in the non-ROI of enhancement layer zone behind the coding, the video sequence that is stitched together behind a plurality of code stream decodings that obtain has at last namely been realized the effect that dynamic ROI zone strengthens.
8. an area-of-interest double-stream coding transmission system that is used for the described method of claim 1-7 is characterized in that, comprising: the monitor video transmitting terminal, and network channel and monitor video receiving terminal, wherein:
The monitor video transmitting terminal comprises: event and regional detection module, real-time streams encoder, enhanced flow encoder, real-time Transmission module and non real-time transport module; Real-time streams encoder, enhanced flow encoder produce respectively above-mentioned real-time streams and enhanced flow, and real-time Transmission module and non real-time transport module carry out encapsulated delivery to real-time streams and enhanced flow respectively; Event and regional detection module be for detection of time of anomalous event and zone, and encode with real-time streams with gained information control enhanced flow coding;
Network channel and monitor video receiving terminal comprise: real-time streams decoder, memory module and display module; Real-time streams decoder and display module provide the service of real-time demonstration for the user; Memory module can be deposited real-time streams and thereby enhanced flow forms high-quality stream, provides the service of afterwards having access to for the user with display device.
9. area-of-interest double-stream coding transmission system according to claim 8, it is characterized in that, described real-time streams encoder is according to detection module information, produce real-time streams, be used for displaying live view, less demanding for picture quality, higher to requirement of real-time, need to and be transferred to immediately the rear end to the video real-time coding.
10. according to claim 8 or 9 described area-of-interest double-stream coding transmission systems, it is characterized in that, described enhanced flow encoder is according to detection module information, produce enhanced flow, it is area-of-interest that transmitting terminal utilizes event checking module to detect the zone that event occurs in time period that a certain event occurs and this time period, coding side can be strengthened coding to this period specific region, transmit in a period of time after enhanced flow is arranged in anomalous event and occurs, comprise code stream and period and the area information of specific time period specific region in the enhanced flow.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210279005.3A CN102905200B (en) | 2012-08-07 | 2012-08-07 | Video interesting region double-stream encoding and transmitting method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210279005.3A CN102905200B (en) | 2012-08-07 | 2012-08-07 | Video interesting region double-stream encoding and transmitting method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102905200A true CN102905200A (en) | 2013-01-30 |
| CN102905200B CN102905200B (en) | 2015-01-14 |
Family
ID=47577204
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210279005.3A Expired - Fee Related CN102905200B (en) | 2012-08-07 | 2012-08-07 | Video interesting region double-stream encoding and transmitting method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102905200B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297754A (en) * | 2013-05-02 | 2013-09-11 | 上海交通大学 | Monitoring video self-adaption interesting area coding system |
| WO2016095696A1 (en) * | 2014-12-15 | 2016-06-23 | 江南大学 | Video-outline-based method for monitoring scalable coding of video |
| CN106791910A (en) * | 2016-12-02 | 2017-05-31 | 浙江宇视科技有限公司 | Frame of video processing method and processing device |
| CN106937118A (en) * | 2017-03-13 | 2017-07-07 | 西安电子科技大学 | A kind of bit rate control method being combined based on subjective area-of-interest and time-space domain |
| CN107925765A (en) * | 2015-08-19 | 2018-04-17 | 索尼公司 | Dispensing device, sending method, reception device and method of reseptance |
| CN108810538A (en) * | 2018-06-08 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Method for video coding, device, terminal and storage medium |
| CN110022463A (en) * | 2019-04-11 | 2019-07-16 | 重庆紫光华山智安科技有限公司 | Video interested region intelligent coding method and system are realized under dynamic scene |
| CN110288592A (en) * | 2019-07-02 | 2019-09-27 | 中南大学 | A method of the zinc flotation dosing state evaluation based on probability semantic analysis model |
| CN110401847A (en) * | 2019-07-17 | 2019-11-01 | 咪咕文化科技有限公司 | Compression storage method, electronic equipment and system for cloud DVR video |
| CN111027448A (en) * | 2019-12-04 | 2020-04-17 | 成都考拉悠然科技有限公司 | Video behavior category identification method based on time domain inference graph |
| CN111263192A (en) * | 2018-11-30 | 2020-06-09 | 华为技术有限公司 | Video processing method and related equipment |
| CN112307943A (en) * | 2020-10-29 | 2021-02-02 | 宁波海棠信息技术有限公司 | Water area man-boat target detection method, system, terminal and medium |
| CN112367520A (en) * | 2020-11-11 | 2021-02-12 | 郑州师范学院 | Video quality diagnosis system based on artificial intelligence |
| CN113038127A (en) * | 2021-03-30 | 2021-06-25 | 福州大学 | ROI-based real-time video multi-path parallel transmission code rate control method |
| CN113132730A (en) * | 2019-12-30 | 2021-07-16 | 北京积加科技有限公司 | Intelligent video data compression method and device based on multi-frame content |
| CN108476324B (en) * | 2015-10-08 | 2021-10-29 | 皇家Kpn公司 | Method, computer and medium for enhancing regions of interest in video frames of a video stream |
| CN116112645A (en) * | 2023-04-11 | 2023-05-12 | 重庆华悦生态环境工程研究院有限公司深圳分公司 | Multi-image transmission method and device for reservoir environment |
| CN116506664A (en) * | 2023-06-25 | 2023-07-28 | 北京淳中科技股份有限公司 | Video code stream generation method and device, electronic equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778265A (en) * | 2010-01-13 | 2010-07-14 | 上海交通大学 | Exception event detection and double-stream coding-based video monitoring method |
| CN101945287A (en) * | 2010-10-14 | 2011-01-12 | 杭州华三通信技术有限公司 | ROI encoding method and system thereof |
| EP2429190A1 (en) * | 2010-09-13 | 2012-03-14 | NTT DoCoMo, Inc. | Method and apparatus for transferring a video stream |
| CN102612818A (en) * | 2009-11-17 | 2012-07-25 | 高通股份有限公司 | Channel quality indicator design for multiple-user multiple-input and multiple-output in high-speed packet access systems |
| CN102625082A (en) * | 2012-02-29 | 2012-08-01 | 中山大学 | Video monitoring system |
-
2012
- 2012-08-07 CN CN201210279005.3A patent/CN102905200B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102612818A (en) * | 2009-11-17 | 2012-07-25 | 高通股份有限公司 | Channel quality indicator design for multiple-user multiple-input and multiple-output in high-speed packet access systems |
| CN101778265A (en) * | 2010-01-13 | 2010-07-14 | 上海交通大学 | Exception event detection and double-stream coding-based video monitoring method |
| EP2429190A1 (en) * | 2010-09-13 | 2012-03-14 | NTT DoCoMo, Inc. | Method and apparatus for transferring a video stream |
| CN101945287A (en) * | 2010-10-14 | 2011-01-12 | 杭州华三通信技术有限公司 | ROI encoding method and system thereof |
| CN102625082A (en) * | 2012-02-29 | 2012-08-01 | 中山大学 | Video monitoring system |
Non-Patent Citations (1)
| Title |
|---|
| 於佳健: "监控视频的多流编码与传输技术研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297754A (en) * | 2013-05-02 | 2013-09-11 | 上海交通大学 | Monitoring video self-adaption interesting area coding system |
| CN103297754B (en) * | 2013-05-02 | 2016-06-01 | 上海交通大学 | A kind of monitor video self-adaptation encoding region of interest system |
| WO2016095696A1 (en) * | 2014-12-15 | 2016-06-23 | 江南大学 | Video-outline-based method for monitoring scalable coding of video |
| CN107925765B (en) * | 2015-08-19 | 2022-01-18 | 索尼公司 | Transmission device, transmission method, reception device, and reception method |
| CN107925765A (en) * | 2015-08-19 | 2018-04-17 | 索尼公司 | Dispensing device, sending method, reception device and method of reseptance |
| US11412254B2 (en) | 2015-08-19 | 2022-08-09 | Sony Corporation | Transmission device, transmission method, reception device and reception method |
| CN108476324B (en) * | 2015-10-08 | 2021-10-29 | 皇家Kpn公司 | Method, computer and medium for enhancing regions of interest in video frames of a video stream |
| CN106791910A (en) * | 2016-12-02 | 2017-05-31 | 浙江宇视科技有限公司 | Frame of video processing method and processing device |
| CN106791910B (en) * | 2016-12-02 | 2019-11-05 | 浙江宇视科技有限公司 | Video frame processing method and processing device |
| CN106937118B (en) * | 2017-03-13 | 2019-09-13 | 西安电子科技大学 | A kind of bit rate control method combined based on subjective area-of-interest and time-space domain |
| CN106937118A (en) * | 2017-03-13 | 2017-07-07 | 西安电子科技大学 | A kind of bit rate control method being combined based on subjective area-of-interest and time-space domain |
| CN108810538A (en) * | 2018-06-08 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Method for video coding, device, terminal and storage medium |
| CN108810538B (en) * | 2018-06-08 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Video coding method, device, terminal and storage medium |
| CN111263192A (en) * | 2018-11-30 | 2020-06-09 | 华为技术有限公司 | Video processing method and related equipment |
| CN110022463A (en) * | 2019-04-11 | 2019-07-16 | 重庆紫光华山智安科技有限公司 | Video interested region intelligent coding method and system are realized under dynamic scene |
| CN110288592A (en) * | 2019-07-02 | 2019-09-27 | 中南大学 | A method of the zinc flotation dosing state evaluation based on probability semantic analysis model |
| CN110288592B (en) * | 2019-07-02 | 2021-03-02 | 中南大学 | Zinc flotation dosing state evaluation method based on probability semantic analysis model |
| CN110401847A (en) * | 2019-07-17 | 2019-11-01 | 咪咕文化科技有限公司 | Compression storage method, electronic equipment and system for cloud DVR video |
| CN111027448A (en) * | 2019-12-04 | 2020-04-17 | 成都考拉悠然科技有限公司 | Video behavior category identification method based on time domain inference graph |
| CN111027448B (en) * | 2019-12-04 | 2023-05-05 | 成都考拉悠然科技有限公司 | Video behavior category identification method based on time domain inference graph |
| CN113132730A (en) * | 2019-12-30 | 2021-07-16 | 北京积加科技有限公司 | Intelligent video data compression method and device based on multi-frame content |
| CN112307943A (en) * | 2020-10-29 | 2021-02-02 | 宁波海棠信息技术有限公司 | Water area man-boat target detection method, system, terminal and medium |
| CN112307943B (en) * | 2020-10-29 | 2022-06-03 | 宁波海棠信息技术有限公司 | Water area man-boat target detection method, system, terminal and medium |
| CN112367520A (en) * | 2020-11-11 | 2021-02-12 | 郑州师范学院 | Video quality diagnosis system based on artificial intelligence |
| CN112367520B (en) * | 2020-11-11 | 2022-06-21 | 郑州师范学院 | Video quality diagnosis system based on artificial intelligence |
| CN113038127A (en) * | 2021-03-30 | 2021-06-25 | 福州大学 | ROI-based real-time video multi-path parallel transmission code rate control method |
| CN116112645A (en) * | 2023-04-11 | 2023-05-12 | 重庆华悦生态环境工程研究院有限公司深圳分公司 | Multi-image transmission method and device for reservoir environment |
| CN116112645B (en) * | 2023-04-11 | 2023-11-21 | 重庆华悦生态环境工程研究院有限公司深圳分公司 | Multi-image transmission method and device for reservoir environment |
| CN116506664A (en) * | 2023-06-25 | 2023-07-28 | 北京淳中科技股份有限公司 | Video code stream generation method and device, electronic equipment and medium |
| CN116506664B (en) * | 2023-06-25 | 2023-09-15 | 北京淳中科技股份有限公司 | Video code stream generation method and device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102905200B (en) | 2015-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102905200B (en) | Video interesting region double-stream encoding and transmitting method and system | |
| CN101552913B (en) | Multi-channel video communication system and processing method | |
| CN101742321B (en) | Layer decomposition-based Method and device for encoding and decoding video | |
| CN102724498B (en) | The coding method of the scalability techniques of content-based information and equipment | |
| EP2285122B1 (en) | A method and device for reconstructing a sequence of video data after transmission over a network | |
| CN102857746B (en) | Loop filtering decoding method and device | |
| CN101282479A (en) | Method for encoding and decoding airspace with adjustable resolution based on interesting area | |
| CN101707701B (en) | Multi-stream information transmission system and transmission method thereof | |
| CN103248893A (en) | Method for quick inter-frame transcoding from H. 264/AVC standard to HEVC standard and transcoder thereof | |
| CN1460231A (en) | Video coding method | |
| CN103141099A (en) | Optimized filter selection for reference picture processing | |
| CN104704835A (en) | Method and apparatus of motion information management in video coding | |
| CN101674475A (en) | Self-adapting interlayer texture prediction method of H.264/SVC | |
| CN101313588A (en) | Scalability techniques based on content information | |
| CN104995916A (en) | Video data decoding method and video data decoding apparatus | |
| CN102006483B (en) | Video coding and decoding method and device | |
| CN101895753B (en) | Network congestion degree based video transmission method, system and device | |
| Bai et al. | Multiple description video coding based on human visual system characteristics | |
| CN105765978A (en) | Method for encoding/decoding image and device using same | |
| Hu et al. | Multiple description coding for best-effort delivery of light field video using GNN-based compression | |
| CN101389031A (en) | Transformation coefficient processing method and apparatus | |
| CN103475878A (en) | Video coding method and encoder | |
| CN104135662B (en) | Improved H.264 compressed encoding method for video file under limited bandwidth and emission rate condition | |
| CN103402119B (en) | A kind of SVC code stream extracting method towards transmission and system | |
| Lian et al. | Rate and quality control with embedded coding for mobile robot with visual patrol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150114 Termination date: 20210807 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |