CN106454432B - A kind of video frame treating method and apparatus - Google Patents
A kind of video frame treating method and apparatus Download PDFInfo
- Publication number
- CN106454432B CN106454432B CN201610906960.3A CN201610906960A CN106454432B CN 106454432 B CN106454432 B CN 106454432B CN 201610906960 A CN201610906960 A CN 201610906960A CN 106454432 B CN106454432 B CN 106454432B
- Authority
- CN
- China
- Prior art keywords
- video frame
- grade
- current video
- frame
- buffer queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
- H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
Abstract
The present embodiments relate to field of image processing more particularly to a kind of video frame treating method and apparatus, ignore valid data transmission to solve the problem of the preferential fluency for guaranteeing video in network bandwidth deficiency.In the embodiment of the present invention, determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；According to the congestion level of video frame buffer queue, the highest level of discardable video frame is determined；The video frame for stating the highest level that video frame buffer queue middle grade is less than or equal to discardable video frame is abandoned, the grade of each video frame in video frame buffer queue is determined according at least to the grade of event scenarios is specified included in video frame；Valid data are reduced by the probability of safe transmission because loss of effective data bring is lost when to improve network bandwidth deficiency.
Description
Technical field
The present embodiments relate to field of image processing more particularly to a kind of video frame treating method and apparatus.
Background technique
It is constantly improve with the continuous development of network technology with network infrastructure, safety monitoring and network technology are melted
It closes and also increasingly gos deep into, rely on the wireless access technologys such as internet, 4G/Wifi, more and more users can look into whenever and wherever possible
See real time monitoring audio-video.However, all there are many unstable factors, such as link in either internet or wireless network
Congestion, electromagnetic interference, node failure etc., these factors can all cause the fluctuation of network bandwidth.In the prior art, usually used
It is opened up on dynamic random access memory (Dynamic Random Access Memory, abbreviation DRAM) and sends buffer area
Technology smooths out network fluctuation, but bigger when fluctuating, and the media data of buffering is more than when sending buffer area, to be bound to lose
Part of media data.
It is clear that, in conjunction with audio-video frequency media data characteristics, passing through reasonable matchmaker how under limited network bandwidth
Body frame losing control method transmits data more significant to user, is a very important project.But current big portion
The technology divided all is that video frame is marked according to Decoded Reference relationship, the preferential fluency for guaranteeing video, and has ignored peace
The core value of anti-monitoring --- data validity.
Summary of the invention
The embodiment of the present invention provides a kind of video frame treating method and apparatus, excellent to solve in network bandwidth deficiency
The problem of first guaranteeing the fluency of video and ignoring valid data transmission.
The embodiment of the present invention provides a kind of video frame processing method, comprising:
Determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；
According to the congestion level of video frame buffer queue, the highest level of discardable video frame is determined；
The video frame for stating the highest level that video frame buffer queue middle grade is less than or equal to discardable video frame is abandoned,
The grade of each video frame in video frame buffer queue is come according at least to the grade of specified event scenarios included in video frame
It determines.
Optionally it is determined that current video frame is can independent decoded video frame, comprising:
According to the type of current video frame judge current video frame whether be can independent decoded video frame, and determine current
The initial grade of video frame；
The video frame that video frame buffer queue middle grade is less than or equal to the highest level of discardable video frame is abandoned it
Afterwards, further includes:
Current video frame is put into video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
Optionally, further includes:
If current video frame be not can independent decoded video frame, by video frame buffer queue with current video frame category
In same picture group and grade lower than the grade of the video frame of the initial grade of current video frame be revised as current video frame just
Beginning grade；
Current video frame is added in video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
Optionally, the grade for specifying event scenarios is that basis is preset to the degree of concern of specified event scenarios, right
The the degree of concern of specified event scenarios the high then to specify the higher grade of event scenarios；
Determine the congestion level in video frame buffer queue, according to the congestion level of video frame buffer queue, determination can lose
Abandon the highest level of video frame, comprising:
According to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, discardable view is determined
The highest level of frequency frame.
Optionally, the initial grade of current video frame is determined in the following manner, comprising:
It whether detects in current video frame comprising specified event scenarios；
If so, determining each specified event in current video frame according to the class information of configured specified event scenarios
The grade of scene；
By the highest level in the grade of each specified event scenarios in current video frame, it is determined as the first of current video frame
Beginning grade.
The embodiment of the present invention provides a kind of video frame processing apparatus, comprising:
Judging unit, for determine current video frame be can independent decoded video frame when, determine video frame buffer team
The congestion level of column；
Determination unit determines the highest level of discardable video frame for the congestion level according to video frame buffer queue；
Processing unit, for video frame buffer queue middle grade to be less than or equal to the highest level of discardable video frame
Video frame abandons, and the grade of each video frame in video frame buffer queue is according at least to specified algebra of events included in video frame
The grade of scape determines.
Optionally, judging unit is specifically used for:
According to the type of current video frame judge current video frame whether be can independent decoded video frame, and determine current
The initial grade of video frame；
Optionally, processing unit is also used to:
It is abandoned in the video frame for the highest level that video frame buffer queue middle grade is less than or equal to discardable video frame
Later, current video frame is put into video frame buffer queue, and marking the grade of current video frame is the first of current video frame
Beginning grade.
Optionally, processing unit is also used to:
If current video frame be not can independent decoded video frame, by video frame buffer queue with current video frame category
In same picture group and grade lower than the grade of the video frame of the initial grade of current video frame be revised as current video frame just
Beginning grade；
Current video frame is added in video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
Optionally it is determined that unit is specifically used for:
The grade that specified event scenarios are preset according to the degree of concern to specified event scenarios, to specified event scenarios
Degree of concern more it is high then specify event scenarios higher grade；
According to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, discardable view is determined
The highest level of frequency frame.
Optionally, judging unit is specifically used for:
It whether detects in current video frame comprising specified event scenarios；
If so, determining each specified event in current video frame according to the class information of configured specified event scenarios
The grade of scene；
By the highest level in the grade of each specified event scenarios in current video frame, it is determined as the first of current video frame
Beginning grade.
The embodiment of the present invention provides a kind of video frame treating method and apparatus, is determining that current video frame is independently to decode
Video frame when, determine the congestion level of video frame buffer queue；According to the congestion level of video frame buffer queue, determination can lose
Abandon the highest level of video frame；Video frame buffer queue middle grade is less than or equal to the view of the highest level of discardable video frame
Frequency frame abandons, and the grade of each video frame in video frame buffer queue is according at least to specified event scenarios included in video frame
Grade determine.In order to ensure still can be to remaining after carrying out part frame losing to picture group in the embodiment of the present invention
Video frame is decoded, thus will judge present frame for can independent decoded video frame when as the condition of starting frame losing, therefore
Determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；Due in difference
Network environment in, the congestion level of network bandwidth be it is different, thus, under different congestion levels, discardable view
Frequency frame is also different, therefore the highest level of discardable video frame need to be determined according to the congestion level of video frame buffer queue, will regard
The video frame that frequency frame buffering queue middle grade is less than or equal to the highest level of discardable video frame abandons；Due to video frame buffer
The grade of each video frame is at least according to specifying the grade of event scenarios to determine in video frame, thus carrying out frame losing in queue
When, the video frame preferentially abandoned is the junior video frame comprising specified event scenarios, to improve network bandwidth deficiency
When valid data by the probability of safe transmission, reduce because loss of effective data bring is lost.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced.
Fig. 1 is a kind of method flow schematic diagram of video frame processing provided in an embodiment of the present invention；
Fig. 1 a is the schematic diagram provided in an embodiment of the present invention stopped when being tagged to queue head；
Fig. 1 b is the schematic diagram stopped when label provided in an embodiment of the present invention encounters the I frame of same GOP；
When Fig. 1 c is that label provided in an embodiment of the present invention encounters video frame buffer queue middle grade higher than current video frame
The schematic diagram of stopping；
Fig. 2 is the method flow schematic diagram of another video frame processing provided in an embodiment of the present invention；
Fig. 3 is a kind of structural schematic diagram of the device of video frame processing provided in an embodiment of the present invention.
Specific embodiment
In order to which the purpose of the present invention, technical solution and beneficial effect is more clearly understood, below in conjunction with attached drawing and implementation
Example, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only used to explain this hair
It is bright, it is not intended to limit the present invention.
Fig. 1 illustrates a kind of method flow of video frame processing provided in an embodiment of the present invention, as shown in Figure 1,
The following steps are included:
Step S101, determine current video frame be can independent decoded video frame when, determine video frame buffer queue
Congestion level；
Step S102 determines the highest level of discardable video frame according to the congestion level of video frame buffer queue；
Video frame buffer queue middle grade is less than or equal to the video of the highest level of discardable video frame by step S103
Frame abandons, and the grade of each video frame in video frame buffer queue is according at least to specified event scenarios included in video frame
Grade determines.
In specific implementation, the type of video frame has can independent decoded video frame (such as I frame) and can not independent decoded view
Frequency frame (such as P frame), the view used in Motion Picture Experts Group (Moving Picture Experts Group, abbreviation MPEG)
In frequency compress technique, I frame is usually first frame of each picture group (Group of Pictures, abbreviation GOP), by suitable
Degree ground frame data compression, as the reference point of random access, the I frame delineation details of image background and moving person, when decoding only
With the data of I frame with regard to restructural complete image.I frame does not have to refer to other video frames, can independently be solved when being decoded
Code.P frame is to be come according to the difference of this frame and adjacent former frame (I frame or P frame) come distilled edition frame data, i.e. interframe compression
Improve compression ratio.The adjacent former frame of the decoding dependency of P frame, cannot independently be decoded.
Determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；
Collected video frame is parsed, judges that current video frame is P frame or I frame, when determining current video frame is I frame, determines view
The congestion level of frequency buffering queue.In specific implementation, current video frame be can independent decoded video frame when starting frame losing sentence
It is disconnected, it may insure still be decoded remaining video frame after carrying out part frame losing to a picture group in this way.This hair
The judgement of current video frame type can be periodically carried out in bright embodiment, the setting period can determine according to concrete condition.
The congestion level of video buffer queue can judge according to the memory service condition of video buffer unit, can also use existing skill
Other judgment modes in art, the embodiment of the present invention are not specifically limited.
According to the congestion level of video frame buffer queue, the highest level of discardable video frame is determined；In specific implementation, no
With network environment under, the congestion level of video buffer queue be it is different, for congestion different in video buffer queue
The grade of grade, the corresponding video frame abandoned is also different, so in the congestion level for judging current video buffering queue every time
Afterwards, the highest level of discardable video frame is obtained, in case subsequent frame losing link refers to.It is determined according to different congestion levels different
Frame losing strategy can guarantee the fluency and video transmission quality of network transmission as far as possible.
The video frame that video frame buffer queue middle grade is less than or equal to the highest level of discardable video frame is abandoned, depending on
The grade of each video frame in frequency frame buffering queue is true according at least to specifying the grade of event scenarios included in video frame
It is fixed.In specific implementation, when frame losing, can directly abandon the video frame that all frame grades are less than corresponding congestion level, can also be with sublevel
Duan Jinhang frame losing only abandons the frame that partial frame grade is less than corresponding congestion level such as when first time reaching corresponding congestion level,
All inferior grade video frames are abandoned when repeatedly triggering the congestion level again.The grade of each video frame in video frame buffer queue is extremely
It is to be determined according to the grade of event scenarios is specified included in video frame less, event scenarios is specified included in video frame
Grade be according to being set to the degree of concern of specified event scenarios.By being determined according to the event scenarios in video frame
The grade of video frame, rather than be to carry out frame losing operation according to video frame type, it ensure that the transmission to effective video frame.For thing
Demand of the grade setting of part scene when can be according to concrete application is configured.
By the above method, in network bandwidth deficiency, determined according to the congestion level of video frame buffer queue discardable
The highest level of video frame abandons the video that video buffer queue middle grade is less than or equal to the highest level of discardable video frame
Frame remains the higher video frame of video frame buffer queue middle grade and is sent in network, guarantees the prioritised transmission of valid data,
It reduces because the loss bring of valid data is lost.
Optionally, the initial grade of current video frame is determined in the following manner, comprising:
It whether detects in current video frame comprising specified event scenarios；
If so, determining each specified event in current video frame according to the class information of configured specified event scenarios
The grade of scene；By the highest level in the grade of each specified event scenarios in current video frame, it is determined as current video frame
Initial grade；
Optionally, the grade for specifying event scenarios is that basis is preset to the degree of concern of specified event scenarios, right
The the degree of concern of specified event scenarios the high then to specify the higher grade of event scenarios.
In specific implementation, specified event scenarios refer mainly to the event scenarios of user's concern, and user is according to oneself to specified thing
The degree of concern of part scene configures the grade of specified event scenarios.One video frame can include the algebra of events of multiple types simultaneously
Scape, the type of event scenarios include a variety of, for example move inspection, Face datection, designated shape Articles detecting, cross line detection, make a dash across the red light
Deng.The embodiment of the present invention provides a kind of example of the initial grade of determining current video frame, sets the type of current event scene
It was line detection, and included that no vehicle passes through, detects license plate number, detects vehicle in configured specified event scenarios class information
More three specified event scenarios of line, according to passing through to no vehicle, detect license plate number, detect that vehicle gets over these three specified events of line
The corresponding setting grade of the degree of concern of scene will detect that license plate number is set as 1 grade, and detect vehicle without vehicle by being set as 0 grade
More line is set as 2 grades.It will appear following several situations in the initial grade for determining current video frame:
The specified event scenarios for including in situation one, detection current video frame are that no vehicle passes through, according to configured finger
The class information for determining event scenarios determines that not having vehicle to specify the grade of event scenarios by this in current video frame is 0 grade, when
There is only no vehicles in preceding video frame by this specified event scenarios, then the specified event scenarios in current video frame etc.
Highest level in grade is 0 grade, therefore the initial grade of current video frame is determined as 0 grade.
The specified event scenarios for including in situation two, detection current video frame are to detect license plate number but do not detect vehicle
More line determines according to the class information of configured specified event scenarios and detects this specified thing of license plate number in current video frame
The grade of part scene is 1 grade, there is only this specified event scenarios of license plate number are detected in current video frame, then current video
Highest level in the grade of specified event scenarios in frame is 1 grade, therefore the initial grade of current video frame is determined as 1
Grade.
The specified event scenarios for including in situation three, detection current video frame are to detect that vehicle gets over line but do not detect vehicle
The trade mark determines according to the class information of configured specified event scenarios and detects that vehicle gets over this specified thing of line in current video frame
The grade of part scene is 2 grades, there is only detecting that vehicle gets over this specified event scenarios of line in current video frame, then current video
Highest level in the grade of specified event scenarios in frame is 2 grades, therefore the initial grade of current video frame is determined as 2
Grade.
The specified event scenarios for including in situation four, detection current video frame are to detect license plate number and detect that vehicle is got over
Line determines according to the class information of configured specified event scenarios and detects this specified event of license plate number in current video frame
The grade of scene is 1 grade, and the grade for detecting that vehicle gets over line this specified event scenarios is 2 grades, detecting in current video frame
Vehicle gets over line, and this specifies the grade of event scenarios than the higher grade for detecting this specified event scenarios of license plate number, then works as forward sight
It is 2 grades that the highest level in the grade of event scenarios is specified in frequency frame, therefore the initial grade of current video frame is determined as 2
Grade, can guarantee that the grade of current video frame is consistent with attention rate of the user to event in this way；It can also will be in current video frame
The grade for each specified event scenarios for including carries out weight addition, obtains the initial grade of current video frame, such embodiment
May have in current video frame that there are multiple inferior grade events, but the grade of finally determining current video frame compares
Height causes the grade of current video frame and user may be inconsistent to the attention rate of event.
By the above method, the grade of event scenarios is specified to determine the initial grade of video frame according to in video frame, by
The grade that event scenarios are specified in video frame is that basis presets the degree of concern of specified event scenarios, if in video frame
The specified event scenarios valid data that include it is more, it is to the higher grade for specifying event scenarios to set, if thus working as forward sight
The valid data that frequency frame includes are more, and the initial grade of current video frame will be higher, in network bandwidth deficiency, current video
The probability that frame abandons is also just smaller, to improve the probability of valid data safe transmission in current video frame, reduces because having
Imitate the loss of loss of data bring.
Optionally, it according to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, determines
The highest level of discardable video frame.In specific implementation, the highest level that can set discardable video frame subtracts equal to congestion level
1, following example.
The embodiment of the present invention provides a kind of example of highest level for determining discardable video frame, sets video frame buffer team
Column gross space size is max_buffer_length, and the video requency frame data total size that current video frame buffering queue has buffered is
Busy_buffer_length, the video frame size that current video frame buffering queue has buffered account for the ratio of total buffer space size
For busy_buffer_ratio, current congestion level congest_level, the view that can be abandoned in video frame buffer queue
The highest level of frequency frame be max_drop_level, busy_buffer_ratio be equal to busy_buffer_length divided by
Max_buffer_length, value is bigger, illustrates that network congestion level is higher.3 specified algebra of events are arranged in the embodiment of the present invention
The case where scape grade, is illustrated, correspondingly, congestion level is divided into 4 grades, when busy_buffer_ratio is less than or equal to
When 50%, congest_level is set as 0；When busy_buffer_ratio is greater than 50% and is less than or equal to 80%,
Congest_level is set as 1；When busy_buffer_ratio is greater than 80% and is less than or equal to 95%, congest_level
It is set as 2；When busy_buffer_ratio is greater than 95% and is less than or equal to 100%, congest_level is set as 3.When
When congest_level is greater than 0, max_drop_level is equal to congest_level and subtracts 1.Table 1 illustrates one kind
Determine the example of the highest level of discardable video frame.
A kind of example for the highest level for determining discardable video frame of table 1
| congest_level | 0 | 1 | 2 | 3 |
| max_drop_level | In vain | 0 | 1 | 2 |
| busy_buffer_ratio | [0%, 50%] | (50%, 80%] | (80%, 95%] | (90%, 100%] |
By the above method, due to the video frame grade and network congestion level positive correlation that can be abandoned, and regard
Frequency frame grade is to be determined in video frame according to the grade for the specified event scenarios for including, therefore according to the grade of specified event scenarios
It is pre-configured with corresponding congestion level, when congestion occurs, then according to preconfigured congestion level and discardable video frame
The corresponding relationship of highest level determines the highest level of discardable video frame, thus when carrying out frame losing, according to congestion level energy
Effectively the corresponding video frame low comprising specified event scenarios grade is abandoned.
Optionally, according to the type of current video frame judge current video frame whether be can independent decoded video frame, and
Determine the initial grade of current video frame；
The video frame that video frame buffer queue middle grade is less than or equal to the highest level of discardable video frame is abandoned it
Afterwards, further includes:
Current video frame is put into video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
In specific implementation process, it can judge to gather around using I frame as the trigger condition of frame losing, if current video frame is I frame
Grade is filled in, frame losing is carried out according to congestion level.The type of setting current event scene was line detection, was detected in current video frame
The specified event scenarios for including are to detect license plate number and detect that vehicle gets over line, then the initial grade of current video frame is determined as 2
Grade, congestion level are introduced for the content according to shown in table 1, if current congestion level is 2 grades, can drop video frame
Highest level be 1 grade, then by video frame buffer queue middle grade less than or equal to 1 grade video frame abandon, by current video
Frame is put into video frame buffer queue, and current video frame is labeled as 2 grades.
In above-described embodiment, on the one hand, I frame is the video frame that can be independently decoded, and does not need the view of auxiliary decoder
Frequency frame, it is possible to it directly marks and is put into video buffer queue, on the other hand, each GOP includes an I frame, thus with
The condition that I frame is triggered as frame losing, can within each GOP period according to network congestion level detect whether to need frame losing and
Execute related frame losing work.
Optionally, if current video frame be not can independent decoded video frame, will in video frame buffer queue with it is current
Video frame belongs to same picture group and grade is revised as working as forward sight lower than the grade of the video frame of the initial grade of current video frame
The initial grade of frequency frame；
Current video frame is added in video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
In specific implementation process, if current video frame is P frame, it will belong to together in video frame buffer queue with current video frame
One picture group and grade are revised as the initial etc. of current video frame lower than the grade of the video frame of the initial grade of current video frame
Grade, when carrying out video frame indicia, stopping when meeting following either condition:
Condition one, stopping when being tagged to queue head.Fig. 1 a illustrates provided in an embodiment of the present invention be tagged to
The schematic diagram stopped when queue head, as shown in Figure 1a,
The initial grade of current P frame is set as 2 grades, is indicated with P-2, before current P frame insertion queue, there are 6 etc. in queue
The P frame that grade is 1 grade, is indicated with P-1, is successively 1 grade by grade since queue tail before current P-2 frame insertion queue
Video frame is revised as 2 grades, stop flag when arriving queue head, and P-2 frame is inserted into queue, the queue distribution after being inserted into, i.e.,
Queue is made of 7 P-2 frames after insertion.
Stopping when condition two, label encounter the I frame of same GOP.Fig. 1 b illustrates mark provided in an embodiment of the present invention
The schematic diagram that note stops when encountering the I frame of same GOP, as shown in Figure 1 b,
The initial grade of current P frame is set as 2 grades, is indicated with P-2, before current P frame insertion queue, there are 5 etc. in queue
The P frame that grade is 1 grade, is indicated with P-1, and the I frame that 1 grade is 1 grade is indicated with I-1, since queue head, put in order
It is successively that P-1 frame, P-1 frame, I-1 frame, P-1 frame, P-1 frame, P-1 frame are opened before current P-2 frame insertion queue from queue tail
Begin, the video frame that grade is 1 grade is successively revised as 2 grades, P-2 frame is inserted into queue, obtains by the stopping when label encounters I frame
Queue distribution after to insertion.Since queue head, putting in order is successively P-1 frame, P-1 frame, I-2 frame, P-2 frame, P-2
Frame, P-2 frame, P-2 frame.
Condition three, label stop when encountering frame of the video frame buffer queue middle grade higher than the initial grade of current video frame
Only.Fig. 1 c illustrates label provided in an embodiment of the present invention and encounters video frame buffer queue middle grade higher than current video
The schematic diagram stopped when frame, as illustrated in figure 1 c,
The initial grade of current P frame is set as 1 grade, is indicated with P-1, before current P frame insertion queue, there are 2 etc. in queue
The P frame that grade is 2 grades, is indicated with P-2, and the P frame that 4 grades are 0 grade is indicated with P-0, since queue head, put in order
It is successively that P-2 frame, P-2 frame, P-0 frame, P-0 frame, P-0 frame, P-0 frame are opened before current P-1 frame insertion queue from queue tail
Begin, the video frame that grade is 0 grade is successively revised as 1 grade, when the frame for the initial grade for marking grade to be higher than current video frame
Stop, that is, stopping when encountering P-2 frame, P-1 frame is inserted into queue, the queue distribution after being inserted into.Since head, row
Column sequence is successively P-2 frame, P-2 frame, P-1 frame, P-1 frame, P-1 frame, P-1 frame, P-1 frame.
According to the inherent characteristic of compressed encoding, P frame must have adjacent forward direction P frame or I frame to be referred to decoding, institute
Reference P frame or I frame before a high-grade P frame cannot be abandoned when network congestion needs frame losing, even if these
Video frame do not include user concern event, therefore need to belong in video frame buffer queue with current video frame same GOP and
Grade is revised as the initial grade of current video frame lower than the grade of the video frame of the initial grade of current video frame, avoids P frame
It is dropped because grade is low, so that the P frame comprising valid data be made smoothly to decode and effectively be passed in network bandwidth deficiency
It is defeated.
Above method process is introduced in order to clearer, the embodiment of the present invention provides following example.Fig. 2 is illustrated
The method flow of another kind video frame processing provided in an embodiment of the present invention.
Setting video frame buffering queue gross space size is max_buffer_length, and current video frame buffering queue has delayed
The video requency frame data total size of punching is busy_buffer_length, the video frame size that current video frame buffering queue has buffered
The ratio for accounting for total buffer space size is busy_buffer_ratio, current congestion level congest_level, current volume
Highest all event middle grades that code frame includes are max_event_level, the view that can be abandoned in video frame buffer queue
The highest level of frequency frame is max_drop_level, and current real-time coding video frame is cur_frame, from video frame buffer queue
The video frame of middle acquisition is peek_frame.
As shown in Fig. 2, this method comprises:
Step S201, encoder generate a video frame, are denoted as cur_frame.
Step S202, event detection unit detect cur_frame.
Step S203 judges whether cur_frame includes specified event scenarios, than such as whether detecting license plate number, if
Detect that vehicle gets over line, if it is present step S204 is executed, it is no to then follow the steps S206.
Step S204 determines each specified thing in cur_frame according to the class information of configured specified event scenarios
The grade of part scene, for example detect that license plate number is 1 grade, detecting that vehicle gets over line is 2 grades.
Highest level in the grade of each specified event scenarios is assigned to max_event_level by step S205, such as
It is detected simultaneously by license plate number and vehicle gets over line, take 2 grades of greatest level to be assigned to max_event_level, and execute step S207.
Max_event_level is assigned a value of 0, and executes step S207 by step S206.
The grade of cur_frame is set max_event_level by step S207.
Step S208 judges whether otherwise I frame executes current video frame cur_frame if so, executing step S209
Step S213.
Step S209 obtains current network congestion grade congest_level,
Step S210, judges whether congestion congest_level is greater than 0, if it is step S211 is executed, otherwise executes step
Rapid S219.
Step S211 calculates the highest level max_drop_ for the video frame that can be abandoned in video frame buffer queue
Level, value are equal to congest_level and subtract 1.
Step S212 abandons the video frame that frame grade in video frame buffer queue is less than or equal to max_drop_level,
Execute step S219.
Step S213 obtains rear of queue video frame from video frame buffer queue and is assigned to peek_frame.
Step S214 judges whether peek_frame is sky video frame, if so, executing step S219, otherwise executes step
Rapid S215.
Step S215, judges whether video frame peek_frame grade is less than the grade of current video frame cur_frame, such as
Fruit is to execute step S216, no to then follow the steps S219.
Video frame peek_frame grade is set max_event_level by step S216.
Step S217 judges whether video frame peek_frame is I frame, if so, executing step S219, otherwise executes step
Rapid S218.
Step S218 obtains the previous video frame of peek_frame from video frame buffer queue and is assigned to peek_
Frame executes step S214.
Current video frame cur_frame is put into the tail portion of video frame buffer queue by step S219.
The embodiment of the present invention provides a kind of video frame treating method and apparatus, is determining that current video frame is independently to decode
Video frame when, determine the congestion level of video frame buffer queue.According to the congestion level of video frame buffer queue, determination can lose
Abandon the highest level of video frame.Video frame buffer queue middle grade is less than or equal to the view of the highest level of discardable video frame
Frequency frame abandons, and the grade of each video frame in video frame buffer queue is according at least to specified event scenarios included in video frame
Grade determine.In order to ensure still can be to remaining after carrying out part frame losing to picture group in the embodiment of the present invention
Video frame is decoded, thus will judge present frame for can independent decoded video frame when as the condition of starting frame losing, therefore
Determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；Due in difference
Network environment in, the congestion level of network bandwidth be it is different, thus, under different congestion levels, discardable view
Frequency frame is also different, therefore the highest level of discardable video frame need to be determined according to the congestion level of video frame buffer queue, will regard
The video frame that frequency frame buffering queue middle grade is less than or equal to the highest level of discardable video frame abandons；Due to video frame buffer
The grade of each video frame is at least according to specifying the grade of event scenarios to determine in video frame, thus carrying out frame losing in queue
When, the video frame preferentially abandoned is the junior video frame comprising specified event scenarios, to improve network bandwidth deficiency
When valid data by the probability of safe transmission, reduce because loss of effective data bring is lost.
Based on same idea, Fig. 3 illustrates a kind of device of video frame processing provided in an embodiment of the present invention, such as
Shown in Fig. 3, which includes judging unit 301, determination unit 302, processing unit 303；Wherein:
Judging unit 301, for determine current video frame be can independent decoded video frame when, determine video frame buffer
The congestion level of queue；
Determination unit 302 determines the most high of discardable video frame for the congestion level according to video frame buffer queue
Grade；
Processing unit 303, for video frame buffer queue middle grade to be less than or equal to the most high of discardable video frame
The video frame of grade abandons, and the grade of each video frame in video frame buffer queue is according at least to specified thing included in video frame
The grade of part scene determines.
Optionally, judging unit 301 is specifically used for:
According to the type of current video frame judge current video frame whether be can independent decoded video frame, and determine current
The initial grade of video frame；
Optionally, processing unit 303 is also used to:
It is abandoned in the video frame for the highest level that video frame buffer queue middle grade is less than or equal to discardable video frame
Later, current video frame is put into video frame buffer queue, and marking the grade of current video frame is the first of current video frame
Beginning grade.
Optionally, processing unit 303 is also used to:
If current video frame be not can independent decoded video frame, by video frame buffer queue with current video frame category
In same picture group and grade lower than the grade of the video frame of the initial grade of current video frame be revised as current video frame just
Beginning grade；
Current video frame is added in video frame buffer queue, and marking the grade of current video frame is current video frame
Initial grade.
Optionally it is determined that unit 302 is specifically used for:
The grade that specified event scenarios are preset according to the degree of concern to specified event scenarios, to specified event scenarios
Degree of concern more it is high then specify event scenarios higher grade；
According to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, discardable view is determined
The highest level of frequency frame.
Optionally, judging unit 301 is specifically used for:
It whether detects in current video frame comprising specified event scenarios；
If so, determining each specified event in current video frame according to the class information of configured specified event scenarios
The grade of scene；
By the highest level in the grade of each specified event scenarios in current video frame, it is determined as the first of current video frame
Beginning grade.
The embodiment of the present invention provides a kind of video frame treating method and apparatus, is determining that current video frame is independently to decode
Video frame when, determine the congestion level of video frame buffer queue.According to the congestion level of video frame buffer queue, determination can lose
Abandon the highest level of video frame.Video frame buffer queue middle grade is less than or equal to the view of the highest level of discardable video frame
Frequency frame abandons, and the grade of each video frame in video frame buffer queue is according at least to specified event scenarios included in video frame
Grade determine.In order to ensure still can be to remaining after carrying out part frame losing to picture group in the embodiment of the present invention
Video frame is decoded, thus will judge present frame for can independent decoded video frame when as the condition of starting frame losing, therefore
Determine current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；Due in difference
Network environment in, the congestion level of network bandwidth be it is different, thus, under different congestion levels, discardable view
Frequency frame is also different, therefore the highest level of discardable video frame need to be determined according to the congestion level of video frame buffer queue, will regard
The video frame that frequency frame buffering queue middle grade is less than or equal to the highest level of discardable video frame abandons；Due to video frame buffer
The grade of each video frame is at least according to specifying the grade of event scenarios to determine in video frame, thus carrying out frame losing in queue
When, the video frame preferentially abandoned is the junior video frame comprising specified event scenarios, to improve network bandwidth deficiency
When valid data by the probability of safe transmission, reduce because loss of effective data bring is lost.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the present invention
Form.It is deposited moreover, the present invention can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions each in flowchart and/or the block diagram
The combination of process and/or box in process and/or box and flowchart and/or the block diagram.It can provide these computers
Processor of the program instruction to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices
To generate a machine, so that generating use by the instruction that computer or the processor of other programmable data processing devices execute
In the dress for realizing the function of specifying in one or more flows of the flowchart and/or one or more blocks of the block diagram
It sets.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (8)
1. a kind of video frame processing method characterized by comprising
According to the type of current video frame judge the current video frame whether be can independent decoded video frame, and described in determining
The initial grade of current video frame；
Determine the current video frame be can independent decoded video frame when, determine the congestion level of video frame buffer queue；
According to the congestion level of the video frame buffer queue, the highest level of discardable video frame is determined；
The video frame that the video frame buffer queue middle grade is less than or equal to the highest level of the discardable video frame is lost
It abandons, the current video frame is put into the video frame buffer queue, and be institute by the grade mark of the current video frame
State the initial grade of current video frame；The grade of each video frame in the video frame buffer queue is according at least to institute in video frame
The grades of the specified event scenarios for including determines.
2. the method as described in claim 1, which is characterized in that further include:
If the current video frame be not can independent decoded video frame, by the video frame buffer queue with it is described current
Video frame belongs to same picture group and grade and is revised as institute lower than the grade of the video frame of the initial grade of the current video frame
State the initial grade of current video frame；
The current video frame is added in the video frame buffer queue, and is institute by the grade mark of the current video frame
State the initial grade of current video frame.
3. such as the described in any item methods of claim 1 to 2, which is characterized in that the grade of the specified event scenarios is basis
It is preset to the degree of concern of specified event scenarios, it is more high to the degree of concern of specified event scenarios then to specify event scenarios
Higher grade；
Congestion level in the determining video frame buffer queue is determined according to the congestion level of the video frame buffer queue
The highest level of discardable video frame, comprising:
According to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, the discardable view is determined
The highest level of frequency frame.
4. method as claimed in claim 3, which is characterized in that determine the initial etc. of the current video frame in the following manner
Grade, comprising:
It whether detects in the current video frame comprising specified event scenarios；
If so, determining each specified event in the current video frame according to the class information of configured specified event scenarios
The grade of scene；
By the highest level in the grade of each specified event scenarios in the current video frame, it is determined as the current video frame
Initial grade.
5. a kind of video frame processing apparatus characterized by comprising
Judging unit, for the type according to current video frame judge the current video frame whether be can independent decoded video
Frame, and determine the initial grade of the current video frame；Determine the current video frame be can independent decoded video frame when,
Determine the congestion level of video frame buffer queue；
Determination unit determines the highest level of discardable video frame for the congestion level according to the video frame buffer queue；
Processing unit, for the video frame buffer queue middle grade to be less than or equal to the most high of the discardable video frame
The video frame of grade abandons, and the current video frame is put into the video frame buffer queue, and by the current video frame
Grade mark is the initial grade of the current video frame；The grade at least root of each video frame in the video frame buffer queue
It is determined according to the grade of event scenarios is specified included in video frame.
6. device as claimed in claim 5, which is characterized in that the processing unit is also used to:
If the current video frame be not can independent decoded video frame, by the video frame buffer queue with it is described current
Video frame belongs to same picture group and grade and is revised as institute lower than the grade of the video frame of the initial grade of the current video frame
State the initial grade of current video frame；
The current video frame is added in the video frame buffer queue, and is institute by the grade mark of the current video frame
State the initial grade of current video frame.
7. such as the described in any item devices of claim 5 to 6, which is characterized in that the determination unit is specifically used for:
The grade that the specified event scenarios are preset according to the degree of concern to specified event scenarios, to specified event scenarios
Degree of concern more it is high then specify event scenarios higher grade；
According to the corresponding relationship of preconfigured congestion level and the highest level of discardable video frame, the discardable view is determined
The highest level of frequency frame.
8. device as claimed in claim 7, which is characterized in that the judging unit is specifically used for:
It whether detects in the current video frame comprising specified event scenarios；
If so, determining each specified event in the current video frame according to the class information of configured specified event scenarios
The grade of scene；
By the highest level in the grade of each specified event scenarios in the current video frame, it is determined as the current video frame
Initial grade.
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610906960.3A CN106454432B (en) | 2016-10-18 | 2016-10-18 | A kind of video frame treating method and apparatus |
| PCT/CN2017/106484 WO2018072675A1 (en) | 2016-10-18 | 2017-10-17 | Methods and systems for video processing |
| EP17862412.8A EP3513563A4 (en) | 2016-10-18 | 2017-10-17 | Methods and systems for video processing |
| US16/387,506 US10977498B2 (en) | 2016-10-18 | 2019-04-17 | Methods and systems for video processing |
| US17/198,177 US11527068B2 (en) | 2016-10-18 | 2021-03-10 | Methods and systems for video processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610906960.3A CN106454432B (en) | 2016-10-18 | 2016-10-18 | A kind of video frame treating method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106454432A CN106454432A (en) | 2017-02-22 |
| CN106454432B true CN106454432B (en) | 2019-09-17 |
Family
ID=58175706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610906960.3A Active CN106454432B (en) | 2016-10-18 | 2016-10-18 | A kind of video frame treating method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106454432B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3513563A4 (en) | 2016-10-18 | 2019-07-24 | Zhejiang Dahua Technology Co., Ltd | Methods and systems for video processing |
| CN108347645B (en) * | 2018-01-19 | 2021-02-26 | 浙江大华技术股份有限公司 | Method and device for decoding and displaying video frame |
| CN112822505B (en) * | 2020-12-31 | 2023-03-03 | 杭州星犀科技有限公司 | Audio and video frame loss method, device, system, storage medium and computer equipment |
| CN113068074B (en) * | 2021-03-24 | 2023-03-24 | 浙江大华技术股份有限公司 | Caching method and device, computer-readable storage medium and electronic device |
| CN113542798A (en) * | 2021-05-31 | 2021-10-22 | 浙江大华技术股份有限公司 | Video stream transmission method, electronic device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1468001A (en) * | 2002-06-27 | 2004-01-14 | 上海汉唐科技有限公司 | Media flow self-adapting transmission method based on internet |
| CN101656888A (en) * | 2008-08-21 | 2010-02-24 | 华为技术有限公司 | Method, device and system for video transmission |
| CN102378067A (en) * | 2011-11-21 | 2012-03-14 | 武汉大学 | Robustness mobile video decoding method |
| CN103118241A (en) * | 2012-02-24 | 2013-05-22 | 金三立视频科技（深圳）有限公司 | Mobile video monitoring streaming media transmission self-adaptive adjustment algorithm based on the 3rd generation telecommunication (3G) network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6728213B1 (en) * | 2001-03-23 | 2004-04-27 | Advanced Micro Devices, Inc. | Selective admission control in a network device |
| CN105898358B (en) * | 2016-04-08 | 2019-03-12 | 浙江宇视科技有限公司 | The sending method and device of video data |
-
2016
- 2016-10-18 CN CN201610906960.3A patent/CN106454432B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1468001A (en) * | 2002-06-27 | 2004-01-14 | 上海汉唐科技有限公司 | Media flow self-adapting transmission method based on internet |
| CN101656888A (en) * | 2008-08-21 | 2010-02-24 | 华为技术有限公司 | Method, device and system for video transmission |
| CN102378067A (en) * | 2011-11-21 | 2012-03-14 | 武汉大学 | Robustness mobile video decoding method |
| CN103118241A (en) * | 2012-02-24 | 2013-05-22 | 金三立视频科技（深圳）有限公司 | Mobile video monitoring streaming media transmission self-adaptive adjustment algorithm based on the 3rd generation telecommunication (3G) network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106454432A (en) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106454432B (en) | A kind of video frame treating method and apparatus | |
| CN105830445B (en) | The transmission method and equipment of multiple video strems | |
| US10298970B2 (en) | Image transmission method and apparatus | |
| CN103475882B (en) | Surveillance video encoding and recognizing method and surveillance video encoding and recognizing system | |
| CN106534949A (en) | Method for prolonging video storage time of video monitoring system | |
| WO2016165603A1 (en) | Encoding and decoding method and device for video data | |
| US20130301716A1 (en) | Method and Device for Coding and Decoding Images | |
| CN110868600A (en) | Target tracking video plug-flow method, display method, device and storage medium | |
| CN105554591A (en) | Video analysis method and device | |
| CN110191511A (en) | The method and apparatus of semi-persistent scheduling SPS | |
| CN112351447A (en) | Public place security big data collection method and system based on block chain system | |
| WO2014121165A3 (en) | Devices for reduced overhead paging | |
| CN105898296A (en) | Video coding frame selection method and device | |
| CN106919610A (en) | Car networking data processing method, system and server | |
| EP3310093A1 (en) | Traffic control method and apparatus | |
| CN105681931A (en) | Video data packet transmission method and system based on wireless network | |
| CN104333732B (en) | A kind of distributed video analysis method and system | |
| CN107124610A (en) | A kind of method for video coding and device | |
| CN109271218A (en) | A kind of method and system that virtual desktop data are shown | |
| CN103187083A (en) | Storage method and system based on time domain video fusion | |
| CN110099239B (en) | Video marking method, video tracing method, video processing device and storage medium | |
| CN105635715A (en) | Video format identification method and device | |
| CN104767964A (en) | Video storage control and optimization method and device, and video monitoring system | |
| JP2015510354A5 (en) | ||
| WO2022078266A1 (en) | Systems and methods for data transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |