WO2023125443A1 - 视频传输参数的自适应调节方法和系统 - Google Patents

视频传输参数的自适应调节方法和系统 Download PDF

Info

Publication number
WO2023125443A1
WO2023125443A1 PCT/CN2022/142041 CN2022142041W WO2023125443A1 WO 2023125443 A1 WO2023125443 A1 WO 2023125443A1 CN 2022142041 W CN2022142041 W CN 2022142041W WO 2023125443 A1 WO2023125443 A1 WO 2023125443A1
Authority
WO
WIPO (PCT)
Prior art keywords
transmission
video transmission
video
level
frame
Prior art date
Application number
PCT/CN2022/142041
Other languages
English (en)
French (fr)
Inventor
缪文
林喆
Original Assignee
上海商米科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商米科技集团股份有限公司 filed Critical 上海商米科技集团股份有限公司
Publication of WO2023125443A1 publication Critical patent/WO2023125443A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/114—Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]

Definitions

  • the present invention mainly relates to the field of video transmission, in particular to an adaptive adjustment method and system for video transmission parameters.
  • Bit rate adaptive adjustment has been widely used in audio and video streaming services.
  • the code rate adjustment can be actively detected by the server, or can be adjusted by the client feedback.
  • the active adjustment method of the streaming media server generally makes decisions by monitoring information such as network bandwidth occupation and sending cache. Since the real-time bit rate of IPC (IP camera, network camera) video data is unstable, the active adjustment method of streaming media bit rate is generally not suitable for IPC.
  • IPC IP camera, network camera
  • the IPC video data used for security monitoring has the following characteristics: (1) Dynamic bit rate encoding is mostly used, and the I frame (Intra-coded picture, intra-frame coded image frame) interval is longer; (2) I frame is more than P frame (Predictive -coded Picture, forward predictive coded image frame) has a much larger amount of single frame data, and the real-time bit rate fluctuates greatly. Due to the above-mentioned characteristics of IPC video data, using the traditional bandwidth detection and buffer area detection methods, even if the network bandwidth is adjusted to be higher than the average bit rate, the smooth transmission of I frames cannot be guaranteed, and a GOP (Group of Pictures) may appear. , including one I-frame and multiple P-frames), the P-frame transmission is normal, but the I-frame transmission times out, resulting in delay or freezing of the I-frame picture, affecting user experience.
  • the technical problem to be solved by the present invention is to provide an adaptive adjustment method and system for video transmission parameters, which can effectively reduce the stuttering phenomenon in the process of IPC video data transmission.
  • the present invention provides an adaptive adjustment method of video transmission parameters, including the following steps: configuring multiple video transmission levels, and the multiple video transmission levels are composed of High to low rank order, and each video transmission level includes at least one video parameter setting value; the video is transmitted according to the setting value of the video parameter in the selected video transmission level, and records how many consecutive times in the video The transmission time of a key frame, wherein, the selected video transmission level is any level in the plurality of video transmission levels; and statistics of each transmission time and the size of the first time threshold and the second time threshold Relationship, when the number of consecutive multiple key frames whose transmission time is greater than the first time threshold reaches the first number threshold, and the selected video transmission level is not the lowest level among the multiple video transmission levels, reducing the selected video transmission level, and when the number of consecutive multiple key frames whose transmission time is less than the second time threshold reaches a second number threshold, and the selected video transmission level is not the multiple When the highest of the two video transmission levels is selected, the selected video transmission level
  • the video parameters include resolution and maximum bit rate.
  • the difference in data size of key frames corresponding to adjacent transmission levels satisfies the data difference threshold.
  • the method for adaptively adjusting video transmission parameters further includes judging whether the current frame being transmitted is the key frame, and when the current frame is the key frame, recording the The transmission time of the current frame, and when the current frame is a non-key frame, continue to judge whether the key frame of the same group as the current frame has not completed transmission or is not in the transmission queue, and if the judgment result is yes, then discard The current frame.
  • the method for adaptively adjusting video transmission parameters further includes clearing the transmission queue and discarding the current frame being transmitted.
  • the selected video transmission level is the lowest level among the multiple video transmission levels, when the transmission time is greater than the first time threshold When the number of frames reaches the first number threshold, the selected video transmission level is maintained, and if the selected transmission video level is the highest level among the plurality of video transmission levels, when the transmission time is less than the When the number of consecutive multiple key frames of the second time threshold reaches the second number threshold, the selected video transmission level is maintained.
  • the selected video transmission level is the highest level.
  • the first time threshold is 2 to 4 times the frame interval
  • the second time threshold is 1/4 to 1/2 the frame interval
  • the first number threshold when transmitting the first key frame of the video or the first to nth key frames, is set to 1, Wherein, n is an integer greater than 1.
  • the number of the multiple video transmission levels is 3 or 4.
  • the present invention also provides an adaptive adjustment system for video transmission parameters, including: a transmission level configuration module, which is suitable for configuring multiple video transmission levels, and the multiple video transmission levels have a range from high to high according to the corresponding key frame data size to a low level order, and each video transmission level includes at least one video parameter setting value; the key frame transmission statistics module is configured to transmit when the video is transmitted according to the set value of the video parameter in the selected video transmission level, Recording the transmission time of multiple consecutive key frames in the video, wherein the selected video transmission level is any one of the multiple video transmission levels, and the key frame transmission statistics module is also configured to count each The size relationship between the transmission time and the first time threshold and the second time threshold; and the transmission level decision-making module, configured to reach the first number when the number of consecutive key frames whose transmission time is greater than the first time threshold threshold, and the selected video transmission level is not the lowest level in the plurality of video transmission levels, reduce the selected video transmission level, and when the transmission time is less than the second time threshold When the number of key frames reaches
  • the video parameters include resolution and maximum bit rate.
  • the present invention also provides an adaptive adjustment system for video transmission parameters, including: a memory for storing instructions executable by a processor; and a processor for executing the instructions to realize the above-mentioned adaptive adjustment of video transmission parameters Adjustment method.
  • the present invention also provides a computer-readable medium storing computer program codes, the computer program codes implement the above-mentioned method for adaptively adjusting video transmission parameters when executed by a processor.
  • the present invention has the following advantages:
  • the self-adaptive adjustment method and system of video transmission parameters of the present invention can automatically adjust the resolution and code rate of video encoding by monitoring the transmission of video data frames between the IPC and the receiving end.
  • the bit rate is automatically reduced to reduce video freeze; when the network is smooth, the bit rate is increased to provide a clearer video image.
  • the scheme of the present invention uses the transmission state of the I frame (key frame) as the decision-making condition for whether to adjust the code rate level, which has good pertinence and remarkable effect, especially in network
  • the self-adaptive adjustment scheme of video transmission parameters in the present invention can improve user playback experience.
  • FIG. 1 is a schematic flowchart of an adaptive adjustment method for video transmission parameters according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an image group including key frames and non-key frames in an adaptive adjustment method for video transmission parameters according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for adaptively adjusting video transmission parameters according to another embodiment of the present invention.
  • FIG. 4 is a system block diagram of an adaptive adjustment system for video transmission parameters according to an embodiment of the present invention.
  • Fig. 5 is a system block diagram of an adaptive adjustment system for video transmission parameters according to another embodiment of the present invention.
  • orientation words such as “front, back, up, down, left, right", “horizontal, vertical, vertical, horizontal” and “top, bottom” etc. indicate the orientation Or positional relationship is generally based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the application and simplifying the description. In the absence of a contrary statement, these orientation words do not indicate or imply the device or element referred to It must have a specific orientation or be constructed and operated in a specific orientation, so it should not be construed as limiting the protection scope of the present application; the orientation words “inner and outer” refer to the inner and outer relative to the outline of each component itself.
  • spatially relative terms may be used here, such as “on !, “over !, “on the surface of !, “above”, etc., to describe The spatial positional relationship between one device or feature shown and other devices or features. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, devices described as “above” or “above” other devices or configurations would then be oriented “beneath” or “above” the other devices or configurations. under other devices or configurations”. Thus, the exemplary term “above” can encompass both an orientation of “above” and “beneath”. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptions used herein interpreted accordingly.
  • the flow chart is used in this application to illustrate the operations performed by the system according to the embodiment of this application. It should be understood that the preceding or following operations are not necessarily performed in an exact order. Instead, various steps may be processed in reverse order or concurrently. At the same time, other operations are either added to these procedures, or a certain step or steps are removed from these procedures.
  • FIG. 1 is a schematic diagram of an adaptive adjustment method 10 for video transmission parameters (hereinafter referred to as "adjustment method 10") according to an embodiment of the present application. As shown in FIG. 1 , the present application provides a method 10 for adaptively adjusting video transmission parameters, including the following steps.
  • S1 is to configure multiple video transmission levels.
  • S2 is to transmit the video according to the set value of the video parameter in the selected video transmission level, and record the transmission time of multiple consecutive key frames in the video.
  • S3 is to calculate the relationship between each transmission time and the first time threshold and the second time threshold and adjust the video transmission level.
  • Steps S1-S3 will be further described in detail below.
  • step S1 the plurality of video transmission levels have a rank order from high to low according to the corresponding key frame data size, and each video transmission level includes at least one video parameter setting value.
  • the adjustment method 10 is applicable to the transmission of security monitoring network camera (IP Camera, IPC) video.
  • IP Camera security monitoring network camera
  • the key frame in the adjustment method 10 is an intra-coded picture frame (Intra-coded Picture), that is, an I frame.
  • Intra-coded Picture intra-coded picture frame
  • Predictive-coded Picture forward predictive coded image frame
  • the IPC encoder compresses the collected original images and encodes them into a continuous image group GOP (Group of Pictures).
  • a GOP consists of an I frame (key frame 1) and multiple P frames (non-key frame 1, non-key frame 2...), where the I frame can be decoded independently to obtain a complete
  • the original image of the P frame only records the difference between the current frame and the I frame or the previous P frame in the GOP, so the data volume of the I frame is often much larger than that of the P frame. This has also caused a bottleneck in the prior art that it is difficult to adjust parameters such as bit rate for such video transmission data.
  • the video parameters in step S1 include resolution and maximum bit rate.
  • the resolution is a parameter used to measure the amount of data in the image, usually expressed as ppi (pixel per inch), for example, the resolution of a video is 1920*1080, which means the horizontal and vertical dimensions of the video
  • the effective pixels are 1920ppi and 1080ppi respectively.
  • the resolution affects the clarity of the video.
  • the code rate is the number of data bits transmitted per unit time during data transmission, and the commonly used unit is kbps (thousand bits per second, which can also be expressed in kb/s).
  • the video parameter is the maximum bit rate, not the average bit rate, because as mentioned above, the data volume of key frames is often much larger than that of non-key frames, and for practical application considerations, the key The frame interval is long, resulting in large fluctuations in the real-time bit rate of the video, and the average bit rate is far lower than the maximum bit rate.
  • two parameters, resolution and maximum bit rate are mainly selected as video parameters in step S2.
  • the selected video transmission level may be any one of multiple video transmission levels.
  • the selected video transmission level is the highest level. The selection of the video transmission level and the changes in the actual work process will be described in more detail below.
  • step S3 of FIG. 1 the specific way of adjusting the video transmission level is as follows. When the number of consecutive multiple key frames whose transmission time is greater than the first time threshold reaches the first number threshold, and the selected video transmission level is not the lowest level in the multiple video transmission levels, reduce the selected video transmission level, and When the number of consecutive multiple key frames whose transmission time is less than the second time threshold reaches the second number threshold, and the selected video transmission level is not the highest among the multiple video transmission levels, increase the selected video transmission level.
  • the first time threshold in step S3 is 2-4 times the frame interval
  • the second time threshold is 1/4-1/2 the frame interval.
  • Inter-Frame Space (IFS) indicates the time interval between frame transmissions. For example, if the number of frames transmitted per second is set to 10 frames, the frame interval is 0.1 second. In the normal transmission state, when the transmission time is less than the frame interval, the transmission effect is considered to be better. When the transmission time is greater than the frame interval, there may be a key frame transmission timeout situation, and when multiple consecutive key frames exceeding the first threshold When the number reaches the first number threshold, this situation becomes more obvious, and the key frame picture delays or freezes seriously.
  • the initially selected video transmission level can be reduced, that is, the Video parameters such as the maximum bit rate, etc. to improve the smoothness of video transmission. It can be understood that, when the currently selected video transmission level is already the lowest level among multiple video transmission levels, no downgrade operation is performed.
  • the video transmission level can be further improved to obtain higher video transmission quality, that is, when the transmission data of the key frame is less than the second threshold Decreases the selected video transmission level when the number of keyframes reaches a second number threshold. It is also understandable that when the currently selected video transmission level is already the highest level among multiple video transmission levels, the level-up operation is not performed.
  • FIG. 3 is an adaptive adjustment method 30 of video transmission parameters during single frame transmission in another embodiment of the present application (hereinafter referred to as "adjustment method 30") schematic diagram of the process. It can be understood that the adjustment method 30 shown in FIG. 3 is based on the adjustment method 10 shown in FIG. 1 , and realizes a relatively complete adjustment method in a specific embodiment.
  • step S301 when a new data frame is generated and starts to be transmitted, that is step S301, a current frame is generated, and the current frame undergoes the following steps.
  • Step S302 is to judge whether the current frame being transmitted is a key frame (such as the I frame mentioned above), if the current frame is a key frame, enter step S303, which is the process step of the key frame part. And if the current frame is a non-key frame (for example, the P frame mentioned above), enter step S308 and the subsequent processes, which are the process steps of the non-key frame part, and this part will be introduced in more detail below. The following will firstly describe the further adjustment process if the current frame is a key frame. ;
  • step S303 record the transmission time of the current frame (key frame), and combine its transmission time with the transmission time statistics of a plurality of consecutive key frames before the current frame (key frame)
  • step S303 record the transmission time of the current frame (key frame)
  • step S303 combine its transmission time with the transmission time statistics of a plurality of consecutive key frames before the current frame (key frame)
  • step S304 if the number of consecutive times that the current frame transmission time is higher than the first time threshold T1 reaches the first number threshold N1, perform step S309 to lower the video transmission level, and further perform step S310 to clear the current frame
  • step S311 is executed to discard the current frame being transmitted, so as to ensure that after the new video frame is generated again, it will be transmitted again at the reduced video transmission level, so that the cost can be minimized Improve the fluency of video data transmission.
  • the first number threshold N1 when transmitting the first key frame or the first to nth key frames of the video, is set to 1, where n is an integer greater than 1, and an exemplary n may be 2 or 3. This means that if the transmission time of the first or the first few key frames is greater than the first time threshold T1, then reduce the currently selected video transmission level until the video transmission is stable and then increase the video transmission level through the process in Figure 3.
  • the advantage of setting the first number threshold corresponding to the first (or first few) key frames of the transmitted video to 1 is that when the device is just connected to the network, the transmission level can be quickly adjusted according to the network status to achieve smooth video transmission and play.
  • step S304 determines whether the transmission time of the current frame is lower than the second time threshold T2, and comprehensively consider the continuous multiple key frames before the current frame.
  • Transmission time determine whether the consecutive times of transmission time lower than T2 have reached N2 after the transmission of the current frame, if the judgment result of step S305 is yes, then execute step S307 to increase the video transmission level. While improving the video transmission level, step S306 is performed to transmit the current frame (key frame) according to the current video transmission level or to add the current frame (key frame) to the transmission queue.
  • Such settings can optimize video transmission Based on the effect, reduce the cost of missing video frames as much as possible.
  • step S305 determines whether the video transmission parameters can be adjusted in time to improve the overall effect of video transmission.
  • step S304 and step S305 are arranged in order, but the present invention is not limited thereto.
  • it may be Switch the order between step S304 and step S305, or comprehensively judge the current frame transmission time and the first time threshold T1, the second time threshold T2, the first number threshold N1, and the second number in the same step.
  • the relationship between the threshold N2 is not limited by the present invention.
  • step S302 if the judgment result of step S302 is no, it means that the current frame is a non-key frame (such as a P frame), and now step S308 is performed to judge whether the key frame of the same group as the current frame (non-key frame) has been transmitted Or in the transmission queue, if the judgment result is yes, continue to wait for the transmission situation of the key frame of the same group, and then make a decision on whether to change the video transmission level.
  • a non-key frame such as a P frame
  • step S308 it means that the key frame of the same group as the current frame (non-key frame) has not been successfully transmitted and is not in the transmission queue, wherein the reason for not being in the transmission queue may be because of the above-described
  • step S310 the same group of key frames is cleared from the transmission queue by the operation of clearing the transmission queue. At this time, it is considered that the data frame of the same group before the non-key frame is already in a state of stagnation/unsuccessful transmission.
  • step S311 to directly discard the current non-key frame, so that the load of the video data transmission queue can be reduced in the shortest possible time, so that it can restore the smooth transmission of video data in the shortest possible time.
  • the number of video transmission levels is set to 3, and two video transmission parameters of different resolution and maximum bit rate are configured in each video transmission level.
  • video transmission level 1 is the highest level, and video transmission levels 2 and 3 are successively lower than video transmission level 1.
  • the data size (unit byte) of the key frame in a certain test scene corresponding to video transmission level 1 is 57835
  • the data size (unit byte) of the key frame in a certain test scene corresponding to video transmission level 2 and 3 The data size (unit byte) is 15534 and 5166 respectively.
  • the key frame data size corresponding to the adjacent level is about 2-3 times the effect, so that it can better distinguish the need to switch the video transmission level when adjusting The key frame data volume segment, which is more in line with the requirements of video transmission effect.
  • the difference in data size of key frames corresponding to adjacent transmission levels in the level order satisfies the data difference threshold (for example, 2 times or 3 times).
  • First execute step S304 to determine whether the number of consecutive multiple key frames whose transmission time of key frame 1 is greater than the first time threshold T1 has reached 3, if so, execute step S309 to lower the video transmission level, and execute step S310 to clear the sending queue And step S311 discards the current frame, and then enters the transmission of the next frame (i.e.
  • step S305 is further executed to judge Whether the number of consecutive multiple key frames whose transmission time is less than the second time threshold T2 reaches 3, if it has been reached, then perform step S307 to improve the video transmission level, and add the current frame (key frame 1) to the sending queue; if not If it is reached, the current frame (key frame 1) will be directly added to the sending queue without adjusting the currently selected video transmission level.
  • the above description exemplarily shows the transmission process of the key frame 1. If the current frame is a non-key frame 1, the specific transmission process is as follows.
  • step S304 is to judge the transmission time of key frame 1 for more than the first time threshold T1. Whether the number of key frames reaches the first number threshold N1, the subsequent process is the same as the key frame sending process described above, and will not be repeated here.
  • this scheme considers that the data volume of key frame 1 is too large, so it allows key frame 1 to have a certain delay in transmission (but generally no more than a few frame intervals), at this time, key frame 1 is in the same group
  • the other non-key frame 1, non-key frame 2... will depend on the actual transmission of the key frame 1 during actual transmission to make a decision to change the currently selected video transmission level.
  • step S304 if the key frame 1 encounters the situation that the current frame transmission time is higher than T1 and the number of consecutive times reaches N1 when executing step S304, then the selected video transmission level needs to be downgraded at this time, and the transmission queue needs to be cleared and the current frame needs to be discarded. frame.
  • the key frame 1 in the current GOP encounters the above-mentioned critical situation that needs to be degraded, the non-key frames in the same group behind the key frame 1 will be directly discarded through step S311, so as to quickly restore the video transmission fluency until keyframes in the next GOP start transmission again.
  • the premise of raising the video transmission level mentioned above is that the current video transmission level is not the highest level.
  • the premise of lowering the video transmission level mentioned above is that the current video transmission level is not the lowest level.
  • the adjustment of the video transmission level needs to be carried out step by step.
  • the self-adaptive adjustment method of a video transmission parameter as described above in this application makes statistics on the historical transmission time information of key frames, so that the transmission time of key frames becomes an important decision-making factor for decision-making of video transmission level transformation, for Solving the problems existing in the prior art has good pertinence and remarkable effect.
  • the adjustment system 40 includes a transmission level configuration module 41 , a key frame transmission statistics module 42 and a transmission level decision module 43 .
  • the transmission level configuration module 41 is suitable for configuring a plurality of video transmission levels, the multiple video transmission levels have a rank order from high to low according to the corresponding key frame data size, and each video transmission level includes at least one video parameter setting value.
  • the key frame transmission statistical module 42 is configured to record the transmission time of a plurality of consecutive key frames in the video when the video is transmitted according to the set value of the video parameter in the selected video transmission level, wherein the selected video transmission level is a plurality of At any level in the video transmission level, the key frame transmission statistics module is further configured to count the relationship between each transmission time and the first time threshold and the second time threshold.
  • Transmission grade decision module 43 is configured to when the number of multiple consecutive key frames whose transmission time is greater than the first time threshold reaches the first number threshold, and the selected video transmission grade is not the lowest among the multiple video transmission grades, reduce the Select the video transmission level, and when the number of consecutive multiple key frames whose transmission time is less than the second time threshold reaches the second number threshold, and the selected video transmission level is not the highest level in the multiple video transmission levels, increase Select the video transmission class.
  • the video parameters are specifically resolution and maximum bit rate.
  • the adjustment system 40 of the present invention can be applied to the adjustment method 10 or 30 described above with reference to FIGS.
  • an embodiment of the present invention also proposes an adaptive adjustment system 50 for video transmission parameters as shown in FIG. 5 .
  • an adaptive adjustment system 50 for video transmission parameters may include an internal communication bus 51 , a processor (Processor) 52 , a read only memory (ROM) 53 , a random access memory (RAM) 54 , and a communication port 55 .
  • the system 50 for adaptive adjustment of video transmission parameters may also include a hard disk 56 .
  • the internal communication bus 51 can realize data communication between components of the adaptive adjustment system 50 for video transmission parameters.
  • the processor 52 can make a judgment and issue a prompt.
  • processor 52 may consist of one or more processors.
  • the communication port 55 can realize data communication between the adaptive adjustment system 50 of the video transmission parameters and the outside.
  • the system 50 for adaptive adjustment of video transmission parameters can send and receive information and data from the network through the communication port 55 .
  • the adaptive adjustment system 50 of video transmission parameter can also comprise different forms of program storage unit and data storage unit, such as hard disk 56, read-only memory (ROM) 53 and random access memory (RAM) 54, can store computer processing and/or or various data files used for communication, and possible program instructions executed by the processor 52 .
  • the processor executes these instructions to implement the main parts of the method.
  • the result processed by the processor is transmitted to the user equipment through the communication port, and displayed on the user interface.
  • another aspect of the present invention also proposes a computer-readable medium storing computer program codes, and the computer program codes implement the above-mentioned method for adaptively adjusting video transmission parameters when executed by a processor.
  • the processor can be one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DAPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors , a controller, a microcontroller, a microprocessor, or a combination thereof.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DAPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gate Arrays
  • aspects of the present application may be embodied as a computer product comprising computer readable program code on one or more computer readable media.
  • computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tape%), optical disks (e.g., compact disk CDs, digital versatile disks DVD%), smart cards, and flash memory devices ( For example, cards, sticks, key drives).
  • a computer readable medium may contain a propagated data signal embodying a computer program code, for example, in baseband or as part of a carrier wave.
  • the propagated signal may take many forms, including electromagnetic, optical, etc., or a suitable combination.
  • the computer-readable medium can be any computer-readable medium, except computer-readable storage media, that can communicate, propagate, or transfer the program for use by being coupled to an instruction execution system, apparatus, or device.
  • Program code on a computer readable medium may be transmitted over any suitable medium, including radio, electrical cables, fiber optic cables, radio frequency signals, or the like, or combinations of any of the foregoing.
  • numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of the embodiments use the modifiers "about”, “approximately” or “substantially” in some examples. grooming. Unless otherwise stated, “about”, “approximately” or “substantially” indicates that the figure allows for a variation of ⁇ 20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximations that can vary depending upon the desired characteristics of individual embodiments. In some embodiments, numerical parameters should take into account the specified significant digits and adopt the general digit reservation method. Although the numerical ranges and parameters used in some embodiments of the present application to confirm the breadth of the scope are approximate values, in specific embodiments, such numerical values are set as precisely as practicable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明提供了一种视频传输参数的自适应调节方法和系统。调节方法包括如下步骤：配置多个视频传输等级；使视频按照选定视频传输等级中的视频参数的设定值进行传输，并记录视频中连续多个关键帧的传输时间；以及统计每个传输时间与第一时间阈值和第二时间阈值的大小关系，并调整当前选定的视频传输等级。本发明的视频传输参数的自适应调节方法和系统，能有效减少IPC视频数据传输过程中的卡顿现象。

Description

视频传输参数的自适应调节方法和系统 技术领域
本发明主要涉及视频传输领域，尤其涉及一种视频传输参数的自适应调节方法和系统。
背景技术
码率自适应调节在影音流媒体服务中已被普遍应用。码率的调节可以由服务器主动检测，也可以由客户端反馈调整。流媒体服务器的主动调整方式一般通过监测网络带宽占用、发送缓存等信息进行决策。由于IPC(IP camera，网络摄像机)的视频数据的实时码率不稳定，流媒体码率的主动调节方式一般不适合IPC。
用于安防监控的IPC视频数据有以下特点：(1)多采用动态码率编码，I帧(Intra-coded picture，帧内编码图像帧)间隔较长；(2)I帧比P帧(Predictive-coded Picture，前向预测编码图像帧)的单帧数据量大很多，实时码率波动大。由于IPC视频数据的上述特点，使用传统的带宽检测和缓存区检测方式，即使经过调整，网络带宽高于平均码率所需，也不能保证I帧的流畅传输，可能出现一个GOP(Group of Pictures，包括一个I帧和多个P帧)周期内，P帧传输正常，但I帧传输超时的情况，造成在I帧画面的延迟或卡顿，影响用户体验。
发明内容
本发明要解决的技术问题是提供一种视频传输参数的自适应调节方法和系统，能有效减少IPC视频数据传输过程中的卡顿现象。
为解决上述技术问题，本发明提供了一种视频传输参数的自适应调节方法，包括如下的步骤：配置多个视频传输等级，所述多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值；使视频按照选定视频传输等级中的视频参数的设定值进行传输，并记录所述视频中连续多个关键帧的传输时间，其中，所述选定视频 传输等级为所述多个视频传输等级中的任一级；以及统计每个所述传输时间与第一时间阈值和第二时间阈值的大小关系，当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最低级时，降低所述选定视频传输等级，而当所述传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最高级时，提高所述选定视频传输等级。
可选地，在本发明的一实施例中，视频参数包括分辨率和最大码率。
可选地，在本发明的一实施例中，在等级次序中，相邻传输等级所对应的关键帧数据大小的差别满足数据差阈值。
可选地，在本发明的一实施例中，视频传输参数的自适应调节方法还包括判断正在传输的当前帧是否为所述关键帧，当所述当前帧为所述关键帧时，记录所述当前帧的传输时间，而当所述当前帧为非关键帧时，继续判断与所述当前帧同组的关键帧是否未完成传输或未在传输队列中，若判断结果为是，则丢弃所述当前帧。
可选地，在本发明的一实施例中，视频传输参数的自适应调节方法在降低所述选定视频等级后，还包括清空传输队列并丢弃正在传输的当前帧。
可选地，在本发明的一实施例中，若所述选定视频传输等级为所述多个视频传输等级中的最低级，则当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值时，维持所述选定视频传输等级，而若所述选定传输视频等级为所述多个视频传输等级中的最高级，则当传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值时，维持所述选定视频传输等级。
可选地，在本发明的一实施例中，在默认状态下，所述选定视频传输等级为所述最高级。
可选地，在本发明的一实施例中，所述第一时间阈值为帧间隔的2～4倍，所述第二时间阈值为帧间隔的1/4～1/2。
可选地，在本发明的一实施例中，在传输所述视频的第一个关键帧或所述第一个至第n个关键帧时，将所述第一个数阈值设置为1，其中，n为大于1 的整数。
可选地，在本发明的一实施例中，所述多个视频传输等级的个数为3或4个。
本发明还提供了一种视频传输参数的自适应调节系统，包括：传输等级配置模块，适于配置多个视频传输等级，所述多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值；关键帧传输统计模块，配置为当视频按照选定视频传输等级中的视频参数的设定值进行传输时，记录所述视频中连续多个关键帧的传输时间，其中，所述选定视频传输等级为所述多个视频传输等级中的任一级，所述关键帧传输统计模块还配置为统计每个所述传输时间与第一时间阈值和第二时间阈值的大小关系；以及传输等级决策模块，配置为当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最低级时，降低所述选定视频传输等级，而当所述传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最高级时，提高所述选定视频传输等级。
可选地，在本发明的一实施例中，视频参数包括分辨率和最大码率。
本发明还提供了一种视频传输参数的自适应调节系统，包括：存储器，用于存储可由处理器执行的指令；以及处理器，用于执行所述指令以实现上述的视频传输参数的自适应调节方法。
本发明还提供了一种存储有计算机程序代码的计算机可读介质，所述计算机程序代码在由处理器执行时实现上述的视频传输参数的自适应调节方法。
与现有技术相比，本发明具有以下优点：
本发明的视频传输参数的自适应调节方法和系统，通过监测IPC与接收端的视频数据帧传输情况，自动调节视频编码的分辨率与码率。在网络不顺畅时自动降低码率，减少视频卡顿；在网络顺畅时提升码率，提供更清晰的视频画面。本发明与现有技术的区别在于，本发明的方案是将I帧(关键帧)的传输状态作为是否调节码率等级的决策条件，具有很好的针对性和显著的效果，尤其是在网络波动的情况下，本发明中的视频传输参数的自适应调节方案可以提 高用户播放体验。
附图概述
包括附图是为提供对本申请进一步的理解，它们被收录并构成本申请的一部分，附图示出了本申请的实施例，并与本说明书一起起到解释本发明原理的作用。附图中：
图1是本发明一实施例的一种视频传输参数的自适应调节方法的流程示意图；
图2是本发明一实施例的一种一种视频传输参数的自适应调节方法中包含关键帧与非关键帧的图像组示意图；
图3是本发明另外一实施例的一种视频传输参数的自适应调节方法的流程示意图；
图4是本发明一实施例的一种视频传输参数的自适应调节系统的系统框图；以及
图5是本发明另外一实施例的一种视频传输参数的自适应调节系统的系统框图。
本发明的较佳实施方式
为了更清楚地说明本申请的实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本申请的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本申请应用于其他类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。
如本申请和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其他的步骤或元素。
除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本申请的范围。同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下， 所述技术、方法和设备应当被视为授权说明书的一部分。在这里示出和讨论的所有示例中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它示例可以具有不同的值。应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
在本申请的描述中，需要理解的是，方位词如“前、后、上、下、左、右”、“横向、竖向、垂直、水平”和“顶、底”等所指示的方位或位置关系通常是基于附图所示的方位或位置关系，仅是为了便于描述本申请和简化描述，在未作相反说明的情况下，这些方位词并不指示和暗示所指的装置或元件必须具有特定的方位或者以特定的方位构造和操作，因此不能理解为对本申请保护范围的限制；方位词“内、外”是指相对于各部件本身的轮廓的内外。
为了便于描述，在这里可以使用空间相对术语，如“在……之上”、“在……上方”、“在……上表面”、“上面的”等，用来描述如在图中所示的一个器件或特征与其他器件或特征的空间位置关系。应当理解的是，空间相对术语旨在包含除了器件在图中所描述的方位之外的在使用或操作中的不同方位。例如，如果附图中的器件被倒置，则描述为“在其他器件或构造上方”或“在其他器件或构造之上”的器件之后将被定位为“在其他器件或构造下方”或“在其他器件或构造之下”。因而，示例性术语“在……上方”可以包括“在……上方”和“在……下方”两种方位。该器件也可以其他不同方式定位(旋转90度或处于其他方位)，并且对这里所使用的空间相对描述作出相应解释。
此外，需要说明的是，使用“第一”、“第二”等词语来限定零部件，仅仅是为了便于对相应零部件进行区别，如没有另行声明，上述词语并没有特殊含义，因此不能理解为对本申请保护范围的限制。此外，尽管本申请中所使用的术语是从公知公用的术语中选择的，但是本申请说明书中所提及的一些术语可能是申请人按他或她的判断来选择的，其详细含义在本文的描述的相关部分中说明。此外，要求不仅仅通过所使用的实际术语，而是还要通过每个术语所蕴含的意义来理解本申请。
应当理解，当一个部件被称为“在另一个部件上”、“连接到另一个部件”、“耦合于另一个部件”或“接触另一个部件”时，它可以直接在该另一个部件 之上、连接于或耦合于、或接触该另一个部件，或者可以存在插入部件。相比之下，当一个部件被称为“直接在另一个部件上”、“直接连接于”、“直接耦合于”或“直接接触”另一个部件时，不存在插入部件。同样的，当第一个部件被称为“电接触”或“电耦合于”第二个部件，在该第一部件和该第二部件之间存在允许电流流动的电路径。该电路径可以包括电容器、耦合的电感器和/或允许电流流动的其它部件，甚至在导电部件之间没有直接接触。
本申请中使用了流程图用来说明根据本申请的实施例的系统所执行的操作。应当理解的是，前面或下面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各种步骤。同时，或将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
图1是本申请一实施例的一种视频传输参数的自适应调节方法10(以下简称“调节方法10”)示意图。如图1所示，本申请提供了一种视频传输参数的自适应调节方法10，包括如下的步骤。
S1为配置多个视频传输等级。
S2为使视频按照选定视频传输等级中的视频参数的设定值进行传输，并记录视频中连续多个关键帧的传输时间。
S3为统计每个传输时间与第一时间阈值和第二时间阈值的大小关系并调整视频传输等级。
下面对步骤S1-S3作进一步详细说明。
在步骤S1中，多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值。
具体的，在本发明的一实施例中，调节方法10适用于安防监控网络摄像机(IP Camera，IPC)视频的传输。示例性的如图2所示，在IPC视频传输数据的场景中，调节方法10中的关键帧为帧内编码图像帧(Intra-coded Picture)，即I帧。与之相对的还有前向预测编码图像帧(Predictive-coded Picture)，即P帧。在实际应用过程中，IPC编码器将采集到的原始图像进行压缩，编码成为连续的图像组GOP(Group of Pictures)。在图2中可以清楚的看出，一个GOP有一个I帧(关键帧1)和多个P帧(非关键帧1，非关键帧2……)组成，其中，I帧可以独立解码出完整的原始图像，P帧则仅记录当前帧与GOP内I 帧或之前P帧的差异，因此I帧的数据量常常远大于P帧。这也造成了现有技术中对于此类视频传输数据难以对码率等参数进行调整决策的瓶颈。
优选的，在本发明的一实施例中，步骤S1中的视频参数包括分辨率和最大码率。其中分辨率是用于度量图像内数据量多少的参数，通常表示成ppi(每英寸像素，pixel per inch)，例如一视频的分辨率为1920*1080，意指该视频在横向和纵向上的有效像素分别为1920ppi和1080ppi。在实际应用，例如视频传输过程中，分辨率影响着视频的清晰度。码率是数据传输时单位时间内传送的数据位数，常用的单位是kbps(千位每秒，也可用kb/s表示)。码率越大，精度就越高，处理得到的文件就越接近原始文件，同时也对传输带宽以及存储空间有更高的要求。需要强调的是，在本实施例中视频参数中为最大码率，而非平均码率，因为上文中提到了，关键帧的数据量常常远大于非关键帧，且出于实际应用考虑，关键帧的间隔较长，从而导致视频的实时码率波动较大，且平均码率远低于最大码率。此时如果根据平均码率对视频传输参数进行调整，可能导致关键帧的传输超时。因此，作为优选地，本发明的一些实施例中主要选择分辨率和最大码率这两个参数作为步骤S2中的视频参数。
进一步的，在步骤S2中，选定视频传输等级可以是多个视频传输等级中的任一级。优选地，在本申请的一些实施例中，在默认状态下，选定视频传输等级为最高级。关于视频传输等级的选取以及在实际工作过程中的变化，将在下文做出更详细的说明。
最后，在图1的步骤S3中，调整视频传输等级的具体方式如下。当传输时间大于第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且选定视频传输等级不是多个视频传输等级中的最低级时，降低选定视频传输等级，而当传输时间小于第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且选定视频传输等级不是多个视频传输等级中的最高级时，提高选定视频传输等级。
具体的，在本申请的一些实施例中，步骤S3中的第一时间阈值为帧间隔的2～4倍，第二时间阈值为帧间隔的1/4～1/2。帧间隔(Inter-Frame Space，IFS)表示帧传输之间的时间间隔，例如设置每秒传输帧数为10帧，则帧间隔为0.1秒。正常传输状态下，当传输时间小于帧间隔时，视作传输效果较好，当传输 时间大于帧间隔时，可能会存在关键帧传输超时的情况，而当超过第一阈值的连续多个关键帧的个数达到第一个数阈值时，这种情况愈发明显，关键帧画面延迟或卡顿严重，此时根据本申请所提供的技术方案，可以降低初始选定的视频传输等级，即降低视频参数例如最大码率等以提高视频传输的流畅度。可以理解的是，在当前选定视频传输等级已经是多个视频传输等级中的最低级时不进行降低等级操作。
另一方面，当传输时间远小于帧间隔时，传输效果较为流畅，此时可以进一步提高视频传输等级从而获取更高的视频传输质量，即当关键帧的传输数据小于第二阈值的连续多个关键帧的个数达到第二个数阈值时，降低选定视频传输等级。同样可以理解的是，在当前选定视频传输等级已经是多个视频传输等级中的最高级时不进行提高等级操作。
为了更好的说明本发明的视频传输参数的自适应调节方案，图3是本申请另外一实施例的单个帧传输时的视频传输参数的自适应调节方法30(以下简称“调节方法30”)的流程示意图。可以理解的是，图3示出的调节方法30是在图1所示的调节方法10的基础上，实现在一个具体实施例中的较为完整的调节方式。参考图3所示，在本发明的一实施例中，当一个新的数据帧产生并开始传输，即步骤S301，当前帧产生，该当前帧经历下述步骤。
步骤S302为判断正在传输的当前帧是否为关键帧(例如是上文提到的I帧)，若当前帧为关键帧时，进入步骤S303，这是关键帧部分的流程步骤。而若当前帧为非关键帧(例如是上文提到的P帧)时，进入步骤S308以及其之后的流程，这是非关键帧部分的流程步骤，这一部分将在下文做更详细的介绍。下面首先说明，若当前帧为关键帧的进一步调节流程。；
现在已知当前帧为关键帧，继续进行步骤S303，记录当前帧(关键帧)的传输时间，并将其传输时间结合在该当前帧(关键帧)之前的连续多个关键帧的传输时间统计结果，综合的判断是否执行变换视频传输等级的决策。示例性的，在步骤S304中，若当前帧传输时间高于第一时间阈值T1的连续次数达到第一个数阈值N1，则执行步骤S309降低视频传输等级，并进一步的执行步骤S310清空当下的传输队列，以及执行步骤S311丢弃正在传输的当前帧，通过这样的方式确保在此之后的新的视频帧再次产生后，将重新以降低后的视频传 输等级进行传输，从而在成本最小的前提下提高视频数据传输的流畅度。
需要补充说明的是，在本申请一些优选的实施例中，在传输视频的首个关键帧或第一个至第n个关键帧时，将第一个数阈值N1设置为1，其中，n为大于1的整数，示例性的n可以为2或者3。这意味着如果首个或首几个关键帧的传输时间大于第一时间阈值T1，则降低当前选定的视频传输等级，直至视频传输稳定后再通过图3中的流程提高视频传输等级。将传输视频的首个(或首几个)关键帧对应的第一个数阈值设置为1的好处是，当设备刚刚连接入网后，可以根据网络状态快速调节传输等级，以实现视频的流畅传输与播放。
另一方面，若步骤S304的判断结果为否，则继续执行步骤S305，继续判断当前帧传输时间是否处于低于第二时间阈值T2，且综合考虑在此当前帧之前的连续多个关键帧的传输时间，判定在此当前帧传输后，传输时间低于T2的连续次数是否已经达到N2，若步骤S305的判断结果为是，则执行步骤S307提高视频传输等级。而在提高视频传输等级的同时，执行步骤S306，将当前帧(关键帧)仍按照当前的视频传输等级进行传输或将该当前帧(关键帧)加入传输队列，这样的设置可以在优化视频传输效果的基础上，尽可能的降低丢失视频帧的成本。
相比之下，若步骤S305的判断结果为否，则意味着在图3示出的实施例中，当前帧的传输时间既没有达到可以提高视频传输等级的界限，也没有达到可以降低视频传输等级的界限，此时维持当前的视频传输等级并继续监控下一个产生的数据帧的传输过程，直至在某一个视频帧传输后达到了可以变换视频传输等级的界限，此时再依据具体情况变换视频传输等级，从而可以及时调节视频传输参数，提高视频传输的整体效果。
可以理解的是，在图3示出的实施例中，步骤S304和步骤S305被安排为具有先后顺序，但是本发明不以此为限，例如，在本发明的一些其他的实施例中，可以将步骤S304和步骤S305之间的顺序进行调换，或者在同一个步骤中综合地判断当前帧传输时间与第一时间阈值T1、第二时间阈值T2、第一个数阈值N1以及第二个数阈值N2的关系，本发明不对此做出限制。
下面说明非关键帧相关的流程步骤。如上所述，若步骤S302的判断结果为否，意味着当前帧是非关键帧(例如P帧)，此时执行步骤S308，判断与该 当前帧(非关键帧)同组的关键帧是否已经传输或者在传输队列中，如果判断结果为是，则继续等待其同组的关键帧的传输情况，继而对是否变换视频传输等级进行决策。而若步骤S308的判断结果为否，意味着与当前帧(非关键帧)同组的关键帧未传输成功且未在传输队列中，其中，未在传输队列的原因可能是因为上文描述的步骤S310，同组关键帧被清空传输队列的操作从传输队列中清除。此时认为，在此非关键帧之前的同组数据帧已经处于传输停滞/未传输成功的卡顿状况。在这种情况下，由于同组数据帧当中，非关键帧的解码依赖于同组的关键帧，当关键帧未完成传输时，即便非关键帧传输成功，也无法对其进一步解码，因此，此流程设置执行步骤S311直接丢弃当前的非关键帧，从而可以在最短的时间内为视频数据的传输队列减少负荷，以使其在尽可能短的时间内恢复视频数据的流畅传输。可以看出的是，上述关于非关键帧传输的子流程(即步骤S308、S311)为保障视频传输的流畅性提供了很好的辅助作用。
为了进一步理解图3示出的调节方法30的技术细节，结合图2中的图像组(GOP)示意图对其中的关键帧1和非关键帧1参照图3示出的流程进行进一步的具体说明。
具体的，在本发明的一实施例中，设定视频传输等级个数为3个，每个视频传输等级中配置不同的分辨率和最大码率这两个视频传输参数。具体如下方的表格1所示，其中，视频传输等级1是最高级，视频传输等级2和3在视频传输等级1的基础上依次降低。
表格1
视频传输等级 分辨率 最大码率(kbps)
1 1920*1080 1740
2 704*576 870
3 704*576 108
如表格1所示，在该实施例中，视频传输等级1对应的某测试场景中关键帧的数据大小(单位byte)为57835，而视频传输等级2和3对应的某测试场景中关键帧的数据大小(单位byte)分别为15534和5166，可以看出的是，相邻等级所对应的关键帧数据大小大约在2-3倍作用，从而可以在调节时更好的区分需要切换视频传输等级的关键帧数据量区段，从而更符合视频传输效果的 要求。在本申请包括图3的一些实施例中，在等级次序中相邻传输等级所对应的关键帧数据大小的差别满足数据差阈值(例如是2倍或3倍)。
结合参考图2、图3以及表格1所示，当图2中的当前GOP中的关键帧1开始传输，首先进入步骤S303，即记录当前关键帧1的传输时间，然后进入步骤S304，结合在关键帧1之前的连续多个关键帧传输时间的历史数据，判断当前关键帧1的传输时间大于第一时间阈值T1的连续多个关键帧的个数是否达到第一个数阈值N1。
具体的，若当前的关键帧1并非此次视频传输的首个或首几个关键帧，可以设定第一个数阈值与第二个数阈值为3。首先执行步骤S304判断关键帧1传输时间大于第一时间阈值T1的连续多个关键帧的个数是否达到3个，若已经达到，则执行步骤S309降低视频传输等级，并执行步骤S310清空发送队列以及步骤S311丢弃当前帧，再进入下一帧(即非关键帧1，也是当前GOP中的第一个P帧)的传输；若上述的个数并没有达到阈值3，就进一步执行步骤S305判断传输时间小于第二时间阈值T2的连续多个关键帧的个数是否达到3，若已经达到，则执行步骤S307提高视频传输等级，并将当前帧(关键帧1)加入发送队列；若并未达到，则不调整当前选定的视频传输等级，直接将当前帧(关键帧1)加入发送队列。
上述说明示例性的展示了关键帧1的传输流程，若当前帧为非关键帧1时，其传输的流程具体如下。
首先判断该非关键帧1同组的关键帧1是否已经发送或在发送队列中，若关键帧1并未完成传输且不在传输队列中，直接丢弃当前帧(非关键帧1)，进入下一帧(非关键帧2)的传输；若同组的关键帧1已发送或仍在发送队列中，下一步骤为步骤S304，即判断关键帧1的传输时间大于第一时间阈值T1的连续多个关键帧的个数是否达到第一个数阈值N1，接下来的流程与前文所述关键帧的发送流程相同，此处不再赘述。通过这样的设定，本方案考虑到关键帧1的数据量过大，因此允许关键帧1在传输时具有一定的延迟(但一般不超过几个帧间隔)，此时，关键帧1同组的其他非关键帧1、非关键帧2……在实际传输时将会依赖关键帧1的实际传输情况而进行变换当前选定的视频传输等级的决策。
示例性的，若关键帧1在执行步骤S304时遇到了当前帧传输时间高于T1的连续次数达到N1的情况，则此时选定的视频传输等级需要降级，并需要清空传输队列以及丢弃当前帧。这意味着如果当前GOP中的关键帧1遇到了上述的需要降级的临界情况，在关键帧1后面的同组中的非关键帧会经过步骤S311被直接丢弃，以此来迅速恢复视频传输的流畅度，直至下一个GOP中的关键帧再次开始传输。
需要强调的是，上述所提到的提升视频传输等级的前提是当前视频传输等级不是最高级，同样的，上述所提到的降低视频传输等级的前提是当前视频传输等级不是最低级。在本发明包括图1或图3的实施例中，视频传输等级的调整需要逐级进行。
在例如安防视频等具体应用场景中，关键帧(I帧)所包含的数据量常常远高于非关键帧(P帧)，因此，如何解决因数据量差异较大而导致的难以平衡数据传输质量和速率以及视频分辨率参数等成为了现有技术的瓶颈。本申请所给出的如上文所述的一种视频传输参数的自适应调节方法针对关键帧的历史传输时间信息进行统计，使关键帧的传输时间成为决策视频传输等级变换的重要决策因素，对于解决现有技术存在的问题具有很好的针对性和显著的效果。
根据图4，本发明的另一方面还提出了一种视频传输参数的自适应调节系统40(以下简称“调节系统40”)。调节系统40包括传输等级配置模块41、关键帧传输统计模块42以及传输等级决策模块43。
传输等级配置模块41适于配置多个视频传输等级，多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值。关键帧传输统计模块42配置为当视频按照选定视频传输等级中的视频参数的设定值进行传输时，记录视频中连续多个关键帧的传输时间，其中，选定视频传输等级为多个视频传输等级中的任一级，关键帧传输统计模块还配置为统计每个传输时间与第一时间阈值和第二时间阈值的大小关系。传输等级决策模块43配置为当传输时间大于第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且选定视频传输等级不是多个视频传输等级中的最低级时，降低选定视频传输等级，而当传输时间小于第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且选定视频传输等级不是多 个视频传输等级中的最高级时，提高选定视频传输等级。优选的，在本申请的一些实施例中视频参数具体为分辨率和最大码率。
示例性的，本发明的调节系统40可以适用上述参照图1～图3说明的调节方法10或30，关于调节系统40的更多细节可以参考上述的说明，在此不再赘述。
本发明的一实施例还提出了一种如图5所示的视频传输参数的自适应调节系统50。根据图5，视频传输参数的自适应调节系统50可包括内部通信总线51、处理器(Processor)52、只读存储器(ROM)53、随机存取存储器(RAM)54、以及通信端口55。当应用在个人计算机上时，视频传输参数的自适应调节系统50还可以包括硬盘56。
内部通信总线51可以实现视频传输参数的自适应调节系统50组件间的数据通信。处理器52可以进行判断和发出提示。在一些实施例中，处理器52可以由一个或多个处理器组成。通信端口55可以实现视频传输参数的自适应调节系统50与外部的数据通信。在一些实施例中，视频传输参数的自适应调节系统50可以通过通信端口55从网络发送和接受信息以及数据。
视频传输参数的自适应调节系统50还可以包括不同形式的程序储存单元以及数据储存单元，例如硬盘56，只读存储器(ROM)53和随机存取存储器(RAM)54，能够存储计算机处理和/或通信使用的各种数据文件，以及处理器52所执行的可能的程序指令。处理器执行这些指令以实现方法的主要部分。处理器处理的结果通过通信端口传给用户设备，在用户界面上显示。
除此之外，本发明另一方面还提出了一种存储有计算机程序代码的计算机可读介质，计算机程序代码在由处理器执行时实现上述的视频传输参数的自适应调节方法。
上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述发明披露仅仅作为示例，而并不构成对本申请的限定。虽然此处并没有明确说明，本领域技术人员可能会对本申请进行各种修改、改进和修正。该类修改、改进和修正在本申请中被建议，所以该类修改、改进、修正仍属于本申请示范实施例的精神和范围。
同时，本申请使用了特定词语来描述本申请的实施例。如“一个实施例”、 “一实施例”、和/或“一些实施例”意指与本申请至少一个实施例相关的某一特征、结构或特点。因此，应强调并注意的是，本说明书中在不同位置两次或多次提及的“一实施例”或“一个实施例”或“一替代性实施例”并不一定是指同一实施例。此外，本申请的一个或多个实施例中的某些特征、结构或特点可以进行适当的组合。
本申请的一些方面可以完全由硬件执行、可以完全由软件(包括固件、常驻软件、微码等)执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。处理器可以是一个或多个专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理器件(DAPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器或者其组合。此外，本申请的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。例如，计算机可读介质可包括，但不限于，磁性存储设备(例如，硬盘、软盘、磁带……)、光盘(例如，压缩盘CD、数字多功能盘DVD……)、智能卡以及闪存设备(例如，卡、棒、键驱动器……)。
计算机可读介质可能包含一个内含有计算机程序编码的传播数据信号，例如在基带上或作为载波的一部分。该传播信号可能有多种表现形式，包括电磁形式、光形式等等、或合适的组合形式。计算机可读介质可以是除计算机可读存储介质之外的任何计算机可读介质，该介质可以通过连接至一个指令执行系统、装置或设备以实现通讯、传播或传输供使用的程序。位于计算机可读介质上的程序编码可以通过任何合适的介质进行传播，包括无线电、电缆、光纤电缆、射频信号、或类似介质、或任何上述介质的组合。
同理，应当注意的是，为了简化本申请披露的表述，从而帮助对一个或多个发明实施例的理解，前文对本申请实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本申请对象所需要的特征比权利要求中提及的特征多。实际上，实施例的特征要少于上述披露的单个实施例的全部特征。
一些实施例中使用了描述成分、属性数量的数字，应当理解的是，此类用于实施例描述的数字，在一些示例中使用了修饰词“大约”、“近似”或“大 体上”来修饰。除非另外说明，“大约”、“近似”或“大体上”表明数字允许有±20％的变化。相应地，在一些实施例中，说明书和权利要求中使用的数值参数均为近似值，该近似值根据个别实施例所需特点可以发生改变。在一些实施例中，数值参数应考虑规定的有效数位并采用一般位数保留的方法。尽管本申请一些实施例中用于确认其范围广度的数值域和参数为近似值，在具体实施例中，此类数值的设定在可行范围内尽可能精确。
虽然本申请已参照当前的具体实施例来描述，但是本技术领域中的普通技术人员应当认识到，以上的实施例仅是用来说明本申请，在没有脱离本申请精神的情况下还可作出各种等效的变化或替换，因此，只要在本申请的实质精神范围内对上述实施例的变化、变型都将落在本申请的权利要求书的范围内。

Claims (14)

  1. 一种视频传输参数的自适应调节方法，其特征在于，包括如下的步骤：
    配置多个视频传输等级，所述多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值；
    使视频按照选定视频传输等级中的视频参数的设定值进行传输，并记录所述视频中连续多个关键帧的传输时间，其中，所述选定视频传输等级为所述多个视频传输等级中的任一级；以及
    统计每个所述传输时间与第一时间阈值和第二时间阈值的大小关系，当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最低级时，降低所述选定视频传输等级，而当所述传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最高级时，提高所述选定视频传输等级。
  2. 如权利要求1所述的方法，其特征在于，所述视频参数包括分辨率和最大码率。
  3. 如权利要求1所述的方法，其特征在于，在所述等级次序中，相邻传输等级所对应的关键帧数据大小的差别满足数据差阈值。
  4. 如权利要求1所述的方法，其特征在于，还包括判断正在传输的当前帧是否为所述关键帧，当所述当前帧为所述关键帧时，记录所述当前帧的传输时间，而当所述当前帧为非关键帧时，继续判断与所述当前帧同组的关键帧是否未完成传输或未在传输队列中，若判断结果为是，则丢弃所述当前帧。
  5. 如权利要求1所述的方法，其特征在于，在所述降低所述选定视频传输等级后，还包括清空传输队列并丢弃正在传输的当前帧。
  6. 如权利要求1所述的方法，其特征在于，若所述选定视频传输等级为所述多个视频传输等级中的最低级，则当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值时，维持所述选定视频传输等级，而若所述选定传输视频等级为所述多个视频传输等级中的最高级，则当传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值时，维持所述选定 视频传输等级。
  7. 如权利要求6所述的方法，其特征在于，在默认状态下，所述选定视频传输等级为所述最高级。
  8. 如权利要求1或7所述的方法，其特征在于，在传输所述视频的第一个关键帧或所述第一个至第n个关键帧时，将所述第一个数阈值设置为1，其中n为大于1的整数。
  9. 如权利要求1所述的方法，其特征在于，所述第一时间阈值为帧间隔的2～4倍，所述第二时间阈值为帧间隔的1/4～1/2。
  10. 如权利要求1所述的方法，其特征在于，所述多个视频传输等级的个数为3或4个。
  11. 一种视频传输参数的自适应调节系统，包括：
    传输等级配置模块，适于配置多个视频传输等级，所述多个视频传输等级按照可对应的关键帧数据大小具有由高到低的等级次序，且每个视频传输等级包括至少一个视频参数的设定值；
    关键帧传输统计模块，配置为当视频按照选定视频传输等级中的视频参数的设定值进行传输时，记录所述视频中连续多个关键帧的传输时间，其中，所述选定视频传输等级为所述多个视频传输等级中的任一级，所述关键帧传输统计模块还配置为统计每个所述传输时间与第一时间阈值和第二时间阈值的大小关系；以及
    传输等级决策模块，配置为当传输时间大于所述第一时间阈值的连续多个关键帧的个数达到第一个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最低级时，降低所述选定视频传输等级，而当所述传输时间小于所述第二时间阈值的连续多个关键帧的个数达到第二个数阈值，且所述选定视频传输等级不是所述多个视频传输等级中的最高级时，提高所述选定视频传输等级。
  12. 如权利要求11所述的系统，其特征在于，所述视频参数包括分辨率和最大码率。
  13. 一种视频传输参数的自适应调节系统，包括：
    存储器，用于存储可由处理器执行的指令；以及处理器，用于执行所述指 令以实现如权利要求1-10任一项所述的方法。
  14. 一种存储有计算机程序代码的计算机可读介质，所述计算机程序代码在由处理器执行时实现如权利要求1-10任一项所述的方法。
PCT/CN2022/142041 2021-12-27 2022-12-26 视频传输参数的自适应调节方法和系统 WO2023125443A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111618926.3 2021-12-27
CN202111618926.3A CN114172874B (zh) 2021-12-27 2021-12-27 视频传输参数的自适应调节方法和系统

Publications (1)

Publication Number Publication Date
WO2023125443A1 true WO2023125443A1 (zh) 2023-07-06

Family

ID=80488217

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/142041 WO2023125443A1 (zh) 2021-12-27 2022-12-26 视频传输参数的自适应调节方法和系统

Country Status (2)

Country Link
CN (1) CN114172874B (zh)
WO (1) WO2023125443A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114172874B (zh) * 2021-12-27 2024-07-30 上海商米科技集团股份有限公司 视频传输参数的自适应调节方法和系统
CN116260799B (zh) * 2023-05-16 2023-07-21 北京庭宇科技有限公司 一种调整网络状态的方法和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104125429A (zh) * 2013-04-27 2014-10-29 杭州海康威视数字技术股份有限公司 视频数据传输的调节方法及装置
CN104703051A (zh) * 2014-01-06 2015-06-10 杭州海康威视数字技术股份有限公司 码流发送方法和装置
CN109151612A (zh) * 2017-06-27 2019-01-04 华为技术有限公司 一种视频传输方法、设备及系统
US20190244029A1 (en) * 2016-10-18 2019-08-08 Zhejiang Dahua Technology Co., Ltd. Methods and systems for video processing
CN114172874A (zh) * 2021-12-27 2022-03-11 上海商米科技集团股份有限公司 视频传输参数的自适应调节方法和系统

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013070188A1 (en) * 2011-11-07 2013-05-16 Empire Technology Development Llc Redundant key frame transmission
CN102413307B (zh) * 2011-11-30 2013-11-27 晶门科技（深圳）有限公司 电视视频聊天的自适应码率调整方法及调整装置
CN103533356A (zh) * 2013-10-10 2014-01-22 深圳市智美达科技有限公司 视频数据传输方法和装置
CN104780367B (zh) * 2015-04-13 2018-05-01 浙江宇视科技有限公司 一种动态调整gop长度的方法和装置
US10356406B2 (en) * 2016-01-19 2019-07-16 Google Llc Real-time video encoder rate control using dynamic resolution switching
CN111245769B (zh) * 2018-11-28 2022-09-30 深圳技威时代科技有限公司 一种音视频传输方法、装置和存储介质
CN111669619B (zh) * 2020-06-04 2023-02-28 北京奇艺世纪科技有限公司 一种视频流数据切换方法、装置、终端和可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104125429A (zh) * 2013-04-27 2014-10-29 杭州海康威视数字技术股份有限公司 视频数据传输的调节方法及装置
CN104703051A (zh) * 2014-01-06 2015-06-10 杭州海康威视数字技术股份有限公司 码流发送方法和装置
US20190244029A1 (en) * 2016-10-18 2019-08-08 Zhejiang Dahua Technology Co., Ltd. Methods and systems for video processing
CN109151612A (zh) * 2017-06-27 2019-01-04 华为技术有限公司 一种视频传输方法、设备及系统
CN114172874A (zh) * 2021-12-27 2022-03-11 上海商米科技集团股份有限公司 视频传输参数的自适应调节方法和系统

Also Published As

Publication number Publication date
CN114172874B (zh) 2024-07-30
CN114172874A (zh) 2022-03-11

Similar Documents

Publication Publication Date Title
WO2021244341A1 (zh) 图像编码方法及装置、电子设备及计算机可读存储介质
WO2023125443A1 (zh) 视频传输参数的自适应调节方法和系统
CN101345871B (zh) 视频编解码的方法及其装置
US9071841B2 (en) Video transcoding with dynamically modifiable spatial resolution
CN106454395B (zh) 在服务器中用于自适应提供多码率流媒体的方法及装置
CN112953922B (zh) 一种自适应流媒体控制方法、系统、计算机设备及应用
JP6463041B2 (ja) 画像処理装置、画像処理方法、及びプログラム
CN103945281B (zh) 视频传输处理方法、装置和系统
US20080175475A1 (en) Method of image frame compression
CN112866746A (zh) 一种多路串流云游戏控制方法、装置、设备及存储介质
CN117412049A (zh) 基于多智能体强化学习的端到端码率自适应控制方法
CN102325275A (zh) 一种流媒体播放的丢包方法、装置及服务器
JP2007274066A (ja) コンテンツ配信システム
CN108810468B (zh) 一种优化显示效果的视频传输装置及方法
CN108024111B (zh) 一种帧类型判定方法及装置
WO2024113869A1 (zh) 视频编码方法及相关装置
WO2011030680A1 (ja) 符号化装置、符号化方法、および符号化プログラム
CN115052182B (zh) 基于队列学习和超分辨率的超高清视频传输系统与方法
US20100104009A1 (en) Methods and systems for improving network response during channel change
CN110062003A (zh) 视频数据发送方法、装置、电子设备及存储介质
CN115499654A (zh) 视频编码方法、装置、电子设备以及可读存储介质
KR20230040872A (ko) 비디오 스트림의 전송을 제어하는 방법들 및 장치들
US20120251086A1 (en) Trick mode technique for a bandwidth limited channel
CN115022667A (zh) 一种直播播放控制方法及相关设备
CN115334310A (zh) 视频数据编码方法、装置、电子设备和介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22914697

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22914697

Country of ref document: EP

Kind code of ref document: A1