WO2023174216A1 - Procédé, appareil et dispositif d'ajustement adaptatif de résolution vidéo, et support de stockage - Google Patents

Procédé, appareil et dispositif d'ajustement adaptatif de résolution vidéo, et support de stockage Download PDF

Info

Publication number
WO2023174216A1
WO2023174216A1 PCT/CN2023/081121 CN2023081121W WO2023174216A1 WO 2023174216 A1 WO2023174216 A1 WO 2023174216A1 CN 2023081121 W CN2023081121 W CN 2023081121W WO 2023174216 A1 WO2023174216 A1 WO 2023174216A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
image
processing
resolution
frame rate
Prior art date
Application number
PCT/CN2023/081121
Other languages
English (en)
Chinese (zh)
Inventor
陈俊奎
谢澜
陈新泽
Original Assignee
百果园技术(新加坡)有限公司
陈俊奎
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百果园技术(新加坡)有限公司, 陈俊奎 filed Critical 百果园技术(新加坡)有限公司
Publication of WO2023174216A1 publication Critical patent/WO2023174216A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2405—Monitoring of the internal components or processes of the server, e.g. server load
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/4424—Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used

Definitions

  • Embodiments of the present application relate to the field of video processing technology, and in particular, to a video resolution adaptive adjustment method, device, equipment and storage medium.
  • the anchor side collects video images through the camera, pre-processes the collected video images, and then renders and displays them on the screen of the anchor side.
  • the anchor side encodes the pre-processed video images and packages them into protocol packets, and forwards them to the audience side through the backend server.
  • the audience side decodes and renders them to the audience side's screen for display.
  • Embodiments of the present application provide a video resolution adaptive adjustment method, device, equipment and storage medium, which solves the problem of poor balance between clarity and fluency during video playback, realizes adaptive adjustment of video resolution, and provides Users are provided with a high-quality video experience.
  • embodiments of the present application provide a video resolution adaptive adjustment method, which method includes:
  • the maximum processing frame rate is calculated according to the processing time consumption, and the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;
  • the resolution of the video image is increased.
  • embodiments of the present application also provide a video resolution adaptive adjustment device, including:
  • the data acquisition module is configured to obtain the number of dropped frames of the video image and the processing time of a single frame image
  • a frame rate calculation module configured to calculate the maximum processing frame rate based on the processing time
  • a status information determination module configured to determine status information of the video frame based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;
  • a resolution adjustment module configured to lower the resolution of the video image if the status information indicates performance overload, and to increase the resolution of the video image if the status information indicates excess performance.
  • embodiments of the present application also provide a video resolution adaptive adjustment device, which includes:
  • processors one or more processors
  • a storage device for storing one or more programs
  • the one or more processors are caused to implement the video resolution adaptive adjustment method described in the embodiments of this application.
  • embodiments of the present application also provide a storage medium that stores computer-executable instructions, which when executed by a computer processor are used to perform the video resolution adaptation described in the embodiments of the present application. Adjustment method.
  • embodiments of the present application also provide a computer program product.
  • the computer program product includes a computer program.
  • the computer program is stored in a computer-readable storage medium.
  • At least one processor of the device reads the computer program from the computer-readable storage medium.
  • Obtain and execute the computer program causing the device to execute the video resolution adaptive adjustment method described in the embodiments of this application.
  • the number of dropped frames of the video image and the processing time of a single frame image are obtained.
  • the status information of the video frame is determined based on the number of dropped frames and the maximum processing frame rate.
  • the status information includes performance overload and excess performance. If the status information is performance overload, the resolution of the video image is adjusted downward. If the status information is excess performance, the resolution is adjusted upward. Adjusting the resolution of the video image achieves adaptive adjustment of the video resolution, weighs and optimizes the balance between clarity and smoothness during video playback, and provides users with a high-quality video experience.
  • Figure 1 is a flow chart of a video resolution adaptive adjustment method provided by an embodiment of the present application
  • Figure 2 is a flow chart of a method for determining status information of a video frame provided by an embodiment of the present application
  • Figure 3 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application.
  • Figure 4 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application.
  • Figure 5 is a structural block diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application.
  • first, second, etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first,” “second,” etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple.
  • “and/or” in the description and claims indicates at least one of the connected objects, and the character “/" generally indicates that the related objects are in an "or” relationship.
  • Figure 1 is a flow chart of a video resolution adaptive adjustment method provided by an embodiment of the present application, which can be used to adaptively adjust the resolution of video images.
  • This method can be performed by computing devices such as servers, smart terminals, and laptops. , tablet computers, etc. to execute, including the following steps:
  • Step S101 Obtain the number of dropped frames of the video image and the processing time of a single frame image.
  • the number of lost frames is the number of lost frames during the encoding process of the video image.
  • the video image passes through After processing, it is sent to the encoding queue to be compressed and encoded by the encoder.
  • the pre-processing process includes a series of processing processes such as data cleaning, skin color processing, skin grinding, noise reduction and face positioning for each frame of image collected by the camera.
  • the size of the encoding queue is a fixed length, that is, when the speed of encoding each frame of video image is less than the speed of the image input to the encoding queue, the excess image frames will overflow, causing frame loss and freezing.
  • the number of lost frames can be obtained by counting the number of frames input to the encoding queue and the number of encoding processing frames. For example, the number of frames input to the encoding queue within a set fixed time minus the number of encoding processing frames can be obtained. The number of dropped frames.
  • the processing time of the single frame image may be the processing time of each frame of the video image during the encoding process. It is calculated by the processing time of the acquired single frame image.
  • the calculation process includes: obtaining the processing time of a single frame image, and calculating the processing time of the single frame image based on the processing time and the set motion weighted exponential filtering formula. Specifically, the calculation formula is as follows:
  • t i is the processing time consumed by the actual encoding process of the acquired i-th frame
  • ⁇ is the motion weighting factor
  • the exemplary value is 0.996.
  • Step S102 Calculate the maximum processing frame rate according to the processing time, and determine the status information of the video frame based on the number of lost frames and the maximum processing frame rate.
  • the status information includes performance overload and excess performance.
  • the processing time is the processing time of each frame of image during the video image encoding process
  • the maximum processing frame rate corresponds to the calculated maximum processing frame rate during the encoding process. Specifically, it can be: calculating the number of image frames processed per unit time according to the processing time to obtain the maximum processing frame rate.
  • the maximum processing frame rate is recorded as fps enc_capacity
  • the calculated processing time of a single frame image is:
  • the maximum processing frame rate is 20 frames per second.
  • the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate obtained previously.
  • the status information includes performance overload and excess performance during the encoding process.
  • performance overload means that the current processing speed of video images cannot meet the speed of the input video images, that is, the image resolution needs to be reduced to ensure smooth playback of the video
  • overperformance means that the current processing speed of video images is higher than the input speed.
  • the speed of the video image can increase the image resolution to improve the clarity of the displayed video image.
  • Figure 2 is a flow chart of a method for determining status information of a video frame provided by an embodiment of the present application, which specifically includes:
  • Step S1021 Determine whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold.
  • the frame loss threshold is a dynamically determined threshold for comparison with the number of frames lost;
  • the standard frame rate threshold is a set value.
  • the standard frame rate threshold can be 11, 15, or 20, etc., and can be flexibly set according to different video application scenarios.
  • the step of calculating the frame loss threshold is also included.
  • the frame loss threshold is calculated based on different image resolutions. Specifically: the frame loss threshold is calculated based on the image resolution corresponding to the current gear and the lowered gear and the maximum processing frame rate, where the image resolution corresponding to the lowered gear is smaller than the image resolution corresponding to the current gear.
  • multiple different image resolution gears are preset. Each image resolution gear corresponds to a fixed image resolution value. If there are six gears, the corresponding Exemplary image resolutions are 240P, 360P, 480P, 540P, 720P and 1080P respectively.
  • the frame drop threshold is exemplarily recorded as Th enc_drop , and the frame drop threshold is calculated as follows:
  • fps enc_capacity is the maximum processing frame rate
  • i the image resolution level, the larger the value of i, the higher the corresponding image resolution.
  • the specific process of determining whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold may be: If the number of lost frames is greater than the calculated frame loss threshold, and the maximum processing frame rate is less than the set standard frame rate threshold, it is determined that the processing of video frames is performance overload.
  • Code coveruse represents performance overload
  • fps enc_capacity is the maximum processing frame rate
  • Th enc_fps is the set standard frame rate threshold
  • frames enc_drop is the number of dropped frames
  • Th enc_drop is the calculated ’s frame loss threshold
  • Step S1022 Determine whether the processing of the video frame has excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold.
  • the maximum processing frame rate is compared with the calculated maximum frame rate threshold.
  • the process of determining the maximum frame rate threshold before comparison includes: calculating the maximum frame rate threshold based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate, where the upper gear The image resolution corresponding to the gear position is greater than the image resolution corresponding to the current gear position.
  • multiple different image resolution gears are preset. Each image resolution gear corresponds to a fixed image resolution value. For example, if there are six gears, the gears are The corresponding image resolutions from low to high are, for example, 240P, 360P, 480P, 540P, 720P and 1080P respectively.
  • the set maximum frame rate is recorded as fps max , and the exemplary value may be 24, where the maximum frame rate threshold is calculated based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate.
  • the process can be:
  • the corresponding conditions for satisfying excess performance include: if the maximum processing frame rate is greater than the calculated maximum frame rate threshold, then the processing of the video frame is determined to be excess performance, and Codec underuse represents excess performance.
  • the corresponding judgment conditions can be expressed as follows:
  • Step S103 If the status information indicates performance overload, adjust the resolution of the video image downward; if the status information indicates excessive performance, adjust the resolution of the video image upward.
  • the status information if the status information is performance overload, then Adjust the resolution of the video image. If the status information indicates excess performance, increase the resolution of the video image. Regarding the encoding processing stage of the video image, if it is determined that the performance is overloaded, the current encoding resolution is lowered, and if it is determined that the performance is excessive, the current encoding resolution is increased.
  • multiple different image resolution gears are set for upward and downward adjustment of the image resolution, where each gear corresponds to a fixed resolution value.
  • each gear corresponds to a fixed resolution value.
  • adjust one gear at a time when adjusting the resolution of the video image, adjust one gear at a time.
  • the corresponding determined status information is a normal status if the determined status information is neither excessive performance nor performance overload.
  • the status information is judged every fixed period T. If the same status information is triggered cumulatively N times in the period T, for example, the judgment of performance overload is triggered 6 times within a period of 10 seconds. If so, adjust the image resolution accordingly. If the above conditions are not met, the resolution will not be adjusted. Specifically, when the video image resolution in the encoding stage is increased, in principle it should not exceed the highest image resolution captured by the camera device. Of course, the downward adjustment process is also limited by the image resolution corresponding to the lowest gear set.
  • the video image resolution especially the resolution of the camera image acquisition is limited by the model, it may only provide image acquisition at certain resolutions, and the pre-processing or encoding process is set as needed.
  • it also includes: cropping the video image collected by the camera device through the image cropping algorithm to obtain different gears video images at lower resolutions. That is, the image is cropped through the image cropping algorithm to obtain multiple images with different resolutions for adjusting the image resolution.
  • the status information of the video frame is determined based on the number of dropped frames and the maximum processing frame rate.
  • the status information includes performance overload and excess performance. If the status information indicates performance overload, the resolution of the video image is lowered. If the status information indicates excess performance, the resolution of the video image is raised. This is achieved.
  • the adaptive adjustment of video resolution balances and optimizes the balance between clarity and smoothness during video playback, providing users with a high-quality video experience.
  • the processing time of a single frame image is calculated through the moving weighted exponential filtering formula, making the calculation of the processing time more accurate.
  • the processing time of a single frame image The processing time and the calculated maximum processing frame rate are used as the basis for judging the status information.
  • the image resolution can be effectively adjusted based on this.
  • dynamic calculation is used for the frame loss threshold and the maximum frame rate threshold. Obtained, the effect of resolution adaptive adjustment is greatly optimized.
  • Figure 3 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application. It provides a process of video resolution adjustment in the encoding stage, as shown in Figure 3, which specifically includes:
  • Step S201 Obtain the processing time of a single frame image in the video image encoding processing stage, and calculate the processing time of the single frame image according to the processing time and the set motion weighted exponential filter formula.
  • Step S202 Obtain the number of lost frames in the video image encoding processing stage and the processing time of the single frame image, and calculate the number of image frames processed per unit time according to the processing time to obtain the maximum encoding processing frame rate.
  • Step S203 Calculate the frame loss threshold based on the image resolution corresponding to the current gear and the lowered gear and the maximum encoding frame rate, calculate the frame loss threshold based on the number of frames lost, the maximum encoding frame rate and the set standard The frame rate threshold determines whether the encoding process of video frames is a performance overload.
  • Step S204 Calculate the maximum frame rate threshold based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate, and determine whether the video frame is encoded based on the maximum encoding processing frame rate and the calculated maximum frame rate threshold. For excess performance.
  • Step S205 If the number of lost frames is greater than the calculated frame loss threshold and the maximum encoding processing frame rate is less than the set standard frame rate threshold, it is determined that the encoding processing of the video frame is performance overload. If the maximum encoding processing frame rate is greater than the calculated frame rate If the maximum frame rate threshold is reached, it is determined that the encoding process of the video frame has excessive performance.
  • FIG 4 is a flow chart of another video resolution adaptive adjustment method provided by the embodiment of the present application, which provides the process of video resolution adjustment in the previous processing stage, as shown in Figure 4, which specifically includes:
  • Step S301 Obtain the number of dropped frames in the video image pre-processing stage and the processing time of a single frame image.
  • a pre-processing queue is also maintained when pre-processing video images.
  • Each frame of video image collected by the camera is input into the pre-processing queue in turn.
  • each frame of image in the pre-processing queue is processed in sequence. Taking the live broadcast scene as an example, it includes a series of processing processes such as data cleaning, skin color processing, skin resurfacing, noise reduction, and face positioning. If the speed of image pre-processing is lower than the speed of the input image, frame loss will occur.
  • the number of lost frames in the pre-processing stage can be obtained by counting the number of frames input into the pre-processing queue and the number of frames processed in the pre-processing, such as subtracting the number of frames input into the pre-processing queue within a set fixed time.
  • the specific number of frames lost can be obtained by removing the number of frames processed in pre-processing.
  • the processing time of a single frame image in the pre-processing stage of the video image is the determined time-consuming of the single acquisition frame in the entire pre-processing stage.
  • the calculation method can refer to the processing time of a single frame image in the encoding stage, which is also calculated based on the motion weighted exponential filtering formula and will not be described again here.
  • Step S302 Calculate the maximum pre-processing frame rate based on the processing time, and determine the status information of the video frame based on the number of lost frames and the maximum pre-processing frame rate.
  • the calculation method of obtaining the maximum pre-processing frame rate based on the processing time consumption is referred to the calculation method of obtaining the maximum encoding processing frame rate based on the processing time consumption.
  • the status information of the video frame is determined based on the number of dropped frames and the maximum pre-processing frame rate.
  • the process can also refer to the process of determining the status information of the video frame based on the number of dropped frames and the maximum encoding processing frame rate at the same time, that is, the explanation part of step S102, which will not be described again here.
  • Step S303 If the status information indicates performance overload, lower the pre-processing resolution of the video image. If the status information indicates excessive performance, increase the pre-processing resolution of the video image.
  • step S103 For the specific resolution adjustment method, please refer to the explanation part of step S103 and will not be described again here.
  • the processes of adaptive adjustment of image resolution for the pre-processing stage and encoding processing stage are respectively given.
  • the two stages can adjust the image resolution independently, or use linked Adjust the image resolution in the form.
  • the encoding processing stage when the image resolution is lowered to reduce the image resolution in the pre-processing stage, the encoding processing stage also triggers a downward adjustment of the resolution to be consistent with the resolution lowered in the pre-processing; the current processing stage performs an upward adjustment of the image resolution.
  • the encoding processing stage also triggers a resolution increase to be consistent with the resolution increase in pre-processing. It should be noted that the calculation formula of the above encoding processing and the description of the specific resolution adjustment method are also applicable to the pre-processing stage, and will not be described again here.
  • FIG. 5 is a structural block diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application.
  • the device is used to execute the video resolution adaptive adjustment method provided by the above embodiment, and has functional modules and beneficial effects corresponding to the execution method.
  • the device specifically includes: data acquisition module 101, frame rate calculation module 102, status information determination module 103 and resolution adjustment module 104, wherein,
  • the data acquisition module 101 is configured to obtain the number of dropped frames of the video image and the processing time of the single frame image;
  • the frame rate calculation module 102 is configured to calculate the maximum processing frame rate based on the processing time
  • the status information determination module 103 is configured to determine the status information of the video frame based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;
  • the resolution adjustment module 104 is configured to lower the resolution of the video image if the status information indicates performance overload, and to increase the resolution of the video image if the status information indicates excessive performance.
  • the status information of the video frame is determined based on the number of dropped frames and the maximum processing frame rate.
  • the status information includes performance overload and excess performance. If the status information indicates performance overload, the resolution of the video image is lowered. If the status information indicates excess performance, the resolution of the video image is raised. This is achieved.
  • the adaptive adjustment of video resolution balances and optimizes the balance between clarity and smoothness during video playback, providing users with a high-quality video experience.
  • the data acquisition module 101 is configured as:
  • the device also includes a module for processing time-consuming calculations, configured to:
  • the processing time of a single frame image is calculated based on the processing time and the set motion weighted exponential filtering formula.
  • the frame rate calculation module 102 is configured as:
  • the status information determination module 103 is configured as:
  • the status information determination module 103 is further configured to:
  • the frame loss threshold is calculated based on the image resolution and the maximum processing frame rate, where, the The image resolution corresponding to the lowered gear is smaller than the image resolution corresponding to the current gear.
  • the status information determination module 103 is configured as:
  • the status information determination module 103 is further configured to:
  • the maximum frame is calculated based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate. rate threshold, wherein the image resolution corresponding to the upward gear is greater than the image resolution corresponding to the current gear.
  • the status information determination module 103 is configured as:
  • the data acquisition module 101 is configured as:
  • the resolution adjustment module 104 is configured as:
  • the resolution of the video image in the pre-processing stage and/or the encoding process stage is reduced, and the resolution of the video image in the pre-processing stage and/or the encoding process stage is increased.
  • the resolution adjustment module 104 is configured as:
  • the video image collected by the camera device is cropped through an image cropping algorithm to obtain video images with resolutions in different gears.
  • Figure 6 is a schematic structural diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application.
  • the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the processor in the device
  • the number of 201 may be one or more.
  • one processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device may be connected through a bus or other means.
  • the memory 202 can be used to store software programs, computer-executable programs and modules, such as program instructions/modules corresponding to the video resolution adaptive adjustment method in the embodiment of the present application.
  • the processor 201 executes software programs, instructions and modules stored in the memory 202 to perform various functions of the device. Application and data processing, that is, to implement the above-mentioned video resolution adaptive adjustment method.
  • the input device 203 may be used to receive input numeric or character information and generate key signal inputs related to user settings and functional control of the device.
  • the output device 204 may include a display device such as a display screen.
  • Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform a video resolution adaptive adjustment method described in the above embodiments, wherein, include:
  • the maximum processing frame rate is calculated according to the processing time consumption, and the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;
  • the resolution of the video image is increased.
  • various aspects of the method provided by this application can also be implemented in the form of a program product, which includes program code.
  • the program product When the program product is run on a computer device, the program code is used to The computer device is caused to execute the steps in the method according to the various exemplary embodiments of the present application described above.
  • the computer device may execute the video resolution adaptive adjustment method described in the embodiment of the present application.
  • the program product may be implemented in any combination of one or more readable media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Les modes de réalisation de la présente demande concernent un procédé, un appareil et un dispositif d'ajustement adaptatif de résolution vidéo, et un support de stockage. Le procédé comprend : l'acquisition du nombre de trames perdues d'une image vidéo et du temps de traitement consommé pour une seule trame d'image ; le calcul d'une fréquence de trame de traitement maximale selon le temps de traitement consommé, et sur la base du nombre de trames perdues et de la fréquence de trame de traitement maximale, la détermination d'informations d'état d'une trame vidéo, les informations d'état comprenant une surcharge des performances et un excès des performances ; et si les informations d'état sont une surcharge des performances, la réduction de la résolution de l'image vidéo, et si les informations d'état sont un excès des performances, l'augmentation de la résolution de l'image vidéo. La présente solution accomplit un ajustement adaptatif de la résolution d'une vidéo, et pondère et optimise un équilibre entre la définition et la régularité durant la lecture d'une vidéo, de sorte qu'une expérience vidéo de grande qualité est fournie à un utilisateur.
PCT/CN2023/081121 2022-03-18 2023-03-13 Procédé, appareil et dispositif d'ajustement adaptatif de résolution vidéo, et support de stockage WO2023174216A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210272513.2 2022-03-18
CN202210272513.2A CN114827662B (zh) 2022-03-18 2022-03-18 视频分辨率自适应调节方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2023174216A1 true WO2023174216A1 (fr) 2023-09-21

Family

ID=82529981

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/081121 WO2023174216A1 (fr) 2022-03-18 2023-03-13 Procédé, appareil et dispositif d'ajustement adaptatif de résolution vidéo, et support de stockage

Country Status (2)

Country Link
CN (1) CN114827662B (fr)
WO (1) WO2023174216A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118337998A (zh) * 2024-06-17 2024-07-12 北京欣博电子科技有限公司 多路视频编解码的调度方法、装置、计算机设备及介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827662B (zh) * 2022-03-18 2024-06-25 百果园技术(新加坡)有限公司 视频分辨率自适应调节方法、装置、设备和存储介质
CN117041669B (zh) * 2023-09-27 2023-12-08 湖南快乐阳光互动娱乐传媒有限公司 视频流的超分控制方法、装置及电子设备
CN117478934B (zh) * 2023-12-26 2024-04-16 南昌航天广信科技有限责任公司 一种视频流数据传输方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103702061A (zh) * 2013-12-23 2014-04-02 上海网达软件股份有限公司 一种智能终端视频通话分辨率自动调节方法
CN111225240A (zh) * 2020-01-19 2020-06-02 安徽文香信息技术有限公司 自动调整占用带宽的方法、装置、存储介质与电子设备
CN113038222A (zh) * 2021-03-08 2021-06-25 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及存储介质
CN113242434A (zh) * 2021-05-31 2021-08-10 山东云海国创云计算装备产业创新中心有限公司 一种视频压缩方法及相关装置
WO2022044164A1 (fr) * 2020-08-26 2022-03-03 日本電気株式会社 Dispositif d'estimation de qualité vidéo, procédé d'estimation de qualité vidéo et système d'estimation de qualité vidéo
CN114827662A (zh) * 2022-03-18 2022-07-29 百果园技术(新加坡)有限公司 视频分辨率自适应调节方法、装置、设备和存储介质

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU740066B2 (en) * 1998-06-05 2001-10-25 Canon Kabushiki Kaisha Method and apparatus for hierarchical encoding or decoding video images
US20030169818A1 (en) * 2002-03-06 2003-09-11 Pere Obrador Video transcoder based joint video and still image pipeline with still burst mode
CN100584006C (zh) * 2006-06-20 2010-01-20 中兴通讯股份有限公司 一种基于时间分辨率变换的视频转换处理方法
US8199833B2 (en) * 2008-08-25 2012-06-12 Broadcom Corporation Time shift and tonal adjustment to support video quality adaptation and lost frames
GB2469281B (en) * 2009-04-06 2011-08-10 Motorola Inc Method and apparatus for asynchronous video transmission over a communication network
CN101616120B (zh) * 2009-07-28 2011-09-21 北京大学 一种子载波信噪比估计方法及系统
CN104539917A (zh) * 2015-02-03 2015-04-22 成都金本华科技股份有限公司 一种提高视频图像清晰度的方法
CN107079135B (zh) * 2016-01-29 2020-02-07 深圳市大疆创新科技有限公司 视频数据传输方法、系统、设备和拍摄装置
CN109936744B (zh) * 2017-12-19 2020-08-18 腾讯科技（深圳）有限公司 视频编码处理方法、装置及具有视频编码功能的应用
CN108596834B (zh) * 2018-04-27 2023-11-17 腾讯科技（深圳）有限公司 图像的分辨率处理方法、图像处理装置及系统、存储介质
CN110827380B (zh) * 2019-09-19 2023-10-17 北京铂石空间科技有限公司 图像的渲染方法、装置、电子设备及计算机可读介质
CN111263153B (zh) * 2020-02-20 2022-10-11 广州市百果园信息技术有限公司 视频编码方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103702061A (zh) * 2013-12-23 2014-04-02 上海网达软件股份有限公司 一种智能终端视频通话分辨率自动调节方法
CN111225240A (zh) * 2020-01-19 2020-06-02 安徽文香信息技术有限公司 自动调整占用带宽的方法、装置、存储介质与电子设备
WO2022044164A1 (fr) * 2020-08-26 2022-03-03 日本電気株式会社 Dispositif d'estimation de qualité vidéo, procédé d'estimation de qualité vidéo et système d'estimation de qualité vidéo
CN113038222A (zh) * 2021-03-08 2021-06-25 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及存储介质
CN113242434A (zh) * 2021-05-31 2021-08-10 山东云海国创云计算装备产业创新中心有限公司 一种视频压缩方法及相关装置
CN114827662A (zh) * 2022-03-18 2022-07-29 百果园技术(新加坡)有限公司 视频分辨率自适应调节方法、装置、设备和存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118337998A (zh) * 2024-06-17 2024-07-12 北京欣博电子科技有限公司 多路视频编解码的调度方法、装置、计算机设备及介质

Also Published As

Publication number Publication date
CN114827662A (zh) 2022-07-29
CN114827662B (zh) 2024-06-25

Similar Documents

Publication Publication Date Title
WO2023174216A1 (fr) Procédé, appareil et dispositif d'ajustement adaptatif de résolution vidéo, et support de stockage
WO2021244341A1 (fr) Procédé et appareil de codage d'image, dispositif électronique et support de stockage lisible par ordinateur
US20230393802A1 (en) Real-time screen sharing
WO2021068598A1 (fr) Procédé et dispositif de codage de partage d'écran, support de stockage et équipement électronique
US8928678B2 (en) Media workload scheduler
US10555002B2 (en) Long term reference picture coding
US8452120B2 (en) Image processing device and method
US12022096B2 (en) Human visual system adaptive video coding
US20140281017A1 (en) Jitter buffering system and method of jitter buffering
CN108681439B (zh) 基于帧率控制的均匀显示方法
US8233534B2 (en) Frame buffer compression and memory allocation in a video decoder
TWI715157B (zh) 保持穩定訊框速率之系統和方法
CN106412588B (zh) 影像帧处理方法
WO2021092821A1 (fr) Codage adaptatif de trames vidéo faisant appel à l'analyse de contenu et de réseau
CN108810468B (zh) 一种优化显示效果的视频传输装置及方法
CN112350998B (zh) 一种基于边缘计算的视频流传输方法
CN111405347B (zh) 画面显示方法、装置、电子设备及可读存储介质
CN110912922B (zh) 一种图像传输方法、装置、电子设备及存储介质
TW201842775A (zh) 用於在視訊編碼中的順延後處理的系統及方法
US11825088B2 (en) Adaptively encoding video frames based on complexity
CN110753229A (zh) 基于h.265编码的视频采集装置及方法
CN113192169B (zh) 动态调整LOD Scale的图像显示优化方法及装置
US20240357138A1 (en) Human visual system adaptive video coding
US10135896B1 (en) Systems and methods providing metadata for media streaming
CN116112740B (zh) 一种视频播放方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23769718

Country of ref document: EP

Kind code of ref document: A1