CN114245196A - Screen recording stream pushing method and device, electronic equipment and storage medium - Google Patents

Screen recording stream pushing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114245196A
CN114245196A CN202111512331.XA CN202111512331A CN114245196A CN 114245196 A CN114245196 A CN 114245196A CN 202111512331 A CN202111512331 A CN 202111512331A CN 114245196 A CN114245196 A CN 114245196A
Authority
CN
China
Prior art keywords
frame rate
frame
video
screen recording
preview
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111512331.XA
Other languages
Chinese (zh)
Other versions
CN114245196B (en
Inventor
张艳坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuomi Private Ltd
Original Assignee
Zhuomi Private Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuomi Private Ltd filed Critical Zhuomi Private Ltd
Priority to CN202111512331.XA priority Critical patent/CN114245196B/en
Publication of CN114245196A publication Critical patent/CN114245196A/en
Application granted granted Critical
Publication of CN114245196B publication Critical patent/CN114245196B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334—Recording operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The embodiment of the invention provides a screen recording stream pushing method, a screen recording stream pushing device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video; and coding the adjusted video to obtain a coded video, and pushing the coded video. By adopting the scheme, the preview frame rate of the screen recording video can be obtained, and then the frame rate adjustment is carried out on the screen recording video based on the preview frame rate and the preset frame rate, so that the frame rate of the screen recording video can be matched with the configured code rate, the occurrence of blurred pictures or cut-off in the live broadcasting process is reduced, the clear smoothness of live broadcasting pictures is ensured, and the live broadcasting effect of the screen recording is improved.

Description

Screen recording stream pushing method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a method and an apparatus for recording a screen and pushing a stream, an electronic device, and a storage medium.
Background
With the rapid development of computer and internet technologies, video live broadcasting becomes more and more popular, and screen recording live broadcasting is a very common and convenient live broadcasting mode. In the screen recording and live broadcasting process, a live broadcasting terminal collects screen display data, and then encodes and streams the collected screen data. The push stream is to transmit the encoded video data to the server, and the terminal equipment watching the live broadcast can watch the live broadcast of the main broadcast by pulling the stream from the server.
The frame rate of the screen display picture of the live broadcast terminal is controlled by an operating system, when the anchor operates the screen display picture, the display picture is changed greatly, the frame rate can reach the maximum, and when the screen display picture is not changed, the frame rate can be slowly reduced, even data is not generated to reduce power consumption.
In the current screen recording live broadcast mode, when the frame rate of a screen display picture of a live broadcast terminal is too high, the definition of the picture needs to be reduced in order to match with the set code rate, so that the pushed picture is blurred and even a blue screen is caused; when the frame rate of the screen display picture of the live broadcast terminal is too low, no video stream is pushed out, which may cause the user to cut off the stream when watching the live broadcast, so the screen recording and live broadcast effect is not ideal.
Disclosure of Invention
The embodiment of the invention aims to provide a screen recording stream pushing method, a screen recording stream pushing device, electronic equipment and a storage medium, so as to improve the screen recording live broadcast effect. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a screen recording stream pushing method, where the method includes:
acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate;
based on the size relation between the preview frame rate and a preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video;
and coding the adjusted video to obtain a coded video, and pushing the coded video.
Optionally, the preset frame rate includes a first target frame rate and a second target frame rate, where the first target frame rate is higher than the second target frame rate;
the step of adjusting the frame rate of the screen recording video based on the size relationship between the preview frame rate and the preset frame rate to obtain the adjusted video includes:
respectively determining the size relationship between the preview frame rate and the first target frame rate and the second target frame rate;
if the preview frame rate is greater than the first target frame rate, frame dropping processing is carried out on the screen recording video according to a preset frame dropping strategy to obtain an adjusted video;
and if the preview frame rate is less than the second target frame rate, performing frame supplementing processing on the screen recording video according to a preset frame supplementing strategy to obtain an adjusted video.
Optionally, the step of performing frame loss processing on the screen recording video according to a preset frame loss strategy to obtain an adjusted video includes:
determining a frame loss ratio for a current image frame included in the screen recording video based on the preview frame rate, the first target frame rate and a current adjustment parameter;
if the frame loss ratio is not less than the preset ratio and the number of the currently discarded image frames is less than a first target value, discarding the current image frame and adjusting the current adjusting parameter according to a first preset mode;
taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the frame loss ratio is smaller than the preset ratio or the number of the currently discarded image frames is not smaller than the first target value;
if the frame loss ratio is smaller than the preset ratio and the number of the currently reserved image frames is not smaller than a second target value, discarding the current image frames, setting the number of the currently reserved image frames to be 0, and adjusting the current adjusting parameters according to a second preset mode;
and taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjusting parameter until the current image frame is the last image frame of the screen recording video.
Optionally, the step of adjusting the current adjustment parameter according to a first preset manner includes:
calculating to obtain a difference value overshot based on the preview frame rate, the first target frame rate and the current adjusting parameter;
calculating to obtain the current adjusting parameter overshot modifier according to a formula overshot modifier- (previewFps% overshot)/3, wherein previewFps is the preview frame rate;
the step of adjusting the current adjustment parameter according to a second preset mode includes:
and calculating the current adjusting parameter according to a formula of overshot% output Fps, wherein the output Fps is the first target frame rate.
Optionally, the step of calculating a difference overshot based on the preview frame rate, the first target frame rate, and the current adjustment parameter includes:
according to the formula overshot modifier + (previous Fps-output putFps), the difference overshot is calculated.
Optionally, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference value.
Optionally, the step of performing frame supplementing processing on the screen recording video according to a preset frame supplementing strategy to obtain an adjusted video includes:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
In a second aspect, an embodiment of the present invention provides a screen recording stream pushing apparatus, where the apparatus includes:
the frame rate acquisition module is used for acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate;
the frame rate control module is used for adjusting the frame rate of the screen recording video based on the size relation between the preview frame rate and a preset frame rate to obtain an adjusted video;
and the coding and stream pushing module is used for coding the adjusted video to obtain a coded video and pushing the coded video.
Optionally, the preset frame rate includes a first target frame rate and a second target frame rate, where the first target frame rate is higher than the second target frame rate;
the frame rate control module includes:
a frame rate relation determining submodule, configured to determine a size relation between the preview frame rate and the first target frame rate and between the preview frame rate and the second target frame rate, respectively;
the frame dropping submodule is used for performing frame dropping processing on the screen recording video according to a preset frame dropping strategy to obtain an adjusted video if the preview frame rate is greater than the first target frame rate;
and the frame supplementing submodule is used for supplementing frames to the screen recording video according to a preset frame supplementing strategy if the preview frame rate is less than the second target frame rate to obtain the adjusted video.
Optionally, the frame loss sub-module includes:
a frame loss ratio determining unit, configured to determine, for a current image frame included in the screen recording video, a frame loss ratio based on the preview frame rate, the first target frame rate, and a current adjustment parameter;
a first frame dropping unit, configured to drop the current image frame and adjust the current adjustment parameter according to a first preset manner if the frame dropping rate is not less than a preset rate and the number of the currently dropped image frames is less than a first target value;
a first circulation unit, configured to use a frame next to the current image frame as a current image frame and trigger the frame loss ratio determination unit until the number of the currently discarded image frames is not less than the first target value if the frame loss ratio is less than the preset ratio;
a second frame dropping unit, configured to drop the current image frame, set the number of the current retained image frames to 0, and adjust the current adjustment parameter according to a second preset manner if the frame dropping ratio is smaller than the preset ratio and the number of the current retained image frames is not smaller than a second target value;
and the second circulating unit is used for taking the next frame of the current image frame as the current image frame and triggering the frame loss ratio determining unit until the current image frame is the last image frame of the screen recording video.
Optionally, the first frame loss unit includes:
a difference calculating subunit, configured to calculate a difference overshot based on the preview frame rate, the first target frame rate, and the current adjustment parameter;
a first adjusting parameter calculating subunit, configured to calculate, according to a formula overshot modifier- (previewFps% overshot)/3, to obtain the current adjusting parameter overshot modifier, where previewFps is the preview frame rate;
the second frame loss unit includes:
and a second adjustment parameter calculating subunit, configured to calculate the current adjustment parameter according to a formula, where output fps is the first target frame rate.
Optionally, the difference calculating subunit is configured to calculate the difference overshot according to a formula overshot modifier + (previous fps-output fps).
Optionally, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference value.
Optionally, the frame-complementing sub-module includes:
and the frame supplementing unit is used for taking the acquired last frame screen display picture as a supplementary video frame to obtain an adjusted video.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
a processor adapted to perform the method steps of any of the above first aspects when executing a program stored in the memory.
In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of any one of the above first aspects.
The embodiment of the invention has the following beneficial effects:
in the scheme provided by the embodiment of the invention, the electronic equipment can acquire the screen recording video and the frame rate of the screen display picture, and the frame rate is used as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video; and further, encoding the adjusted video to obtain an encoded video, and pushing the encoded video. By adopting the scheme, the preview frame rate of the screen recording video can be obtained, and then the frame rate adjustment is carried out on the screen recording video based on the preview frame rate and the preset frame rate, so that the frame rate of the screen recording video can be matched with the configured code rate, the occurrence of blurred pictures or cut-off in the live broadcasting process is reduced, the clear smoothness of live broadcasting pictures is ensured, and the live broadcasting effect of the screen recording is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.
Fig. 1 is a flowchart of a screen recording stream pushing method according to an embodiment of the present invention;
FIG. 2 is a flowchart based on step S102 in the embodiment shown in FIG. 1;
FIG. 3 is a specific flowchart based on step S202 in the embodiment shown in FIG. 2;
FIG. 4 is a diagram illustrating a frame loss result of a frame loss operation performed according to the embodiment shown in FIG. 3;
fig. 5 is a schematic diagram of an overall flow of screen recording stream pushing according to the embodiment shown in fig. 1;
fig. 6 is a schematic structural diagram of a screen recording stream pushing device according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a specific structure of a frame loss sub-module according to the embodiment shown in fig. 6;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention are within the scope of the present invention.
In order to reduce the occurrence of blurred images or cut-off in the live video recording process to ensure the clarity and smoothness of the live video, embodiments of the present invention provide a method and an apparatus for video recording and stream pushing, an electronic device, a computer-readable storage medium, and a computer program product.
The screen recording and stream pushing method provided by the embodiment of the invention can be applied to any electronic equipment needing screen recording and stream pushing, can acquire the screen recording video and the frame rate of the screen display picture of the live broadcast terminal, and can encode and then push the acquired screen recording video, for example, the screen recording and stream pushing method can be used for a server corresponding to the live broadcast terminal. For clarity of description, the electronic device is referred to hereinafter.
As shown in fig. 1, a screen recording and stream pushing method includes:
s101, acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate.
And S102, adjusting the frame rate of the screen recording video based on the size relation between the preview frame rate and a preset frame rate to obtain an adjusted video.
S103, encoding the adjusted video to obtain an encoded video, and pushing the encoded video.
Therefore, in the scheme provided by the embodiment of the invention, the electronic equipment can acquire the screen recording video and the frame rate of the screen display picture, and the frame rate is used as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video; and further, encoding the adjusted video to obtain an encoded video, and pushing the encoded video. By adopting the scheme, the preview frame rate of the screen recording video can be obtained, and then the frame rate adjustment is carried out on the screen recording video based on the preview frame rate and the preset frame rate, so that the frame rate of the screen recording video can be matched with the configured code rate, the occurrence of blurred pictures or cut-off in the live broadcasting process is reduced, the clear smoothness of live broadcasting pictures is ensured, and the live broadcasting effect of the screen recording is improved.
In order to reduce the blurring or the cutoff of pictures in the screen recording live broadcasting process, the frame rate of the screen recording video needs to be matched with the set code rate. The frame rate is the frequency of continuous appearance of bitmap images in units of frames on a display screen, and the frame rate in the embodiment of the invention is the frequency of continuous appearance of image frames included in a screen recording video on the display screen, and the higher the frame rate is, the smoother the picture of the screen recording video is. The code rate is the bit rate which represents the number of transmitted bits in unit time, the code rate in the embodiment of the invention is the number of transmitted bits per second of the encoder, and the higher the code rate is, the faster the speed of transmitting the bits by the encoder is. For example, the code rate is 800kbps, meaning that the encoder can transmit 800 kilobits per second.
In the process of screen recording and live broadcasting, the code rate is usually fixed and unchanged, and if the frame rate of a screen recording video is too high, the screen recording video can be normally encoded only by reducing the definition of a picture; if the frame rate of the screen recording video is too low, no video stream is pushed out, so that the electronic equipment can adjust the frame rate of the screen recording video to ensure that a live broadcast picture is clear and smooth.
In the step S101, the electronic device may acquire the screen recording video and the frame rate of the screen display, and use the frame rate as the preview frame rate.
The electronic equipment can record the screen display picture of the live broadcast terminal, so that a screen recording video is obtained. The frame rate of the screen display picture is controlled by an operating system, the operating system can change the frame rate according to the operation of the anchor on the screen display picture, and when the frame rate of the screen display picture is changed, the frame rate of the screen recording video is changed accordingly. In order to accurately obtain the frame rate of the screen-recording video, a time period may be preset, for example, the time period may be 2 seconds, 3 seconds, 5 seconds, and the like, which is not limited herein. The electronic device may obtain the number of image frames included in the screen recording video in the time period, and calculate a quotient of the number of image frames and a preset time period as a preview frame rate of the screen recording video.
For example, the preset time period is 3 seconds, the electronic device obtains the number of image frames included in the screen recording video once in each time period, and the number of image frames obtained by the electronic device in the current time period is 90, so that the preview frame rate of the screen recording video in the current time period is set to be
Figure BDA0003398936560000081
I.e. 30 frames per second.
After the screen recording video and the preview frame rate of the screen recording video are obtained, the electronic device may adjust the frame rate of the screen recording video based on the size relationship between the preview frame rate and the preset frame rate to obtain an adjusted video, that is, execute step S102.
The obtained preview frame rates of the screen recording videos in all time periods are possibly different, the highest preview frame rate can reach the maximum value of the screen display frame rate of the live broadcast terminal, and the minimum preview frame rate can reach 0; therefore, for each time period, the electronic equipment can adjust the frame rate of the screen recording video to control the frame rate of the screen recording video, so that the screen recording video can be encoded and continuously pushed without changing the definition. The electronic equipment can control the frame rate of the screen recording video within a certain frame rate range, and within the frame rate range, the screen recording video cannot cause the blurring of live broadcast pictures due to too high frame rate, and cannot cause live broadcast cut-off due to too low frame rate.
In one embodiment, a frame rate range may be preset, and when the frame rate of the screen recording video is within the frame rate range, the screen recording video may be encoded and continuously streamed without changing the definition. The electronic equipment can judge whether the preview frame rate of the screen recording video is within the preset frame rate range, if the preview frame rate is larger than the maximum value of the preset frame rate range, the preview frame rate is too high, a live broadcast picture may be blurred, and the electronic equipment can perform frame dropping operation on the screen recording video, so that the frame rate of the video after adjustment is within the preset frame rate range. If the preview frame rate is smaller than the minimum value of the preset frame rate range, it is indicated that the preview frame rate is too low, live broadcast may be cut off, and the electronic device may perform frame supplementing operation on the screen recording video, so that the frame rate of the video after adjustment is within the preset frame rate range. The frame rate of the video recorded on the screen is adjusted, the electronic equipment obtains the adjusted video, the frame rate of the adjusted video is matched with the set code rate, and encoding and continuous stream pushing can be carried out without changing the definition.
Next, in step S103, the electronic device may encode the adjusted video to obtain an encoded video, and perform stream pushing on the encoded video.
The frame rate of the adjusted video is matched with the set code rate, and the electronic equipment can encode the adjusted video. The video coding is to convert the adjusted video into a file in another video format by a compression technique, for example, the coding algorithm may be H261, H263, H264, etc., and is not limited in detail here. The data volume of the coded video is reduced, the occupation of a CPU and a memory is reduced, and the transmission is more convenient.
The electronic device may perform plug-streaming on the encoded video, which is a process of transmitting the encoded video to a server. The Protocol of the video stream may be RTMP (Real Time Messaging Protocol), and the like, which is not limited herein.
By adopting the scheme provided by the embodiment of the invention, the electronic equipment can adjust the frame rate of the screen recording video based on the preview frame rate and the preset frame rate of the screen recording video, encode the adjusted video and push the encoded video. The adjusted video frame rate is matched with the set code rate, the occurrence of picture blurring or cut-off in the live broadcasting process is reduced, and the cut-off is not generated, so that the success rate of a user entering a live broadcasting room is effectively improved; the method has the advantages that the blurring of the picture is reduced, the clarity and smoothness of the live broadcast picture are guaranteed, a user can better interact with the anchor, the reserved data is higher, and the screen recording and live broadcast effects are improved.
As an implementation manner of the embodiment of the present invention, as shown in fig. 2, the preset frame rate may include a first target frame rate and a second target frame rate, where the first target frame rate is higher than the second target frame rate.
In one embodiment, the first target frame rate is a maximum frame rate at which the adjusted video can be encoded and continuously streamed without changing the definition, and the second target frame rate is a minimum frame rate at which the adjusted video can be encoded and continuously streamed without changing the definition. When the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, encoding and continuous plug flow can be performed without changing the definition.
Correspondingly, the step of adjusting the frame rate of the screen recording video based on the size relationship between the preview frame rate and the preset frame rate to obtain the adjusted video may include:
s201, respectively determining the size relationship between the preview frame rate and the first target frame rate and the second target frame rate; if the preview frame rate is greater than the first target frame rate, executing step S202; if the preview frame rate is less than the second target frame rate, step S203 is executed.
After the electronic device obtains the video recorded on the screen and the preview frame rate, the size relationship between the preview frame rate and the first target frame rate and the size relationship between the preview frame rate and the second target frame rate can be respectively determined. If the preview frame rate of the screen recording video is greater than the first target frame rate, it indicates that the preview frame rate of the screen recording video is too high, and if the frame rate of the screen recording video is not adjusted, the live broadcast picture may be blurred or even blue screen, so that frame dropping processing needs to be performed on the screen recording video, and step S202 may be executed.
If the preview frame rate of the screen recording video is less than the second target frame rate, it indicates that the preview frame rate of the screen recording video is too low, and if the screen recording video is not adjusted, the screen recording live broadcast may be interrupted, so that the frame supplementing processing needs to be performed on the screen recording video, and step S203 can be executed.
If the preview frame rate of the screen recording video is not greater than the first target frame rate and not less than the second target frame rate, the preview frame rate of the screen recording video is neither too high nor too low, encoding and continuous stream pushing can be performed without changing the definition, the preview frame rate of the screen recording video does not affect a live broadcast picture, and frame rate adjustment is not required.
S202, frame dropping processing is carried out on the screen recording video according to a preset frame dropping strategy, and the adjusted video is obtained.
When the preview frame rate of the screen recording video is greater than the first target frame rate, the electronic equipment can perform frame dropping processing on the screen recording video according to a preset frame dropping strategy, and the frame dropping processing can drop partial image frames included in the screen recording video, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, the adjusted video can be encoded and continuously pushed without changing the definition, and a live broadcast picture can not generate a fuzzy or blue screen.
And S203, performing frame supplementing processing on the screen recording video according to a preset frame supplementing strategy to obtain an adjusted video.
When the preview frame rate of the screen recording video is less than the second target frame rate, the electronic device may perform frame supplementing processing on the screen recording video according to a preset frame supplementing policy. In an embodiment, the frame supplementing processing may be to obtain a last frame of screen display picture as a supplementary video frame, and add the supplementary video frame to the screen recording video, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, the adjusted video may be encoded and continuously streamed without changing the definition, and no cut-off phenomenon may occur in the screen recording live broadcast.
In this embodiment, based on the preset first target frame rate and the second target frame rate, the electronic device may determine whether frame loss processing or frame supplement processing needs to be performed on the video recorded on the screen, and if adjustment is needed, the processing may be performed according to a corresponding policy, and the frame rate of the video after adjustment is not greater than the first target frame rate and not less than the second target frame rate. Therefore, the frame rate of the adjusted video reaches the preset frame rate range, the video can be coded and continuously pushed without changing the definition, and in the process of screen recording and live broadcasting, the live broadcasting picture cannot generate a fuzzy or blue screen, and the live broadcasting cannot generate a cut-off phenomenon.
As an implementation manner of the embodiment of the present invention, as shown in fig. 3, the step of performing frame dropping processing on the video on the screen recording according to the preset frame dropping policy to obtain an adjusted video may include:
s301, aiming at a current image frame included in the screen recording video, determining a frame loss ratio based on the preview frame rate, the first target frame rate and a current adjusting parameter; if the frame loss ratio is not less than the preset ratio and the number of the currently discarded image frames is less than the first target value, executing step S302; if the frame loss ratio is less than the preset ratio and the number of the currently reserved image frames is not less than the second target value, step S304 is performed.
When the preview frame rate of the screen recording video is greater than the first target frame rate, the electronic device needs to perform frame dropping processing on the screen recording video. If the preview frame rate is much greater than the first target frame rate, it means that the number of image frames that need to be discarded when the electronic device performs frame dropping processing on the video recorded on the screen is large, and the frame rate of the video after adjustment is not greater than the first target frame rate and not less than the second target frame rate, so a frame dropping strategy that can quickly drop a large number of image frames is needed. If the difference value between the preview frame rate and the first target frame rate is small, it indicates that the number of image frames that need to be discarded when the electronic device performs frame dropping processing on the video recorded on the screen is small, and the frame rate of the video after adjustment can be made not greater than the first target frame rate and not less than the second target frame rate, so a frame dropping strategy capable of maintaining the smoothness of the image as much as possible is needed.
The electronic device may determine a frame loss ratio based on the preview frame rate, the first target frame rate, and the current adjustment parameter. The frame loss ratio represents the ratio of image frames which need to be discarded in the screen recording video, and the higher the frame loss ratio is, the larger the number of image frames which need to be discarded is. The electronic device can determine whether discarding is necessary for all image frames included in the video on screen one by one.
For a current image frame included in the screen recording video, if the frame dropping rate is not less than the preset rate, it indicates that the number of image frames that need to be dropped currently is larger, and if the number of image frames that have been dropped currently is less than the first target value, it indicates that the number of image frames that have been dropped is still smaller, so the electronic device may execute step S302 to continue dropping image frames. If the frame dropping rate is less than the preset rate, it indicates that the number of image frames that need to be dropped currently is small, and if the number of image frames that have been retained currently is not less than the second target value, it indicates that the number of image frames that have been retained currently is large, the electronic device may execute step S304 to meet the requirement of dropping a small number of image frames.
The initial value of the current adjusting parameter can be preset, in the frame loss processing process, the current adjusting parameter is calculated after the last frame loss is completed, and the current adjusting parameter can be continuously changed in the whole frame loss processing process, so that the frame loss ratio is adjusted, and the frame loss strategy is changed to ensure that the number of the discarded image frames meets the actual requirement.
In one embodiment, the electronic device may calculate a difference between the preview frame rate and the first target frame rate, calculate a sum of the difference and the current adjustment parameter as a frame rate difference, and calculate a quotient of the frame rate difference and the preview frame rate as a frame loss ratio.
The preset ratio may be set according to requirements such as adjusting the frame rate of the screen recording video and maintaining the fluency of the picture, for example, the preset ratio may be 0.5. Then, when the frame dropping rate is not less than 0.5, it means that the number of image frames that need to be dropped in the image frames included in the video recording is greater than the number of image frames that need to be retained, that is, it may be necessary to continuously drop a plurality of image frames, and then the electronic device may continuously perform the frame dropping operation when the number of image frames that have been currently dropped is less than the first target value. When the frame dropping rate is less than 0.5, it is indicated that the number of image frames to be dropped in the image frames included in the video recording is less than the number of image frames to be retained, and in order to ensure that the display image of the video recording is smooth and stable, continuous frame dropping is not performed, that is, at most one image frame is continuously dropped, so that the electronic device may drop one image frame when the number of retained image frames is not less than the second target value.
A schematic diagram of the frame loss result can be shown in fig. 4, where dropatio represents the frame loss ratio, "F" represents the image frame remaining in the screen-recording video, and "x" represents the image frame dropped in the screen-recording video. It can be seen that when the frame dropping rate is not less than 0.5, a plurality of image frames may be dropped continuously, after the plurality of image frames are dropped, the frame rate of the adjusted video may not be greater than the first target frame rate and not less than the second target frame rate, and the electronic device may stop the frame dropping operation, so that a continuous reserved image frame "F" appears in the screen recording video. When the frame loss ratio is less than 0.5, the frame rate of the adjusted video can be enabled not to be more than the first target frame rate and not to be less than the second target frame rate without continuous frame loss.
S302, discarding the current image frame, and adjusting the current adjusting parameter according to a first preset mode.
When the frame loss ratio is not less than the preset ratio and the number of the currently discarded image frames is less than the first target value, the electronic device may discard the current image frame. The number of discarded image frames is increased by 1, and the number of image frames to be discarded is decreased, so that the electronic device may adjust the current adjustment parameter according to the first preset manner to change the frame loss ratio. Next, the electronic device may continue to perform step S303.
S303, using the next frame of the current image frame as the current image frame, and returning to step S301 until the frame loss ratio is smaller than the preset ratio, or the number of the currently discarded image frames is not smaller than the first target value.
After the electronic device completes the discarding operation of the image frame each time, the next frame of the current image frame may be used as the current image frame, and the process returns to step S301, so as to recalculate the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter, and determine whether the current image frame needs to be discarded. Until the frame loss ratio is less than the preset ratio, or the number of the currently discarded image frames is not less than the first target value.
If the frame loss ratio is smaller than the preset ratio, the video recording screen only needs to discard a small number of image frames at the moment, so the electronic equipment can perform frame loss operation according to another frame loss strategy; if the number of the currently discarded image frames is not less than the first target value, it indicates that the number of the currently discarded image frames is large, and the frame rate adjustment requirement can be met, and the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, and the electronic device may stop the frame dropping operation.
S304, discarding the current image frame, setting the number of the current reserved image frames to be 0, and adjusting the current adjusting parameter according to a second preset mode;
if the number of the image frames which are reserved currently is less than the second target value, the electronic device can reserve the current image frames.
If the number of the image frames that have been currently reserved is not less than the second target value, which indicates that the number of the image frames that have been reserved at this time is greater, the electronic device may discard the current image frame in order to meet the frame dropping requirement. In order to ensure smooth pictures and avoid continuous frame dropping as much as possible, the electronic device may set the number of currently reserved image frames to 0 to ensure that continuous frame dropping is not performed.
The number of discarded image frames is increased by 1, and the number of image frames to be discarded is decreased, so that the electronic device may adjust the current adjustment parameter according to the second preset manner to change the frame loss ratio. Next, the electronic device may continue to perform step S305.
S305, taking the next frame of the current image frame as the current image frame, and returning to the step S301 until the current image frame is the last image frame of the screen recording video.
After the electronic device finishes the operation of retaining or discarding the current image frame each time, the next frame of the current image frame may be used as the current image frame, and the step S301 is returned to, so as to recalculate the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter, and determine whether the current image frame needs to be discarded. And ending the frame loss processing until the current image frame is the last image frame of the screen recording video.
In this embodiment, the electronic device may determine a frame loss ratio for a current image frame included in the screen recording video, and based on a size relationship between the frame loss ratio and a preset ratio, the electronic device may perform frame loss processing on the screen recording video according to two frame loss strategies. When the frame loss ratio is large, the electronic equipment can continuously discard image frames, when the frame loss ratio is small, the electronic equipment can discard a small number of image frames, and the obtained adjusted video can be encoded and continuously pushed without changing the definition. Therefore, the efficiency of the electronic equipment for completing the frame dropping operation can be improved, and the picture smoothness of the screen recording video cannot be influenced. In the live screen recording process, the occupation of a CPU and an internal memory can be reduced by reducing the frame rate of the screen recording video, the occurrence probability of blurred pictures or blue screens is reduced, the watching experience of a user is improved, the user can better interact with a main broadcast, and the reserved data is effectively improved.
As an implementation manner of the embodiment of the present invention, the step of adjusting the current adjustment parameter according to the first preset manner may include:
calculating to obtain a difference value overshot based on the preview frame rate, the first target frame rate and the current adjusting parameter; and calculating the current adjusting parameter overshot modifier according to a formula of overshot modifier- (previewFps% overshot)/3.
In order to adjust the current adjustment parameter and further change the frame loss ratio, the electronic device may calculate a difference overshot based on the preview frame rate, the first target frame rate, and the current adjustment parameter, where the difference overshot represents the number of image frames that need to be dropped in the screen recording video.
After the electronic device calculates the difference overshot, it may calculate a current adjustment parameter overshot modifier according to a formula overshot modifier- (previewFps% overshot)/3, where the current adjustment parameter may reduce the frame loss ratio, thereby changing the frame loss strategy and the number of discarded image frames. Wherein previewFps is the preview frame rate.
Correspondingly, the step of adjusting the current adjustment parameter according to the second preset manner may include:
and calculating the current adjusting parameter according to a formula of overshot% output Fps.
After the electronic device calculates the difference overshot, the current adjustment parameter may be calculated according to a formula overshot modifier ═ overshot% outputFps, where outputFps is the first target frame rate. The current adjustment parameter may adjust the frame dropping rate to change the number of image frames dropped until the current image frame is the last image frame of the video on-screen.
In this embodiment, in different frame dropping strategies, the electronic device may adjust the current adjustment parameter according to the first preset mode or the second preset mode, and adjust the frame dropping ratio, so that the change of the frame dropping strategy and the number of dropped image frames may be accurately controlled.
As an implementation manner of the embodiment of the present invention, the step of calculating a difference overshot based on the preview frame rate, the first target frame rate, and the current adjustment parameter may include:
according to the formula overshot modifier + (previous Fps-output putFps), the difference overshot is calculated.
After the electronic device performs frame dropping processing on the current image frame, the current adjustment parameter may be adjusted according to a first preset mode or a second preset mode, and a next frame of the current image frame is used as the current image frame, at this time, the electronic device needs to recalculate a frame dropping rate to determine a frame dropping strategy. The electronic device may calculate the difference overshoot according to a formula overshoot modifier + (previous fps-output fps).
Different frame loss strategies can obtain different current adjusting parameters, and the difference value can also be changed differently. The current adjusting parameter obtained by the frame dropping strategy for quickly dropping a large number of image frames can reduce the difference value and reduce the frame dropping rate, thereby changing the frame dropping strategy and the number of dropped image frames. The difference value can be increased by the current adjusting parameter obtained by the frame dropping strategy which can keep the fluency of the picture as much as possible, and the frame dropping rate is adjusted, thereby changing the number of dropped image frames.
In this embodiment, the electronic device may accurately calculate the difference overshot, thereby determining the number of image frames that need to be discarded in the screen recording video, and may accurately calculate the frame loss ratio based on the difference overshot, after the operation of discarding the image frames is completed, the electronic device may also accurately adjust the current adjustment parameters, and the accuracy of the frame loss processing of the screen recording video is improved.
As an implementation manner of the embodiment of the present invention, the first target value may be a quotient of the difference value and the first target frame rate; the second target value may be a quotient of the preview frame rate and the difference.
In the frame dropping strategy for rapidly dropping a large number of image frames, the difference value is reduced after each image frame dropping, the first target frame rate is determined, the first target value is continuously reduced, and the frame dropping rate is also continuously reduced. Then as the number of the currently discarded image frames increases, there occurs a case where the frame loss ratio is smaller than the preset ratio or the number of the currently discarded image frames is not smaller than the first target value, thereby changing the frame loss strategy and the number of the discarded image frames.
In the frame dropping strategy capable of keeping the fluency of the picture as much as possible, the difference value is increased after each image frame is dropped, the preview frame rate is determined, the second target value is continuously reduced, and the electronic equipment can adjust the frame dropping rate by comparing the number of the current reserved image frames with the second target value, so as to control the number of the reserved image frames.
In this embodiment, the electronic device may accurately calculate the first target value, thereby controlling the number of discarded image frames in the frame loss processing process; the second target value is accurately calculated, the electronic equipment can control the number of the image frames reserved in the frame loss processing process, and the accuracy of the frame loss processing of the screen recording video is improved.
As an implementation manner of the embodiment of the present invention, the step of performing frame supplementing processing on the screen recording video according to a preset frame supplementing policy to obtain an adjusted video may include:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
If the preview frame rate of the screen recording video is less than the second target frame rate, the preview frame rate of the screen recording video is too low, and if the frame rate of the screen recording video is not adjusted, no video stream is pushed out in the stream pushing process, so that the problem of flow break occurs. Therefore, the electronic device can perform frame supplementing processing on the screen recording video when the preview frame rate is less than the second target frame rate.
As an embodiment, in a time period in which the last preview frame rate is not less than the second target frame rate, the last image frame included in the screen recording video may be used as the last frame screen display. The electronic device can acquire the last frame of screen display picture as a supplementary video frame, and further add the supplementary video frame into the screen recording video, so that the frame rate of the screen recording video is not greater than the first target frame rate and not less than the second target frame rate, and the adjusted video is obtained.
For example, the second target frame rate is 1, the first target frame rate is 30, and if the preview frame rate of the screen recording video is less than 1, it indicates that the frame rate of the screen recording video is 0, and no video stream is pushed out in the process of pushing stream, thereby causing stream interruption. At this time, the electronic device may obtain a screen display picture of the last frame, and compose the image frame into a screen recording video according to the number of 15 frames per second, where the frame rate of the screen recording video is smaller than 30 and greater than 1, and may perform encoding and continuous streaming. When the preview frame rate of the screen recording video is not less than 1, the electronic device may stop performing frame supplementing processing on the screen recording video, and perform frame rate adjustment on the screen recording video based on the size relationship between the preview frame rate and the preset frame rate.
In this embodiment, the electronic device performs frame supplementing processing on the screen recording video with the preview frame rate less than the second target frame rate, so that the frame rate of the adjusted video is not greater than the first target frame rate and not less than the second target frame rate, and encoding and continuous streaming can be performed. Therefore, in the process of screen recording and live broadcasting, the stream pushing end cannot push out a video stream, so that the stream breaking cannot be generated, a user enters a live broadcasting room, and can see the screen picture of a main broadcast during stream pulling, and the success rate of entering the live broadcasting room is effectively improved.
The following describes, by way of example, a flow of screen recording and stream pushing provided in an embodiment of the present invention with reference to fig. 5, which may include screen data acquisition, frame rate control, video encoding, and stream pushing processes, and as shown in fig. 5, the screen recording and stream pushing process in the Android system may specifically include:
s501, collecting screen data.
The Imagereader is an image reader, can call a function get Surface () to provide a Surface layer Surface, the mediaproject Manager is a media projection Manager, the mediaproject Manager can call the function get mediaproject () to obtain a media projection mediaproject, the media projection can create a virtual display VirtualDisplay based on the Surface layer and the function createVirtualDisplay (), and then generate data, namely a screen recording video, and the virtual display can send the generated data to the image reader through the Surface layer. Therefore, the electronic equipment can acquire the screen recording video of the screen display picture.
And S502, adjusting the frame rate.
The electronic device can adjust the frame rate of the screen recording video based on the size relation between the preview frame rate of the screen recording video and the preset frame rate to obtain the adjusted video.
And S503, video coding.
The electronic device may encode the adjusted video to obtain an encoded video.
And S504, pushing flow.
The electronic device may stream the encoded video onto a server.
In this embodiment, the electronic device may obtain the screen recording video of the screen display image through virtual display, perform frame rate adjustment on the screen recording video, and encode and stream the adjusted video onto the server. When the screen frame rate is too high, a part of image frames can be dropped according to a frame dropping strategy, the condition that the recorded screen video is not blurred is ensured, the watching experience of a user is improved, and the occupation of a CPU and an internal memory can be reduced. When the screen frame rate is too low, a certain number of image frames can be supplemented according to the last frame of screen display picture, the condition of current cutoff is avoided, and the success rate of a user entering a live broadcast room can be effectively improved.
Corresponding to the screen recording stream pushing method, an embodiment of the present invention further provides a screen recording stream pushing device, and the following introduces a screen recording stream pushing device provided by an embodiment of the present invention.
As shown in fig. 6, a screen recording and stream pushing device includes:
the frame rate obtaining module 601 is configured to obtain a screen recording video and a frame rate of a screen display image, and use the frame rate as a preview frame rate.
A frame rate control module 602, configured to perform frame rate adjustment on the screen recording video based on a size relationship between the preview frame rate and a preset frame rate, so as to obtain an adjusted video.
And an encoding and stream pushing module 603, configured to encode the adjusted video to obtain an encoded video, and push stream the encoded video.
Therefore, in the scheme provided by the embodiment of the invention, the electronic equipment can acquire the screen recording video and the frame rate of the screen display picture, and the frame rate is used as the preview frame rate; based on the size relation between the preview frame rate and the preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video; and further, encoding the adjusted video to obtain an encoded video, and pushing the encoded video. By adopting the scheme, the preview frame rate of the screen recording video can be obtained, and then based on the preview frame rate and the preset frame rate, the frame rate adjustment is carried out on the screen recording video, so that the frame rate of the screen recording video can be matched with the configured code rate, the occurrence of fuzzy pictures or cut-off in the live broadcast process is reduced, the clear smoothness of live broadcast pictures is ensured, and the live broadcast effect of the screen recording is improved.
As an implementation manner of the embodiment of the present invention, the preset frame rate includes a first target frame rate and a second target frame rate, and the first target frame rate is higher than the second target frame rate;
the frame rate control module 602 may include:
and the frame rate relation determining submodule is used for respectively determining the size relation between the preview frame rate and the first target frame rate and the second target frame rate.
And the frame dropping submodule is used for performing frame dropping processing on the screen recording video according to a preset frame dropping strategy to obtain an adjusted video if the preview frame rate is greater than the first target frame rate.
And the frame supplementing submodule is used for supplementing frames to the screen recording video according to a preset frame supplementing strategy if the preview frame rate is less than the second target frame rate to obtain the adjusted video.
As an implementation manner of the embodiment of the present invention, as shown in fig. 7, the frame loss sub-module may include:
a frame loss ratio determining unit 701, configured to determine, for a current image frame included in the video recording, a frame loss ratio based on the preview frame rate, the first target frame rate, and a current adjustment parameter.
A first frame dropping unit 702, configured to drop the current image frame and adjust the current adjustment parameter according to a first preset manner if the frame dropping ratio is not less than a preset ratio and the number of currently dropped image frames is less than a first target value.
A first loop unit 703, configured to use a frame next to the current image frame as the current image frame and trigger the frame loss ratio determination unit 701 until the number of the currently discarded image frames is not less than the first target value if the frame loss ratio is less than the preset ratio.
A second frame dropping unit 704, configured to drop the current image frame, set the number of the current retained image frames to 0, and adjust the current adjustment parameter according to a second preset manner, if the frame dropping ratio is smaller than the preset ratio and the number of the current retained image frames is not smaller than a second target value.
A second loop unit 705, configured to use a next frame of the current image frame as a current image frame, and trigger the frame loss ratio determination unit 701 until the current image frame is a last image frame of the screen recording video.
As an implementation manner of the embodiment of the present invention, the first frame loss unit 702 may include:
and the difference value calculating subunit is configured to calculate a difference value overlaershot based on the preview frame rate, the first target frame rate, and the current adjustment parameter.
A first adjusting parameter calculating subunit, configured to calculate, according to a formula, an overshot modifier ═ - (previewFps% overshot)/3, to obtain the current adjusting parameter overshot modifier;
wherein previewFps is the preview frame rate.
The second frame loss unit 704 may include:
a second adjustment parameter calculating subunit, configured to calculate the current adjustment parameter according to a formula overshot modifier ═ overshot% output popps;
wherein the outputFps is the first target frame rate.
As an implementation manner of the embodiment of the present invention, the difference operator unit may be configured to calculate the difference overshot according to a formula overshot ═ overshot modifier + (previous fps-output fps).
As an implementation manner of the embodiment of the present invention, the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference value.
As an implementation manner of the embodiment of the present invention, the frame-complementing sub-module may include:
and the frame supplementing unit is used for taking the acquired last frame screen display picture as a supplementary video frame to obtain an adjusted video.
An embodiment of the present invention further provides an electronic device, as shown in fig. 8, which includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete mutual communication through the communication bus 804,
a memory 803 for storing a computer program;
the processor 801 is configured to implement the method steps according to any of the above embodiments when executing the program stored in the memory 803.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In a further embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, which, when being executed by a processor, implements the steps of the method of any of the above embodiments.
In a further embodiment provided by the present invention, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method steps of any of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, the electronic device, the computer-readable storage medium, and the computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (10)

1. A screen recording and stream pushing method is characterized by comprising the following steps:
acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate;
based on the size relation between the preview frame rate and a preset frame rate, performing frame rate adjustment on the screen recording video to obtain an adjusted video;
and coding the adjusted video to obtain a coded video, and pushing the coded video.
2. The method of claim 1, wherein the preset frame rate comprises a first target frame rate and a second target frame rate, and wherein the first target frame rate is higher than the second target frame rate;
the step of adjusting the frame rate of the screen recording video based on the size relationship between the preview frame rate and the preset frame rate to obtain the adjusted video includes:
respectively determining the size relationship between the preview frame rate and the first target frame rate and the second target frame rate;
if the preview frame rate is greater than the first target frame rate, frame dropping processing is carried out on the screen recording video according to a preset frame dropping strategy to obtain an adjusted video;
and if the preview frame rate is less than the second target frame rate, performing frame supplementing processing on the screen recording video according to a preset frame supplementing strategy to obtain an adjusted video.
3. The method of claim 2, wherein the step of performing frame loss processing on the video on the screen recording according to a preset frame loss strategy to obtain an adjusted video comprises:
determining a frame loss ratio for a current image frame included in the screen recording video based on the preview frame rate, the first target frame rate and a current adjustment parameter;
if the frame loss ratio is not less than the preset ratio and the number of the currently discarded image frames is less than a first target value, discarding the current image frame and adjusting the current adjusting parameter according to a first preset mode;
taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjustment parameter until the frame loss ratio is smaller than the preset ratio or the number of the currently discarded image frames is not smaller than the first target value;
if the frame loss ratio is smaller than the preset ratio and the number of the currently reserved image frames is not smaller than a second target value, discarding the current image frames, setting the number of the currently reserved image frames to be 0, and adjusting the current adjusting parameters according to a second preset mode;
and taking the next frame of the current image frame as the current image frame, and returning to the step of determining the frame loss ratio based on the preview frame rate, the first target frame rate and the current adjusting parameter until the current image frame is the last image frame of the screen recording video.
4. The method of claim 3, wherein said step of adjusting said current adjustment parameter in a first predetermined manner comprises:
calculating to obtain a difference value overshot based on the preview frame rate, the first target frame rate and the current adjusting parameter;
calculating to obtain the current adjusting parameter overshot modifier according to a formula overshot modifier- (previewFps% overshot)/3, wherein previewFps is the preview frame rate;
the step of adjusting the current adjustment parameter according to a second preset mode includes:
and calculating the current adjusting parameter according to a formula of overshot% output Fps, wherein the output Fps is the first target frame rate.
5. The method of claim 4, wherein the step of calculating a difference overshot based on the preview frame rate, the first target frame rate and the current adjustment parameter comprises:
according to the formula overshot modifier + (previous Fps-output putFps), the difference overshot is calculated.
6. The method of claim 5, wherein the first target value is a quotient of the difference value and the first target frame rate; the second target value is a quotient of the preview frame rate and the difference value.
7. The method according to any one of claims 1 to 6, wherein the step of performing frame supplement processing on the screen recording video according to a preset frame supplement strategy to obtain an adjusted video comprises:
and taking the acquired last frame of screen display picture as a supplementary video frame to obtain an adjusted video.
8. A screen recording and stream pushing device, the device comprising:
the frame rate acquisition module is used for acquiring a screen recording video and a frame rate of a screen display picture, and taking the frame rate as a preview frame rate;
the frame rate control module is used for adjusting the frame rate of the screen recording video based on the size relation between the preview frame rate and a preset frame rate to obtain an adjusted video;
and the coding and stream pushing module is used for coding the adjusted video to obtain a coded video and pushing the coded video.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 7 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
CN202111512331.XA 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium Active CN114245196B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111512331.XA CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111512331.XA CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114245196A true CN114245196A (en) 2022-03-25
CN114245196B CN114245196B (en) 2024-04-19

Family

ID=80754884

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111512331.XA Active CN114245196B (en) 2021-12-08 2021-12-08 Screen recording and stream pushing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114245196B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116996639A (en) * 2023-02-13 2023-11-03 深圳Tcl新技术有限公司 Screen-projection frame rate acquisition method and device, computer equipment and storage medium
WO2024022427A1 (en) * 2022-07-28 2024-02-01 北京字跳网络技术有限公司 Video recording method and apparatus, device, storage medium, and program product

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060087585A1 (en) * 2004-10-26 2006-04-27 Samsung Electronics Co., Ltd. Apparatus and method for processing an image signal in a digital broadcast receiver
CN108900919A (en) * 2018-06-29 2018-11-27 乐蜜有限公司 Company's wheat method, apparatus, server and terminal in a kind of live streaming
CN109089131A (en) * 2018-09-21 2018-12-25 广州虎牙信息科技有限公司 A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system
CN110912922A (en) * 2019-12-03 2020-03-24 锐捷网络股份有限公司 Image transmission method and device, electronic equipment and storage medium
CN111641858A (en) * 2020-04-29 2020-09-08 上海推乐信息技术服务有限公司 Audio and video synchronization method and system
CN111918093A (en) * 2020-08-13 2020-11-10 腾讯科技（深圳）有限公司 Live broadcast data processing method and device, computer equipment and storage medium
CN112218148A (en) * 2020-09-11 2021-01-12 杭州易现先进科技有限公司 Screen recording method and device, computer equipment and computer readable storage medium
CN113115111A (en) * 2021-03-16 2021-07-13 广州朗国电子科技有限公司 Video frame rate control method, system and storage medium for screen recording live broadcast

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060087585A1 (en) * 2004-10-26 2006-04-27 Samsung Electronics Co., Ltd. Apparatus and method for processing an image signal in a digital broadcast receiver
CN108900919A (en) * 2018-06-29 2018-11-27 乐蜜有限公司 Company's wheat method, apparatus, server and terminal in a kind of live streaming
CN109089131A (en) * 2018-09-21 2018-12-25 广州虎牙信息科技有限公司 A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system
CN110912922A (en) * 2019-12-03 2020-03-24 锐捷网络股份有限公司 Image transmission method and device, electronic equipment and storage medium
CN111641858A (en) * 2020-04-29 2020-09-08 上海推乐信息技术服务有限公司 Audio and video synchronization method and system
CN111918093A (en) * 2020-08-13 2020-11-10 腾讯科技（深圳）有限公司 Live broadcast data processing method and device, computer equipment and storage medium
CN112218148A (en) * 2020-09-11 2021-01-12 杭州易现先进科技有限公司 Screen recording method and device, computer equipment and computer readable storage medium
CN113115111A (en) * 2021-03-16 2021-07-13 广州朗国电子科技有限公司 Video frame rate control method, system and storage medium for screen recording live broadcast

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Z. ZHANG ET AL.: "Dynamic Frame-Skipping Scheme for Live Video Encoders", 《2010 INTERNATIONAL CONFERENCE ON MULTIMEDIA TECHNOLOGY》, pages 1 - 3 *
黄强 等: "基于IP城域网的4K超高清融合媒体服务平台", 《广播电视网络》, vol. 28, no. 01, pages 94 - 98 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024022427A1 (en) * 2022-07-28 2024-02-01 北京字跳网络技术有限公司 Video recording method and apparatus, device, storage medium, and program product
CN116996639A (en) * 2023-02-13 2023-11-03 深圳Tcl新技术有限公司 Screen-projection frame rate acquisition method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN114245196B (en) 2024-04-19

Similar Documents

Publication Publication Date Title
CN110636346B (en) Code rate self-adaptive switching method and device, electronic equipment and storage medium
JP6226490B2 (en) Low latency rate control system and method
US20100211690A1 (en) Block partitioning for a data stream
CN114245196B (en) Screen recording and stream pushing method and device, electronic equipment and storage medium
US20120057629A1 (en) Rho-domain Metrics
US12015770B2 (en) Method for encoding video data, device, and storage medium
CN110267100B (en) Code rate switching method and device for FLV (flash video), electronic equipment and storage medium
CN110582012B (en) Video switching method, video processing device and storage medium
CN108881931B (en) Data buffering method and network equipment
US11694316B2 (en) Method and apparatus for determining experience quality of VR multimedia
CN108989832B (en) Image data processing method and equipment, storage medium and terminal thereof
CN110913245A (en) Method and device for controlling video transcoding code rate
WO2021047287A1 (en) Video uploading method and apparatus, electronic device, and storage medium
US8867611B2 (en) Video transmitting method and system with image quality adjusting function
CN104918123A (en) Method and system for playback of motion video
CN112866746A (en) Multi-path streaming cloud game control method, device, equipment and storage medium
CN111918134A (en) Method for correcting video stream flow, set top box and computer readable storage medium
CN111107387B (en) Video transcoding method and device and computer storage medium
CN116962613A (en) Data transmission method and device, computer equipment and storage medium
CN115883848A (en) Code control method, device, equipment, storage medium and product
CN115297343A (en) Video data sending method and device and electronic equipment
CN111510703B (en) Video playing method, device and system
CN115442615A (en) Video coding method and device, electronic equipment and storage medium
US20220103899A1 (en) A client and a method for managing, at the client, a streaming session of a multimedia content
WO2021013558A1 (en) A media client with adaptive buffer size and the related method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant