CN112995776A - Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content - Google Patents

Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content Download PDF

Info

Publication number
CN112995776A
CN112995776A CN202110122146.3A CN202110122146A CN112995776A CN 112995776 A CN112995776 A CN 112995776A CN 202110122146 A CN202110122146 A CN 202110122146A CN 112995776 A CN112995776 A CN 112995776A
Authority
CN
China
Prior art keywords
screen
frame rate
current
determining
rate coefficient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110122146.3A
Other languages
Chinese (zh)
Other versions
CN112995776B (en
Inventor
葛绍山
左枝凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to CN202110122146.3A priority Critical patent/CN112995776B/en
Publication of CN112995776A publication Critical patent/CN112995776A/en
Application granted granted Critical
Publication of CN112995776B publication Critical patent/CN112995776B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications
    • H04L65/401—Support for services or applications wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference
    • H04L65/4015—Support for services or applications wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The disclosure provides a method, a device, equipment and a storage medium for determining a screen capture frame rate of shared screen content. One embodiment of the method comprises: in response to determining that the screen image changes, determining a screen change parameter of the current screen image compared to a previous screen image; determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameters; determining a current screen capturing frame rate coefficient according to the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image, wherein the current screen capturing frame rate coefficient is positively correlated with the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image respectively; and determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, wherein the current screen capturing frame rate is positively correlated with the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient respectively. The implementation mode realizes real-time dynamic determination of the screen capture frame rate according to the shared screen content, and reduces the screen capture calculation amount.

Description

Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content
Technical Field
The embodiment of the disclosure relates to the technical field of video processing, in particular to a method, a device, equipment and a storage medium for determining a screen capture frame rate of shared screen content.
Background
Most of the current video conference application or online education application provide a screen sharing function, in order to share a screen picture to other participants at a far end, a user terminal sharing the screen can capture a screen image and generate a video to be sent to a server providing services for the video conference application or online education application under a screen sharing scene so as to share the screen image to other participant terminal devices for presentation.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for determining a screen capture frame rate of shared screen content, a terminal device and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a method for determining a screen capture frame rate of shared screen content, where the method includes: in response to determining that the screen image changes, determining a screen change parameter of the current screen image compared to a previous screen image; determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameter; determining a current screen capturing frame rate coefficient according to a previous screen capturing frame rate coefficient and a screen capturing frame rate coefficient corresponding to the current screen image, wherein the current screen capturing frame rate coefficient is positively correlated with the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image respectively; and determining the current screen capturing frame rate according to a preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, wherein the current screen capturing frame rate is positively correlated with the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient respectively.
In some optional embodiments, the above responding to determining a screen image change comprises: in response to detecting the preset interaction event, determining a screen image change.
In some optional embodiments, the determining the current screen capture frame rate according to the preset maximum screen capture frame rate and the current screen capture frame rate coefficient includes: and determining the product of the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient as the current screen capturing frame rate.
In some optional embodiments, after determining the current screen capture frame rate according to the preset maximum screen capture frame rate and the current screen capture frame rate coefficient, the method further includes: and determining the last screen capturing frame rate coefficient as the current screen capturing frame rate coefficient.
In some optional embodiments, the screen change parameter includes a screen change rate parameter and a screen change ratio; and the determining the screen change parameter of the current screen image compared with the previous screen image comprises: determining a screen change rate parameter and a screen change ratio of a current screen image compared with a previous screen image; and determining a screen capture frame rate coefficient corresponding to the current screen image according to the screen change parameter, including: and determining that the screen capture frame rate coefficient corresponding to the current screen image and the screen capture frame rate coefficient corresponding to the current screen image are positively correlated with the screen change rate parameter and the screen change ratio respectively according to the screen change rate parameter and the screen change ratio.
In some optional embodiments, the determining the screen change rate parameter and the screen change ratio of the current screen image compared with the previous screen image includes: calculating a timestamp difference value between the timestamp corresponding to the current screen image and the timestamp corresponding to the previous screen image; and determining a screen change rate parameter of the current screen image compared with the previous screen image according to the timestamp difference, wherein the screen change rate parameter of the current screen image compared with the previous screen image is inversely proportional to the time difference.
In some optional embodiments, the determining the screen change rate parameter and the screen change ratio of the current screen image compared with the previous screen image includes: determining the number of changed pixel points of which the pixel values are changed compared with the previous screen image; and determining the ratio of the number of the changed pixels to the total number of the pixels in the current screen image as the screen change ratio.
In some optional embodiments, the determining a screen capture frame rate coefficient corresponding to the current screen image according to the screen change rate parameter and the screen change ratio includes: and carrying out weighted summation on the screen change rate parameter and the screen change ratio according to a preset change rate weight and a preset change ratio weight respectively to obtain a screen capturing frame rate coefficient corresponding to the current screen image.
In some optional embodiments, the determining a current screen capture frame rate coefficient according to a last screen capture frame rate coefficient and a screen capture frame rate coefficient corresponding to the current screen image includes: and weighting and summing the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image according to the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight respectively to obtain the current screen capturing frame rate coefficient.
In some optional embodiments, the method further comprises: and capturing screen images according to the current screen capturing frame rate to generate a video.
In some optional embodiments, the method further comprises: and sending the generated video to a target server.
In some optional embodiments, the method further comprises: and capturing the screen image according to the current screen capturing frame rate, and sending the captured screen image to the target server.
In a second aspect, an embodiment of the present disclosure provides an apparatus for determining a screen capture frame rate of shared screen content, the apparatus including: a first determining unit configured to determine a screen change parameter of a current screen image compared to a previous screen image in response to determining a screen image change; a second determining unit configured to determine a screen capture frame rate coefficient corresponding to the current screen image according to the screen change parameter; a third determining unit, configured to determine a current screen capture frame rate coefficient according to a previous screen capture frame rate coefficient and a screen capture frame rate coefficient corresponding to the current screen image, where the current screen capture frame rate coefficient is positively correlated with the previous screen capture frame rate coefficient and the screen capture frame rate coefficient corresponding to the current screen image, respectively; a fourth determining unit, configured to determine a current screen capture frame rate according to a preset maximum screen capture frame rate and the current screen capture frame rate coefficient, where the current screen capture frame rate is positively correlated to the preset maximum screen capture frame rate and the current screen capture frame rate coefficient, respectively.
In some optional embodiments, the above responding to determining a screen image change comprises: in response to detecting the preset interaction event, determining a screen image change.
In some optional embodiments, the fourth determining unit is further configured to: and determining the product of the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient as the current screen capturing frame rate.
In some optional embodiments, the apparatus further comprises: a fifth determination unit configured to: and after determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, determining the last screen capturing frame rate coefficient as the current screen capturing frame rate coefficient.
In some optional embodiments, the screen change parameter includes a screen change rate parameter and a screen change ratio; and the determining the screen change parameter of the current screen image compared with the previous screen image comprises: determining a screen change rate parameter and a screen change ratio of a current screen image compared with a previous screen image; and the second determination unit is further configured to: and determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change rate parameter and the screen change ratio, wherein the screen capturing frame rate coefficient corresponding to the current screen image is positively correlated with the screen change rate parameter and the screen change ratio respectively.
In some optional embodiments, the first determining unit is further configured to: calculating a timestamp difference value between the timestamp corresponding to the current screen image and the timestamp corresponding to the previous screen image; and determining a screen change rate parameter of the current screen image compared with the previous screen image according to the timestamp difference, wherein the screen change rate parameter of the current screen image compared with the previous screen image is inversely proportional to the time difference.
In some optional embodiments, the first determining unit is further configured to: determining the number of changed pixel points of which the pixel values are changed compared with the previous screen image; and determining the ratio of the number of the changed pixels to the total number of the pixels in the current screen image as the screen change ratio.
In some optional embodiments, the second determining unit is further configured to: and carrying out weighted summation on the screen change rate parameter and the screen change ratio according to a preset change rate weight and a preset change ratio weight respectively to obtain a screen capturing frame rate coefficient corresponding to the current screen image.
In some optional embodiments, the third determining unit is further configured to: and weighting and summing the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image according to the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight respectively to obtain the current screen capturing frame rate coefficient.
In some optional embodiments, the apparatus further comprises: and the video generating unit is configured to capture the screen image according to the current screen capture frame rate to generate a video.
In some optional embodiments, the apparatus further comprises: a video transmitting unit configured to transmit the generated video to the target server.
In some optional embodiments, the apparatus further comprises: and the image capturing and sending unit is configured to capture the screen image according to the current screen capturing frame rate and send the captured screen image to the target server.
In a third aspect, an embodiment of the present disclosure provides a terminal device, including: one or more processors; a storage device, on which one or more programs are stored, which, when executed by the one or more processors, cause the one or more processors to implement the method as described in any implementation manner of the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by one or more processors, implements the method as described in any of the implementations of the first aspect.
In order to capture a screen image and send the captured screen image to a server or generate a video based on the captured screen image and send the video to the server in a shared screen scene, an existing terminal device generally captures the screen image at a fixed frame rate, for example, at a fixed frame rate of 15 frames per second.
Since the screen content changes less in most of the time when the user shares the document in the shared screen scenario, the screen content may change more only when the user frequently operates (e.g., the user scrolls the screen document content). In the case of video sharing by users, the screen content changes more if the video content changes faster (e.g., action-like movie works), and less if the video content changes slower (e.g., recording film-like movie jobs).
According to the method, the device, the terminal device and the storage medium for determining the screen capturing frame rate of the shared screen content, the screen change parameter of the current screen image compared with the last screen image is determined under the condition that the screen image is determined to be changed; determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameters; then, determining a current screen capturing frame rate coefficient according to the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image, wherein the current screen capturing frame rate coefficient is respectively in positive correlation with the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image; and finally, determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, wherein the current screen capturing frame rate is positively correlated with the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient respectively. That is, under the condition that the shared screen image content is determined to be changed, the current screen capture frame rate is dynamically determined according to the current screen image change parameter: if the current screen image and the previous screen image have larger changes, the screen capturing frame rate is increased so that the terminals receiving the shared screen content restore the shared screen content as much as possible; and if the current screen image and the previous screen image have less change, reducing the screen capturing frame rate to realize the purpose of giving consideration to the shared screen content received by the terminal for receiving the shared screen content and not wasting the computing resources required by the screen capturing computing operation. In a shared screen scene, compared with the traditional fixed screen capturing frame rate, when the content of a screen image changes less, the lower screen capturing frame rate is adopted, the calculation frequency of screen capturing is correspondingly reduced, the calculation frequency of video encoding to generate a video based on the screen capturing image is correspondingly reduced, and the network bandwidth cost of transmitting the screen capturing image or the video to a server by a terminal device can be further reduced.
Drawings
Other features, objects, and advantages of the disclosure will become apparent from a reading of the following detailed description of non-limiting embodiments which proceeds with reference to the accompanying drawings. The drawings are only for purposes of illustrating the particular embodiments and are not to be construed as limiting the invention. In the drawings:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present disclosure may be applied;
FIG. 2 is a flow diagram for one embodiment of a shared screen content screen capture frame rate determination method according to the present disclosure;
3A-3D are line graphs of real-time dynamic screen capture frame rate during sharing of different types of content using a shared screen content screen capture frame rate determination method according to one embodiment of the present disclosure;
FIG. 4 is a flow diagram of yet another embodiment of a shared screen content screen capture frame rate determination method according to the present disclosure;
FIG. 5 is a schematic structural diagram illustrating an embodiment of a device for determining a frame rate of a shared screen content capture according to the present disclosure;
FIG. 6 is a block diagram of a computer system suitable for use with a terminal device implementing an embodiment of the disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the shared screen content screen capture frame rate determination methods and apparatus of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various communication client applications, such as a video conference application, an online education application, a short video social application, a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various terminal devices having a sound collecting device (e.g. a microphone), a video collecting device (e.g. a camera), and a display screen, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III, motion Picture Experts Group Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion Picture Experts Group Audio Layer 4), a laptop portable computer, a desktop computer, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the above-listed terminal apparatuses. It may be implemented as multiple software or software modules (e.g., to provide a shared screen content screen capture frame rate determination service), or as a single software or software module. And is not particularly limited herein.
The server 105 may be a server that provides various services, such as a background server that provides support for a video conference-like application or an online education-like application displayed on the terminal devices 101, 102, 103. The background server may analyze and perform other processing on the received data such as the shared screen request, and feed back a processing result (e.g., shared screen video data) to the terminal device.
It should be noted that the method for determining the screen capture frame rate of the shared screen content provided by the present disclosure is generally executed by the terminal devices 101, 102, 103, and accordingly, the means for determining the screen capture frame rate of the shared screen content is generally disposed in the terminal devices 101, 102, 103.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continuing reference to FIG. 2, a flow 200 of one embodiment of a shared screen content screen capture frame rate determination method according to the present disclosure is shown, the shared screen content screen capture frame rate determination method comprising the steps of:
in step 201, in response to determining the screen image change, a screen change parameter of the current screen image compared with the previous screen image is determined.
In the present embodiment, the executing subject (for example, the terminal devices 101, 102, 103 shown in fig. 1) of the shared screen content screen capture frame rate determining method may determine the screen change parameter of the current screen image compared with the last screen image in various implementations when determining that the currently shared screen image changes.
In practice, for example, the execution main body may monitor whether the screen image content changes in real time by using a Callback function (i.e., a Callback function) provided by the operating system, and determine the screen image change if it is monitored that the screen image content changes.
In some optional embodiments, the execution subject may further determine the screen image change by detecting a preset interaction event, and in a case that the preset interaction event is detected.
Here, the preset interaction event may be various interaction events that may affect the content displayed on the screen, and may also be understood as an event that may affect the content displayed on the screen and is generated by the user interacting with the terminal device using various interaction devices. For example, the preset interaction event may be a mouse scroll event, a movement event, a click event, or a double click event caused by a mouse scroll operation, a movement operation, a click operation, or a double click operation of a user, and the preset interaction event may also be a keyboard input event caused by a character operation of the user using a keyboard to input characters in the keyboard. The preset interaction event may also be a corresponding interaction event caused by a user performing an operation, such as clicking, double-clicking, moving, inputting characters, and the like, using a screen input device, such as a tablet or a touch screen. Since the preset interactive event may affect the screen display content, the screen image change may be determined in case the preset interactive event is detected.
Here, the current screen image is the currently shared screen image. And the last screen image is a shared screen image of which the corresponding screen capturing time is earlier than that of the current screen image in the screen sharing process. For example, the previous screen image may be the latest previous frame screen capture screen image earlier than the screen capture at the current time point. The last screen image may also be a screen capture screen image corresponding to a specified frame (for example, the earliest frame or the latest frame) captured within a preset time length (for example, 1 second) before the current time point, and the like.
The screen change parameter of the current screen image compared to the previous screen image may include various related parameters for characterizing a change in the screen image of the current screen image compared to the previous screen image. For example, the screen change parameter may include how many pixel points of the current screen image are changed in pixel value compared to the previous screen image, and the like. For another example, the screen change parameter may further include a change degree of the current screen image content compared with the previous screen image content, and the change degree may be determined by various implementations. For example, the difference between the average gray-level value of the current screen image and the average gray-level value of the previous screen image may be calculated, and then the ratio between the difference and the average gray-level value of the previous screen image may be calculated, and the calculated ratio may be used as the change degree value of the content of the current screen image compared with the content of the previous screen image.
In some alternative embodiments, the screen change parameters may include a screen change rate parameter and a screen change ratio.
Here, the screen change rate parameter is used to represent the speed at which the screen image is changed, i.e., the number of times the screen content is changed per unit time. The screen change rate parameter of the current screen image compared with the previous screen image can be understood as that the screen image has changed once in the time length (t2-t1) between the screen capture time t1 corresponding to the previous screen image and the screen capture time t2 corresponding to the current screen image, and the screen change rate parameter of the current screen image compared with the previous screen image can be calculated as follows: 1/(t2-t1), and here (t2-t1) different time units may be specifically employed to calculate the screen change rate parameter of the current screen image compared to the previous screen image.
In some alternative embodiments, the screen change rate parameter of the current screen image compared to the previous screen image may be determined as follows:
first, a timestamp difference between a timestamp corresponding to a current screen image and a timestamp corresponding to a previous screen image may be calculated.
Then, a screen change rate parameter of the current screen image compared to the previous screen image may be determined according to the timestamp difference, wherein the screen change rate parameter of the current screen image compared to the previous screen image is inversely proportional to the time difference.
For example, the screen change rate parameter of the current screen image compared to the previous screen image may be calculated according to the following formula 1:
updateRate knpsec/(deltaNs × kMaxUpdateRate) (equation 1)
Wherein, updateRate is a screen change rate parameter obtained by calculation, kNsPerSec is a constant-10 to the power of 9, namely, the number of nanoseconds included in one second is the power of 9 of 10. deltaNs is the difference in time stamp between the time stamp corresponding to the current screen and the time stamp corresponding to the previous screen. kMaxUpdateRate is a manually set constant characterizing the maximum screen refresh rate, for example kMaxUpdateRate may be set to 50, i.e. typically the screen maximum refresh rate is refreshed 50 times in 1 second.
The screen change rate parameter determined according to the above-described alternative embodiment is relatively high in accuracy because it is in nanoseconds, and can be determined more accurately.
Here, the screen change ratio is used to represent the degree to which the current screen image changes from the previous screen image. For example, a pixel mean value difference between a pixel mean value of each pixel of the current screen image and a pixel mean value of each pixel of the previous screen image may be determined, and then a ratio of the obtained pixel mean value difference to the pixel mean value of each pixel of the previous screen image may be determined as a screen change ratio of the current screen image to the previous screen image.
In some alternative embodiments, the screen change ratio of the current screen image compared to the previous screen image may be determined as follows:
first, the number of changed pixels whose pixel values are changed from the previous screen image in the current screen image may be determined.
Then, the ratio of the number of the changed pixels to the total number of the pixels in the current screen image is determined as the screen change ratio of the current screen image to the previous screen image.
Specifically, the screen change ratio of the current screen image compared to the previous screen image can be calculated by the following formula 2:
updatate ratio ═ updatate area/displayArea (equation 2)
The updateRatio is a calculated screen change ratio, the number of changed pixel points of which the pixel value of the updateArea current screen image changes compared with the previous screen image is the updated pixel value, and the displayArea is the total number of the pixel points in the current screen image.
According to the screen change ratio determined by the optional embodiment, since each pixel point is accurate, the relative accuracy is high, and the screen change ratio can be determined more accurately, that is, each pixel point can be accurate.
Step 202, determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameter.
Here, the screen capture frame rate coefficient corresponding to the current screen image is used to represent the screen capture frame rate coefficient corresponding to the current screen image, which is determined only according to the screen change rate parameter and the screen change ratio of the current screen image compared with the previous screen image, without considering the screen capture frequency coefficient corresponding to each frame of screen images captured before.
It can be understood that, if it is determined that the screen image has more changed area, higher changed degree, or faster changed according to the screen change parameter determined in step 201, in order to enable the shared party to timely know the change of the screen content shared by the sharing party, the screen capture frame rate needs to be increased, that is, the screen capture frame rate coefficient corresponding to the current screen image needs to be increased accordingly. On the contrary, if it is determined that the screen image has a smaller change area, a lower change degree, or a slower change according to the screen change parameter determined in step 201, in order to both present the screen image content and save the computing resource, the screen capture frame rate may be correspondingly reduced, that is, the screen capture frame rate coefficient corresponding to the current screen image needs to be correspondingly reduced.
Correspondingly, it can be understood that if the screen change rate corresponding to the current screen image is fast, or the screen change corresponding to the current screen image is large, that is, the screen content change degree of the current shared screen is large, it indicates that more information different from the previous information presentation is involved in the current screen image, and the screen capture frame rate needs to be increased. On the contrary, if the screen change rate corresponding to the current screen image is slow, or the screen change corresponding to the current screen image is small, that is, the screen content change degree of the current shared screen is small, it indicates that the current screen image is less related to information presentation different from the previous screen image, and the screen capture frame rate needs to be reduced. In summary, the screen capture frame rate coefficient corresponding to the current screen image should be positively correlated with the screen change rate parameter and the screen change ratio corresponding to the current screen image, respectively. That is, the higher the screen change rate of the current screen image compared to the previous screen image is, the higher the screen capture frame rate coefficient corresponding to the current screen image is. The higher the screen change ratio of the current screen image compared with the previous screen image is, the higher the screen capture frame rate coefficient corresponding to the current screen image is.
In some alternative embodiments, step 202 may proceed as follows:
and carrying out weighted summation on the screen change rate parameter and the screen change ratio according to the preset change rate weight and the preset change ratio weight respectively to obtain a screen-grabbing frame rate coefficient corresponding to the current screen image.
Specifically, the screen capture frame rate coefficient corresponding to the current screen image may be determined according to the screen change rate parameter and the screen change ratio using the following equation 3:
curCoefficient＝kRateWeight×updateRate+kRatioWeight×updateRatio
(formula 3)
The current Coefficient is a screen capture frame rate coefficient corresponding to a current screen image obtained through calculation, kRateWeight is a preset change rate weight, kRatioWeight is a preset change ratio weight, updateRate is a screen change rate parameter obtained through previous calculation, and updateRate is a screen change ratio obtained through previous calculation.
According to the optional implementation mode, the occupation ratio condition of the screen change rate parameter and the screen change ratio in the screen capturing frame rate coefficient corresponding to the current screen image can be adjusted by setting different preset change rate weights and preset change ratio weights, and corresponding adjustment can be performed according to practical effects so as to achieve the purpose of giving consideration to both the video generation quality and the screen capturing calculated amount. Alternatively, the preset change rate weight and the preset change ratio weight may both be values between 0 and 1, and the sum of the two is 1.
And 203, determining the current screen capturing frame rate coefficient according to the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image.
Because the user has certain continuity in time in the shared screen scene, the interaction of the user to the shared screen content also has certain continuity in time correspondingly to the change of the shared screen image content. Although the screen capture frame rate coefficient corresponding to the current screen image is determined in step 202, if the current screen capture frame rate is determined only according to the screen capture frame rate coefficient corresponding to the current screen image, a jump of the screen capture frame rate may be caused, and then a jump of the picture quality of the shared screen content seen by the shared party sharing the screen may be caused, which is not favorable for the shared party sharing the screen to acquire the shared screen content. Therefore, in order to improve the user's experience of the video content of the shared screen, the screen capture frame rate should not change too much, that is, the screen capture frame rate should change more smoothly, so as to avoid the phenomenon of skipping of the picture quality of the shared screen acquired by the sharing party of the shared screen. Therefore, when determining the current screen capture frame rate coefficient, in addition to the screen capture frame rate coefficient corresponding to the current screen image, the previous screen capture frame rate coefficient should be considered, and the previous screen capture frame rate coefficient further includes the current screen capture frame rate coefficient corresponding to the previous screen capture image and the previous screen capture frame rate coefficient, so that it can be understood that the current screen capture frame rate coefficient includes a plurality of historical screen capture frame rate coefficients in addition to the screen capture frame rate coefficient corresponding to the current screen image, so as to realize smooth transition of the screen capture frame rate, and further realize continuity of picture quality of the shared screen images.
Here, the last screen capture frame rate coefficient is the last screen capture frame rate coefficient determined.
In this embodiment, the executing entity may determine the current screen capture frame rate coefficient according to the last screen capture frame rate coefficient and the screen capture frame rate coefficient corresponding to the current screen image by using various embodiments. The current screen capturing frame rate coefficient is positively correlated with the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image.
In some alternative embodiments, step 203 may be performed as follows:
and respectively carrying out weighted summation on the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image according to the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight to obtain the current screen capturing frame rate coefficient. Wherein, the sum of the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight is 1.
Specifically, the following equation 4 can be expressed as follows:
coefficient ═ kHistoryWeight × prevCoefficient + (1-kHistoryWeight) × (formula 4)
Wherein, coefficient is the current screen capture frame rate coefficient obtained by calculation, kHistoryWeiight is the preset historical frame rate coefficient weight, prevCoefficient is the last screen capture frame rate coefficient, (1-kHistoryWeiight) is the preset current frame rate coefficient weight, and currCoefficient is the screen capture frame rate coefficient corresponding to the current screen image obtained by previous calculation.
According to the optional implementation mode, the occupation ratio condition of the historical frame rate coefficient and the screen capture frame rate coefficient corresponding to the current screen image in the current frame rate coefficient obtained through calculation can be adjusted by setting different preset historical frame rate coefficient weights and preset current frame rate coefficient weights, and corresponding adjustment can be carried out according to practical effects so as to achieve the purpose of giving consideration to both the video generation quality and the screen capture calculation amount. For example, the preset historical frame rate coefficient weight kHistoryWeight may be set to 0.5, and accordingly, the preset current frame rate coefficient weight may also be 0.5.
And 204, determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient.
In this embodiment, the executing main body may determine the current screen capture frame rate according to the preset maximum screen capture frame rate and the current screen capture frame rate coefficient calculated in step 203 by using various implementation manners. The current screen capturing frame rate is positively correlated with the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient respectively.
In some alternative embodiments, step 204 may proceed as follows:
and determining the product of the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient as the current screen capturing frame rate.
Specifically, it can be expressed as follows by the following equation 5:
fps ═ kMaxFps × coeffient (equation 5)
fps is the current screen capture frame rate obtained through calculation, kMaxFps is the preset maximum screen capture frame rate, and coefficient is the current screen capture frame rate coefficient obtained through previous calculation.
According to the optional implementation mode, the current screen capture frame rate can be directly determined in real time through the current screen capture frame rate coefficient dynamically determined in real time.
By combining the formulas 3 to 5, it can be known that the current screen capture frame rate coefficient can be calculated according to the following formula 6:
fps ═ kMaxFps × (kHistoryWeight × prevCoefficient + (1-kHistoryWeight) × (kRateWeight × updateRate + kRatioWeight × updateRatio)) (formula 6)
In some optional embodiments, the executing main body may further perform the following step 205 after performing step 204:
in step 205, the previous screen capture frame rate coefficient is determined as the current screen capture frame rate coefficient.
That is, the last screen-grabbing frame rate coefficient is updated by the current screen-grabbing frame rate coefficient determined in step 204. And then detecting the change of the screen image next time, and using the updated last screen capturing frame rate coefficient in the process of executing the steps 201 to 204.
In some optional implementations of this embodiment, after the executing step 205 is completed, the executing step 206 may further be executed:
and step 206, capturing the screen image according to the current screen capturing frame rate, and sending the captured screen image to the target server.
Here, the target server may be a server that provides support for a video conference application or a network video application providing a shared screen function for an online education application or the like.
In this way, after receiving the screen image sent by the execution main body, the target server may generate a corresponding video stream based on the screen image received from the execution main body and forward the video stream to the shared terminal device corresponding to the current shared screen image. And then, the target server generates the video stream with the corresponding frame rate in real time according to the shared screen content, and compared with the target server which generates the video stream with the fixed frame rate, the calculation amount required by the target server for generating the video stream can be reduced when the current screen capturing frame rate is lower than the fixed frame rate.
With continued reference to fig. 3A to 3D, fig. 3A to 3D are line graphs of real-time dynamic screen capture frame rate in the process of sharing different types of content by using the method for determining screen capture frame rate of shared screen content according to the present embodiment. In fig. 3A to 3D, the abscissa is time, and the ordinate is a screen capture frame rate value corresponding to the corresponding time.
Wherein:
FIG. 3A is a line graph of real-time dynamic screen capture frame rates in a scene sharing static documents. As can be seen from fig. 3A, the real-time dynamic screen capture frame rate is 0 in most cases, that is, the screen capture is not performed, and the real-time dynamic screen capture frame rate reaches 2-5 frames per second in some cases, so that the calculation amount generated by screen capture is greatly reduced.
FIG. 3B is a line graph of real-time dynamic screen frame rate of scrolling documents and manipulating screen scenes during sharing. As can be seen from FIG. 3B, in this scenario, the fluctuation range of the real-time dynamic screen capture frame rate is relatively large, a large portion is located between 5-13 frames per second, and a portion is 0. When the user scrolls the shared document, if the speed of the user scrolling the screen is high, the screen-grabbing frame rate is correspondingly high, for example, 13 frames can be reached; if the speed of the user for scrolling the screen is slow, the frame rate of the screen grabbing is correspondingly small, for example, 5 frames can be reached; and if the user does not scroll the document, the corresponding screen capturing frame rate is 0, and the actual situation is relatively met.
Fig. 3C is a line graph of a real-time dynamic screen capture frame rate in a scene in which a video is played in a shared half-screen. As can be seen from fig. 3C, in the shared half-screen video scene, the real-time dynamic screen capturing frame rate is mainly concentrated in the range of 10-14 frames per second, and the fluctuation range is smaller than that of the scene shown in fig. 3B, so as to meet the requirement of the user for the stable high frame rate when sharing the video scene. This is because, in the shared half-screen play video scene, only half of the screen is shared, and the shared screen is playing video. The screen image change rate parameter and the screen change ratio of the shared video are generally larger than those of the shared document, so the screen capture frame rate in the scene is generally higher than that of 5-13 frames corresponding to the scrolling screen when the document is shared, but the screen capture frame rate of 10-14 frames exists because half of the screen is shared. In addition, under the condition of sharing half-screen playing video, the video content generally has continuity and does not jump too much, so that the corresponding variation range of the screen-grabbing frame rate is not large and is between 10 and 14 frames.
Fig. 3D is a line graph of a real-time dynamic screen capture frame rate in a scene of sharing a full-screen playing video. As can be seen from fig. 3D, in a scene of sharing a full-screen playing video, since video content often has continuity, the change amplitude of the real-time dynamic screen capture frame rate is relatively small and stable, and the real-time dynamic screen capture frame rate is mostly between 11-14 frames per second. In addition, because the whole screen is shared to play the video, the screen change is slightly higher than that of the video played by sharing the half screen, compared with the video played by sharing the half screen, the frame rate of the whole real-time dynamic screen capture is 11-14 frames which is slightly higher than that of the half screen video played by 10-14 frames, and the requirement of the user on sharing the scene of the video played in the full screen is met.
According to the method for determining the screen capture frame rate of the shared screen content provided by the embodiment of the disclosure, the screen capture frame rate is correspondingly and dynamically adjusted in real time based on the change of the shared screen content, and a proper screen capture frame rate is selected under different sharing scenes, so that the screen capture calculation amount and the video coding calculation amount of the terminal equipment are reduced, and the network bandwidth cost between the terminal equipment and the server can be further reduced.
With continued reference to fig. 4, a flow 400 of yet another embodiment of a shared screen content screen capture frame rate determination method according to the present disclosure is shown. The method for determining the screen capturing frame rate of the shared screen content comprises the following steps:
in response to determining the screen image change, a screen change parameter of the current screen image compared to the previous screen image is determined, step 401.
Step 402, determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameter.
Step 403, determining the current screen capturing frame rate coefficient according to the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image.
And step 404, determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient.
Step 405, determining the last screen capture frame rate coefficient as the current screen capture frame rate coefficient.
In this embodiment, the specific operations of step 401, step 402, step 403, step 404, and step 405 are substantially the same as the operations of step 201, step 202, step 203, step 204, and step 205 in the embodiment shown in fig. 2, and are not described herein again.
And 406, capturing screen images according to the current screen capturing frame rate to generate a video.
In this embodiment, an executing subject (e.g., the terminal devices 101, 102, 103 shown in fig. 1) of the shared screen content screen capture frame rate determining method may capture a screen image generation video according to the current screen capture frame rate determined in step 404.
Step 407, the generated video is sent to the target server.
Here, the target server may be a server that provides support for a video conference application or a network video application providing a shared screen function for an online education application or the like.
As can be seen from fig. 4, compared with the embodiment corresponding to fig. 2, the flow 400 of the method for determining the screen capture frame rate of the shared screen content in this embodiment has more steps of generating a video and sending the video to the target server, that is, the screen capture frame rate is dynamically determined in real time according to the shared screen content in steps 401 to 404, the screen image is captured according to the screen capture frame rate dynamically determined in real time in step 406 to generate a video, and the video is sent to the server in step 407 for other participants to view the shared screen image. Furthermore, the calculation amount of the video generated by the terminal equipment can be reduced, and the bandwidth resource occupation between the terminal equipment and the target server can be reduced.
With further reference to fig. 5, as an implementation of the method shown in the above-mentioned figures, the present disclosure provides an embodiment of a device for determining a screen capture frame rate of shared screen content, where the embodiment of the device corresponds to the embodiment of the method shown in fig. 2, and the device may be specifically applied to various terminal devices.
As shown in fig. 5, the device 500 for determining the frame rate of screen capture of shared screen content according to the present embodiment includes: a first determination unit 501, a second determination unit 502, a third determination unit 503, and a fourth determination unit 504. Wherein, the first determining unit 501 is configured to determine a screen change parameter of the current screen image compared with the previous screen image in response to determining the screen image change; a second determining unit 502 configured to determine a screen capture frame rate coefficient corresponding to the current screen image according to the screen change parameter; a third determining unit 503, configured to determine a current screen capture frame rate coefficient according to a previous screen capture frame rate coefficient and a screen capture frame rate coefficient corresponding to the current screen image, where the current screen capture frame rate coefficient is positively correlated to the previous screen capture frame rate coefficient and the screen capture frame rate coefficient corresponding to the current screen image, respectively; a fourth determining unit 504, configured to determine a current screen capture frame rate according to a preset maximum screen capture frame rate and the current screen capture frame rate coefficient, where the current screen capture frame rate is positively correlated to the preset maximum screen capture frame rate and the current screen capture frame rate coefficient, respectively.
In this embodiment, specific processes of the first determining unit 501, the second determining unit 502, the third determining unit 503 and the fourth determining unit 504 of the device 500 for determining a screen capturing frame rate of a shared screen content and technical effects brought by the specific processes may respectively refer to relevant descriptions of step 201, step 202, step 203 and step 204 in the corresponding embodiment of fig. 2, and are not repeated herein.
In some optional embodiments, the above responding to determining a screen image change comprises: in response to detecting the preset interaction event, determining a screen image change.
In some optional embodiments, the fourth determining unit 504 may be further configured to: and determining the product of the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient as the current screen capturing frame rate.
In some optional embodiments, the apparatus 500 may further include: a fifth determining unit 505 configured to: and after determining the current screen capturing frame rate according to the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, determining the last screen capturing frame rate coefficient as the current screen capturing frame rate coefficient.
In some optional embodiments, the screen change parameter includes a screen change rate parameter and a screen change ratio; and the determining the screen change parameter of the current screen image compared with the previous screen image comprises: determining a screen change rate parameter and a screen change ratio of a current screen image compared with a previous screen image; and the second determination unit is further configured to: and determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change rate parameter and the screen change ratio, wherein the screen capturing frame rate coefficient corresponding to the current screen image is positively correlated with the screen change rate parameter and the screen change ratio respectively.
In some optional embodiments, the first determining unit 501 may be further configured to: calculating a timestamp difference value between the timestamp corresponding to the current screen image and the timestamp corresponding to the previous screen image; and determining a screen change rate parameter of the current screen image compared with the previous screen image according to the timestamp difference, wherein the screen change rate parameter of the current screen image compared with the previous screen image is inversely proportional to the time difference.
In some optional embodiments, the first determining unit 501 may be further configured to: determining the number of changed pixel points of which the pixel values are changed compared with the previous screen image; and determining the ratio of the number of the changed pixels to the total number of the pixels in the current screen image as the screen change ratio.
In some optional embodiments, the second determining unit 502 may be further configured to: and carrying out weighted summation on the screen change rate parameter and the screen change ratio according to a preset change rate weight and a preset change ratio weight respectively to obtain a screen capturing frame rate coefficient corresponding to the current screen image.
In some optional embodiments, the third determining unit 503 may be further configured to: and weighting and summing the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image according to the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight respectively to obtain the current screen capturing frame rate coefficient.
In some optional embodiments, the apparatus 500 may further include: and a video generating unit 506 configured to capture the screen image according to the current screen capture frame rate to generate a video.
In some optional embodiments, the apparatus may further include: a video transmitting unit 507 configured to transmit the generated video to the target server.
In some optional embodiments, the apparatus may further include: an image capture and transmission unit 508 configured to capture a screen image at the current screen capture frame rate and transmit the captured screen image to the target server.
It should be noted that, for implementation details and technical effects of each unit in the shared screen content screen capturing frame rate determining device provided by the embodiment of the present disclosure, reference may be made to descriptions of other embodiments in the present disclosure, and details are not described herein again.
Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use in implementing the terminal devices of the present disclosure is shown. The computer system 600 shown in fig. 6 is only one example and should not bring any limitations to the functionality or scope of use of embodiments of the present disclosure.
As shown in fig. 6, computer system 600 may include a processing device (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage device 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the computer system 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the computer system 600 to communicate with other devices, wireless or wired, to exchange data. While fig. 6 illustrates a computer system 600 having a terminal device with various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be included in the terminal device; or may exist separately without being assembled into the terminal device.
The computer readable medium carries one or more programs, which when executed by the terminal device, enable the terminal device to implement the method for determining the frame rate for capturing the shared screen content as shown in the embodiment shown in fig. 2 and its optional implementation, and/or the method for determining the frame rate for capturing the shared screen content as shown in the embodiment shown in fig. 4 and its optional implementation.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a cell does not in some cases constitute a limitation on the cell itself, for example, the first determination unit may also be described as a "cell that determines a screen change parameter of a current screen image compared to a previous screen image in response to determining a screen image change".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Claims (15)

1. A method for determining a screen capture frame rate of shared screen content, the method comprising:
in response to determining that the screen image changes, determining a screen change parameter of the current screen image compared to a previous screen image;
determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change parameter;
determining a current screen capturing frame rate coefficient according to a previous screen capturing frame rate coefficient and a screen capturing frame rate coefficient corresponding to the current screen image, wherein the current screen capturing frame rate coefficient is positively correlated with the previous screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image respectively;
determining a current screen capturing frame rate according to a preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient, wherein the current screen capturing frame rate is positively correlated with the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient respectively.
2. The method of claim 1, wherein said responding to determining a screen image change comprises:
in response to detecting the preset interaction event, determining a screen image change.
3. The method of claim 1, wherein the determining a current screen capture frame rate according to a preset maximum screen capture frame rate and the current screen capture frame rate coefficient comprises:
and determining the product of the preset maximum screen capturing frame rate and the current screen capturing frame rate coefficient as the current screen capturing frame rate.
4. The method of claim 1, wherein after determining the current screen capture frame rate according to a preset maximum screen capture frame rate and the current screen capture frame rate coefficient, the method further comprises:
and determining the last screen capturing frame rate coefficient as the current screen capturing frame rate coefficient.
5. The method of claim 1, wherein the screen change parameters include a screen change rate parameter and a screen change ratio; and
the determining the screen change parameter of the current screen image compared with the last screen image comprises the following steps:
determining a screen change rate parameter and a screen change ratio of a current screen image compared with a previous screen image; and
determining a screen capture frame rate coefficient corresponding to the current screen image according to the screen change parameter, including:
and determining a screen capturing frame rate coefficient corresponding to the current screen image according to the screen change rate parameter and the screen change ratio, wherein the screen capturing frame rate coefficient corresponding to the current screen image is positively correlated with the screen change rate parameter and the screen change ratio respectively.
6. The method of claim 5, wherein the determining a screen change rate parameter and a screen change ratio of the current screen image compared to the previous screen image comprises:
calculating a timestamp difference value between the timestamp corresponding to the current screen image and the timestamp corresponding to the previous screen image;
and determining a screen change rate parameter of the current screen image compared with the last screen image according to the timestamp difference, wherein the screen change rate parameter of the current screen image compared with the last screen image is inversely proportional to the time difference.
7. The method of claim 5, wherein the determining a screen change rate parameter and a screen change ratio of the current screen image compared to the previous screen image comprises:
determining the number of changed pixel points of which the pixel values are changed compared with the previous screen image in the current screen image;
and determining the ratio of the number of the changed pixels to the total number of pixels in the current screen image as the screen change ratio.
8. The method of claim 5, wherein the determining a screen capture frame rate coefficient corresponding to the current screen image from the screen change rate parameter and the screen change ratio comprises:
and carrying out weighted summation on the screen change rate parameter and the screen change ratio according to a preset change rate weight and a preset change ratio weight respectively to obtain a screen capturing frame rate coefficient corresponding to the current screen image.
9. The method of claim 1, wherein determining a current screen capture frame rate coefficient according to a last screen capture frame rate coefficient and a screen capture frame rate coefficient corresponding to the current screen image comprises:
and carrying out weighted summation on the last screen capturing frame rate coefficient and the screen capturing frame rate coefficient corresponding to the current screen image according to the preset historical frame rate coefficient weight and the preset current frame rate coefficient weight respectively to obtain the current screen capturing frame rate coefficient.
10. The method according to any one of claims 1-9, wherein the method further comprises:
and capturing screen images according to the current screen capturing frame rate to generate a video.
11. The method of claim 10, wherein the method further comprises:
and sending the generated video to a target server.
12. The method according to any one of claims 1-9, wherein the method further comprises:
and capturing screen images according to the current screen capturing frame rate, and sending the captured screen images to a target server.
13. A shared screen content screen capture frame rate determination apparatus, the apparatus comprising:
a first determining unit configured to determine a screen change parameter of a current screen image compared to a previous screen image in response to determining a screen image change;
a second determining unit configured to determine a screen capture frame rate coefficient corresponding to the current screen image according to the screen change parameter;
a third determining unit, configured to determine a current screen capture frame rate coefficient according to a previous screen capture frame rate coefficient and a screen capture frame rate coefficient corresponding to the current screen image, where the current screen capture frame rate coefficient is positively correlated with the previous screen capture frame rate coefficient and the screen capture frame rate coefficient corresponding to the current screen image, respectively;
a fourth determining unit, configured to determine a current screen capture frame rate according to a preset maximum screen capture frame rate and the current screen capture frame rate coefficient, where the current screen capture frame rate is positively correlated with the preset maximum screen capture frame rate and the current screen capture frame rate coefficient, respectively.
14. A terminal device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-12.
15. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by one or more processors, implements the method of any one of claims 1-12.
CN202110122146.3A 2021-01-26 2021-01-26 Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content Active CN112995776B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110122146.3A CN112995776B (en) 2021-01-26 2021-01-26 Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110122146.3A CN112995776B (en) 2021-01-26 2021-01-26 Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content

Publications (2)

Publication Number Publication Date
CN112995776A true CN112995776A (en) 2021-06-18
CN112995776B CN112995776B (en) 2023-04-07

Family

ID=76345713

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110122146.3A Active CN112995776B (en) 2021-01-26 2021-01-26 Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content

Country Status (1)

Country Link
CN (1) CN112995776B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113885980A (en) * 2021-09-24 2022-01-04 北京航天新立科技有限公司 Desktop sharing method and system based on local area network
CN114338550A (en) * 2022-03-14 2022-04-12 杭州子默网络科技有限公司 Cloud computer real-time coding method and system for reducing bandwidth
CN116931864A (en) * 2023-09-18 2023-10-24 广东保伦电子股份有限公司 Screen sharing method and intelligent interaction panel
WO2023241485A1 (en) * 2022-06-16 2023-12-21 中兴通讯股份有限公司 Video processing method, video sending end, video receiving end, storage medium and program product

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102883135A (en) * 2012-11-01 2013-01-16 成都飞视美视频技术有限公司 Screen sharing and control method thereof
US20160132282A1 (en) * 2014-11-11 2016-05-12 Samsung Electronics Co., Ltd. Display apparatus and display methods thereof
CN106954004A (en) * 2017-03-17 2017-07-14 宇龙计算机通信科技(深圳)有限公司 A kind of method and device of Screen sharing
CN108810610A (en) * 2017-05-05 2018-11-13 腾讯科技（深圳）有限公司 screen sharing method and device
CN109104610A (en) * 2017-06-20 2018-12-28 微软技术许可有限责任公司 Real time screen is shared
WO2020042350A1 (en) * 2018-08-27 2020-03-05 广州视源电子科技股份有限公司 Desktop screen projection method and device, apparatus, and storage medium
CN111182303A (en) * 2019-10-08 2020-05-19 腾讯科技（深圳）有限公司 Encoding method and device for shared screen, computer readable medium and electronic equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102883135A (en) * 2012-11-01 2013-01-16 成都飞视美视频技术有限公司 Screen sharing and control method thereof
US20160132282A1 (en) * 2014-11-11 2016-05-12 Samsung Electronics Co., Ltd. Display apparatus and display methods thereof
CN106954004A (en) * 2017-03-17 2017-07-14 宇龙计算机通信科技(深圳)有限公司 A kind of method and device of Screen sharing
CN108810610A (en) * 2017-05-05 2018-11-13 腾讯科技（深圳）有限公司 screen sharing method and device
CN109104610A (en) * 2017-06-20 2018-12-28 微软技术许可有限责任公司 Real time screen is shared
WO2020042350A1 (en) * 2018-08-27 2020-03-05 广州视源电子科技股份有限公司 Desktop screen projection method and device, apparatus, and storage medium
CN111182303A (en) * 2019-10-08 2020-05-19 腾讯科技（深圳）有限公司 Encoding method and device for shared screen, computer readable medium and electronic equipment

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113885980A (en) * 2021-09-24 2022-01-04 北京航天新立科技有限公司 Desktop sharing method and system based on local area network
CN113885980B (en) * 2021-09-24 2023-10-24 北京航天新立科技有限公司 Desktop sharing method and system based on local area network
CN114338550A (en) * 2022-03-14 2022-04-12 杭州子默网络科技有限公司 Cloud computer real-time coding method and system for reducing bandwidth
WO2023241485A1 (en) * 2022-06-16 2023-12-21 中兴通讯股份有限公司 Video processing method, video sending end, video receiving end, storage medium and program product
CN116931864A (en) * 2023-09-18 2023-10-24 广东保伦电子股份有限公司 Screen sharing method and intelligent interaction panel
CN116931864B (en) * 2023-09-18 2024-02-09 广东保伦电子股份有限公司 Screen sharing method and intelligent interaction panel

Also Published As

Publication number Publication date
CN112995776B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
CN112995776B (en) Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content
CN112261459B (en) Video processing method and device, electronic equipment and storage medium
US9509741B2 (en) Snapshot capture for a communication session
CN110784754A (en) Video display method and device and electronic equipment
CN112291502B (en) Information interaction method, device and system and electronic equipment
CN112256223B (en) Method and device for adjusting display fluency and electronic equipment
CN112272226B (en) Picture loading method and device and readable storage medium
CN111818383B (en) Video data generation method, system, device, electronic equipment and storage medium
CN112312225B (en) Information display method and device, electronic equipment and readable medium
CN112261349B (en) Image processing method and device and electronic equipment
CN110809166B (en) Video data processing method and device and electronic equipment
EP4207775A1 (en) Method and apparatus for determining object addition mode, electronic device, and medium
CN112040328B (en) Data interaction method and device and electronic equipment
CN114760309A (en) Business interaction method, device, equipment and medium of terminal based on cloud service
CN111385638B (en) Video processing method and device
CN114066722A (en) Method and device for acquiring image and electronic equipment
CN112312224A (en) Information display method and device and electronic equipment
CN113965779A (en) Cloud game data transmission method, device and system and electronic equipment
CN113114955B (en) Video generation method and device and electronic equipment
CN115134614B (en) Task parameter configuration method, device, electronic equipment and computer readable storage medium
CN111367592A (en) Information processing method and device
CN111291254A (en) Information processing method and device
CN115134639B (en) Video gear determining method, device, server, storage medium and system
WO2024022427A1 (en) Video recording method and apparatus, device, storage medium, and program product
CN109600641B (en) Method and apparatus for processing information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant