CN114185630B

CN114185630B - Screen recording method, device, computer equipment and storage medium

Info

Publication number: CN114185630B
Application number: CN202111435016.1A
Authority: CN
Inventors: 方耀; 赵山河; 朱禹; 李锦南; 章杨清; 邬稳
Original assignee: Zhaolian Consumer Finance Co ltd
Current assignee: Zhaolian Consumer Finance Co ltd
Priority date: 2021-11-29
Filing date: 2021-11-29
Publication date: 2024-04-23
Anticipated expiration: 2041-11-29
Also published as: CN114185630A

Abstract

The application relates to a screen recording method, a screen recording device, computer equipment and a storage medium. The method comprises the following steps: setting a page to be recorded in a target application as a preset page; the target application is an application needing to be recorded; extracting features based on the preset page and the target application to obtain a preset page configuration file; the screen sharing picture is sent to a background streaming server in real time, and a current page judgment result sent by the background streaming server is received; the current page judgment result is generated after the background streaming server judges the preset page and switches frames of the screen sharing picture based on the preset page configuration file; and recording the screen sharing picture based on the current page judging result. By adopting the method, the excessive collection of the screen recording information of the user can be avoided.

Description

Screen recording method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of internet technologies, and in particular, to a screen recording method, apparatus, computer device, and storage medium.

Background

The recording screen may also be referred to as a screen recording, and may be used to demonstrate the flow of operations to the screen for scene reproduction. Taking the intelligent terminal as an example, a user can record the operation flow of the screen of the intelligent terminal to obtain a screen recording file, and after obtaining the screen recording file, other users can perform corresponding operation on the intelligent terminal according to the operation flow displayed in the screen recording file, so that scene reproduction is realized.

Taking common Android and IOS as examples, current mobile phone screen sharing software depends on an implementation scheme of a system bottom layer and is limited by a system level. When the application for starting screen sharing is suspended in the background, the switching action of other applications cannot be detected by a system callback method. When user specified information is collected through the screen recording sharing function, if the user does not operate according to the prompt, information which does not meet the requirements and is unnecessary is collected in the screen recording process.

Therefore, the existing screen recording method often has the problem of excessive acquisition of screen recording information of the user.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a screen recording method, apparatus, computer device, and computer-readable storage medium that can avoid excessive collection of screen recording information for a user.

In a first aspect, the present application provides a screen recording method. The method comprises the following steps:

setting a page to be recorded in a target application as a preset page; the target application is an application needing to be recorded;

extracting features based on the preset page and the target application to obtain a preset page configuration file;

The screen sharing picture is sent to a background streaming server in real time, and a current page judgment result sent by the background streaming server is received; the current page judgment result is generated after the background streaming server judges the preset page and switches frames of the screen sharing picture based on the preset page configuration file;

and recording the screen sharing picture based on the current page judging result.

In one embodiment, the recording the screen sharing picture based on the current page judgment result includes:

if the screen sharing picture is a preset page, recording the screen of the current page;

If the screen sharing picture is switched in the application, prompting the user to return to the page to be recorded;

if the screen sharing picture is switched between applications, prompting a user to enter a target application;

if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording;

if the current page is in the switch between the application and the homepage and the current page is not in the homepage, terminating the screen recording.

In one embodiment, the recording the screen sharing picture based on the current page judgment result further includes:

If the screen sharing picture is not in a preset page and page switching state and the screen sharing picture is in a homepage, prompting a user to return to a target application for recording;

If the screen sharing picture is not in the preset page and page switching state, and the screen sharing picture is not in the homepage, ending the screen recording.

In a second aspect, the present application provides a screen recording method. The method comprises the following steps:

loading a preset page configuration file, and receiving a screen sharing picture sent by a front-end screen recording application in real time;

Extracting the characteristics of the screen sharing picture to obtain a current page configuration file containing page characteristics;

Performing preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating a current page judgment result;

And sending the current page judging result to the front-end screen recording application.

In one embodiment, the extracting the features of the screen sharing picture, and obtaining the current page configuration file including the page features includes:

extracting text features of the screen sharing picture to obtain the text features of the screen sharing picture; the text features are regular expressions and/or coordinate range points of keywords which can be matched by the background;

extracting image features of the screen sharing picture to obtain image features of multiple dimensions of the screen sharing picture; the image features of the multiple dimensions comprise at least one of color features, texture features, feature vectors, feature values, edge features, grid features, image local areas and image sub-templates;

and generating a current page configuration file based on the text characteristics and the image characteristics of the screen sharing picture.

In one embodiment, the performing preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating the current page judgment result includes:

Carrying out preset page judgment on the screen sharing picture based on the preset page configuration file, and judging whether the screen sharing picture is in a preset page or not;

If the screen sharing picture is not in the preset page, judging whether the screen sharing picture is in a page switching state or not;

If the screen sharing picture is in a page switching state, judging that the screen sharing picture is in intra-application switching, inter-application switching or application and homepage switching; if the screen sharing picture is switched between the application and the homepage, further judging whether the screen sharing picture is in the homepage or not;

and if the screen sharing picture is not in the preset page and page switching state, judging whether the screen sharing picture is in a homepage or not.

In a third aspect, the application further provides a screen recording device. The device comprises:

The page setting module is used for setting a page to be recorded in the target application as a preset page; the target application is an application needing to be recorded;

the configuration acquisition module is used for extracting characteristics based on the preset page and the target application to acquire a preset page configuration file;

the sending and receiving module is used for sending the screen sharing picture to the background streaming server in real time and receiving the current page judgment result sent by the background streaming server; the current page judgment result is generated after the background streaming server judges the preset page and switches frames of the screen sharing picture based on the preset page configuration file;

And the picture recording module is used for recording the screen sharing picture based on the current page judging result.

In a fourth aspect, the application further provides a screen recording device. The device comprises:

The configuration loading module is used for loading a preset page configuration file and receiving a screen sharing picture sent by the front-end screen recording application in real time;

the feature extraction module is used for extracting features of the screen sharing picture and obtaining a current page configuration file containing page features;

The page judging module is used for carrying out preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating a current page judging result;

And the result sending module is used for sending the current page judging result to the front-end screen recording application.

In a fifth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

In a sixth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

According to the screen recording method, device, computer equipment and storage medium, the page to be recorded in the target application is set to be the preset page, feature extraction is carried out based on the preset page and the target application, the preset page configuration file is obtained, the screen sharing picture is sent to the background streaming server in real time, the current page judgment result sent by the background streaming server is received, screen recording is carried out on the screen sharing picture based on the current page judgment result, screen recording is carried out on the current page only when the current page is in the preset page through carrying out preset page judgment and switching frame judgment, monitoring on application and page switching is achieved, and excessive collection of screen recording information of a user is avoided.

Drawings

FIG. 1 is a diagram of an application environment for a screen recording method in one embodiment;

FIG. 2 is a flowchart of a screen recording method according to an embodiment;

FIG. 3 is a flow chart of a feature extraction step in one embodiment;

FIG. 4 is a flowchart of a screen recording method according to another embodiment;

FIG. 5 is a flowchart of a screen recording method according to another embodiment;

FIG. 6 is a flowchart illustrating a switching frame determination step in one embodiment;

FIG. 7 is a block diagram of a screen recording apparatus in one embodiment;

FIG. 8 is a block diagram of a screen recording apparatus in one embodiment;

fig. 9 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The screen recording method provided by the embodiment of the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In one embodiment, as shown in fig. 2, a screen recording method is provided, and the method is applied to the terminal in fig. 1 for illustration, and includes the following steps:

step 202, setting a page to be recorded in a target application as a preset page; the target application is an application that needs to be recorded.

Specifically, an application needing to collect data in the screen is called a target application, and a page needing to collect data in the target application is called a preset page. Firstly, setting a target application which needs to be recorded on a screen in the screen recording process of a terminal, and arranging a designated page which needs to be recorded in the target application into a preset page.

And 204, extracting features based on the preset page and the target application, and obtaining a preset page configuration file.

Specifically, the target application and the image features and text features of the preset page are analyzed, and a corresponding preset page configuration file is generated according to the acquired text features and image features. Extracting characteristics of a target application and a target page in the screen recording, and generating a preset page configuration file containing page characteristics. Fig. 3 is a schematic flow chart of a feature extraction step in an embodiment, as shown in fig. 3, in which page features in a preset page configuration file are divided into two dimensions, namely text features and image features. For the extraction of text features, the embodiment is not particularly limited, for example, text content and coordinate positions existing in an image may be extracted by OCR technology, and a corresponding regular expression for background matching and/or coordinate range points of corresponding keywords are generated as text features according to a keyword/keyword matching rule of a preset page.

For extraction of image features, the image features of the page are acquired through multiple dimensions, and the extraction of the image features is a multi-dimensional extraction, including but not limited to the following extraction modes: (1) Obtaining the distribution range of the color values of each pixel of a preset page and the color characteristics of a color interval of a main tone and the like by a method for calculating the color histogram and the color moment of the picture; (2) Extracting texture features of a preset page through a gray level co-occurrence matrix method (GLCM) and a Local Binary Pattern (LBP) operator; (3) Extracting feature map information of a preset page through a deep learning convolutional network (CNN), and generating a feature value or a feature vector after leveling or changing; (4) Judging whether the input page is a preset page or not by setting a specific image local area and an image sub-template as image features and utilizing a template matching and image similarity calculation mode; (5) And extracting characteristics such as edges, grids and the like through convolution kernels such as Sobel operators and the like.

Step 206, the screen sharing picture is sent to a background streaming server in real time, and the current page judgment result sent by the background streaming server is received; and the current page judgment result is generated after the background streaming server judges the preset page and switches frames of the screen sharing picture based on the preset page configuration file.

Specifically, screen sharing allows a user to share the screen of the user device or a particular application with a remote audience during a real-time session, and when the user opens the screen sharing, the front-end screen recording application loads the screen recording configuration, opening the screen recording session. And loading preset page configuration related in the screen recording session by the rear end according to the information in the screen recording configuration. The front-end screen recording application sends the screen sharing picture to a background streaming server in real time, the background streaming server transmits the pulled image to an image analysis service, and the current page judgment result sent by the background streaming server is received. The current page judgment result is generated after the background streaming server carries out preset page judgment and frame switching judgment on the screen sharing picture based on a preset page configuration file, and the preset page judgment is the detection of judging whether the current detection page is in the preset page or not; in the process of screen sharing, screen recording software captures screen content frame by frame, the captured video frame with switching action is called switching frame, switching frame judgment is that the user switches from a specific application to other applications or from the specific application to a main screen page or in different tab pages of the specific application in the process of screen sharing, switching action is generated by the user, and the switching action of the user is identified and detected through switching frame judgment. After preset page judgment and switching frame judgment are carried out on the screen sharing picture, the background streaming server sends the generated current page judgment result to the front end.

And step 208, recording the screen sharing picture based on the current page judging result.

Specifically, fig. 4 is a flowchart of a screen recording method in another embodiment, as shown in fig. 4, the back-end image analysis service performs feature extraction on an image, compares the extracted image features and/or text features with features in a preset page configuration, and determines whether the current shared page of the user is the preset page. If the current page is the preset page, continuing the screen recording flow. If the page is not the preset page, the current page is transmitted to a switching frame detection module, and the switching behavior of the user is judged. If the current user has the switching behavior, corresponding screen recording flow guidance is carried out according to different switching types. If the user opens a non-target application, the screen recording is terminated. And when the screen sharing picture is not applied to the target and the target page is displayed, blurring the acquired screen recording content or skipping the acquisition of the stage. And the starting point and the ending point can be calculated, and the non-target recording segment is deleted in the later period.

In the screen recording method, the page to be recorded in the target application is set as the preset page, the characteristic extraction is carried out based on the preset page and the target application, the preset page configuration file is obtained, the screen sharing picture is sent to the background streaming server in real time, the current page judgment result sent by the background streaming server is received, the screen is recorded on the screen sharing picture based on the current page judgment result, the screen recording is carried out only when the current page is in the preset page through carrying out the preset page judgment and the switching frame judgment, the monitoring on the application and the page switching is realized, and the excessive acquisition of the screen recording information of the user is avoided.

Specifically, when recording the screen sharing picture according to the current page judging result, if the screen sharing picture is a preset page, recording the screen of the current page; if the screen sharing picture is switched in the application, but is not in the preset picture, the user is prompted to return to the page to be recorded, and the prompting method can be voice, text popup frame and the like; if the screen sharing picture is switched between applications, the user is prompted to enter the target application at the moment because the user does not enter the target application at the moment; if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording; if the current page is switched between the application and the homepage and the current page is not in the homepage, the user is considered to be switched to the application irrelevant to the screen recording flow, and the screen recording is terminated at the moment.

In this embodiment, according to the judgment result of the current page, the screen is recorded on the screen sharing picture, when the screen sharing picture is a preset page, the screen is recorded on the current page, when the screen sharing picture is switched in the application, the user is prompted to return to the page to be recorded, when the screen sharing picture is switched between the application and the homepage, the user is prompted to enter the target application, when the screen sharing picture is switched between the application and the homepage, and the current page is switched between the application and the homepage, the user is prompted to return to the target application to record the screen, when the screen sharing picture is switched between the application and the homepage, and the current page is not in the homepage, the screen is stopped, and the screen is recorded on the screen sharing picture through the judgment result of the current page, so that the monitoring on the application and the page switching is realized, and the excessive collection of the screen recording information of the user is avoided.

Specifically, when the screen sharing picture is recorded according to the current page judging result, when the screen sharing picture is not in a preset page and page switching state, and the screen sharing picture is in a homepage, the page is not in the target application, and the user is prompted to return to the target application for recording; when the screen sharing picture is not in the preset page and page switching state and the screen sharing picture is not in the homepage, the user is considered to be switched to the application irrelevant to the screen recording flow, and the screen recording is terminated.

In this embodiment, when the screen sharing picture is not in the preset page and the page switching state and the screen sharing picture is in the home page, the user is prompted to return to the target application to record the screen, and when the screen sharing picture is not in the preset page and the page switching state and the screen sharing picture is not in the home page, the screen recording is terminated, and the screen recording is performed on the screen sharing picture through the judgment result of the current page, so that the monitoring on the application and the page switching is realized, and the excessive collection of the screen recording information of the user is avoided.

In one embodiment, as shown in fig. 5, a screen recording method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:

Step 502, loading a preset page configuration file, and receiving a screen sharing picture sent by a front-end screen recording application in real time.

Specifically, when a user starts screen sharing, the front-end screen recording application loads screen recording configuration and starts a screen recording session. And the back end loads preset page configuration related in the screen recording session according to the information in the screen recording configuration, and receives a screen sharing picture sent by the front end screen recording application in real time.

And step 504, extracting the characteristics of the screen sharing picture to obtain a current page configuration file containing page characteristics.

Specifically, feature extraction is performed on the screen sharing picture, the feature extraction includes text feature extraction and image feature extraction, and the embodiment is not limited specifically, for example, text content and coordinate positions existing in the image can be extracted through OCR technology, and a corresponding regular expression and/or coordinate range points of corresponding keywords/keywords for background matching are generated as text features according to a keyword/keyword matching rule of a preset page.

For extraction of image features, the image features of the page are acquired through multiple dimensions, and the extraction of the image features is a multi-dimensional extraction, including but not limited to the following extraction modes: (1) Obtaining the distribution range of the color values of each pixel of a preset page and the color characteristics of a color interval of a main tone and the like by a method for calculating the color histogram and the color moment of the picture; (2) Extracting texture features of a preset page through a gray level co-occurrence matrix method (GLCM) and a Local Binary Pattern (LBP) operator; (3) Extracting feature map information of a preset page through a deep learning convolutional network (CNN), and generating a feature value or a feature vector after leveling or changing; (4) Judging whether the input page is a preset page or not by setting a specific image local area and an image sub-template as image features and utilizing a template matching and image similarity calculation mode; (5) And extracting characteristics such as edges, grids and the like through convolution kernels such as Sobel operators and the like. After feature extraction, a current page configuration file containing page features is obtained according to the text features and the image features.

Step 506, performing preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating a current page judgment result.

Specifically, preset page judgment and switching frame judgment are performed on the current page configuration file based on the preset page configuration file, and a current page judgment result is obtained. In the screen recording process, common switching actions include: intra-application switching, inter-application switching, or inter-application and home page switching. Fig. 6 is a schematic flow chart of a switching frame judging step in one embodiment, as shown in fig. 6, when switching frames, the gradient change of the image along the X-axis is calculated (1) firstly by using gradient extraction operators such as the second derivative and LAPLACIAN, PREWITT, SOBEL of the image. The change in gradient can reflect whether the whole image has a tearing feeling or not, and can be used as a characteristic of the switching action. (2) And extracting the edge characteristics and the linear characteristics of the Y axis of the image, and distinguishing whether the inter-application switching and the intra-application label page switching occur or not according to the linear characteristics and the edge characteristics of the Y axis in the image according to different operation systems and system UI. Taking a Canny operator as an example, smoothing the image through a Gaussian filter, and extracting edge points from the double threshold values according to the gradient of the image through a gradient non-maximum suppression algorithm to obtain vertical edge characteristics. (3) Grid features in the image are calculated, and the Grid features (Grid features) can be obtained by respectively extracting horizontal and vertical straight line features of the image and then combining the features. (4) The image definition is calculated by Brenner, tenengrad equal gradient functions, SMD2 functions, vollath functions, EAV point sharpness algorithm functions and other methods. Judging whether the user switches the application to the desktop or not according to the definition degree of the image.

By judging the switching frame, we can judge the type of the switching frame, wherein a specific judgment algorithm is as follows:

(1) If the ambiguity score in the image is below the set threshold, and the sharpness of some regions in the cut image is significantly higher than other regions, then the switching action applied to the desktop is considered to have occurred.

(2) If the horizontal gradient in the image has larger change, the number of gradient abrupt changes is more than or equal to 2, the relative distance between the two abrupt changes is calculated, and if the relative distance is lower than a set threshold value, the switching between applications is considered to occur. If the relative distance is greater than the set threshold, then a switch between tab pages within the application is deemed to have occurred.

(3) If a large change in the horizontal gradient occurs in the image, but the number of gradient abrupt changes is less than 2, then a switch between application intra-label pages is considered to occur.

(4) For handover types that cannot be determined, a front-to-back frame comparison approach may be adopted. And caching the current frame, continuously pulling the next frame, and calculating the image similarity of the next frame and the cached frame. And calculating the image similarity through a perceptual hash algorithm, a key point matching algorithm and a structural similarity algorithm. If the similarity of the front frame and the rear frame is larger than a set threshold value, or if partial areas are completely consistent before the front frame and the rear frame, the current switching action is considered to be intra-application switching, otherwise, the switching action is considered to be inter-application switching.

In addition, the method for judging the switching frame can be based on a deep learning technology. The switching frame detection is regarded as an image classification task, the characteristics of different types of images are extracted through a convolutional neural network, and a set of switching frame data sets comprising a plurality of categories such as internal cutting application, external cutting application, homepage switching and the like is established. And training the deep learning network to obtain a corresponding detection model.

And step 508, sending the current page judging result to the front-end screen recording application.

Specifically, after preset page judgment and switching frame judgment are performed on the current page, the generated current page judgment result is sent to the front-end screen recording application.

In this embodiment, a preset page configuration file is loaded, a screen sharing picture sent by a front-end screen recording application in real time is received, feature extraction is performed on the screen sharing picture, a current page configuration file containing page features is obtained, preset page judgment and switching frame judgment are performed on the current page configuration file based on the preset page configuration file, a current page judgment result is generated, and the current page judgment result is sent to the front-end screen recording application, so that judgment on the current page is realized, and the problem that a non-target application and a non-target picture cannot be shared and recorded while the target picture of the target application cannot be acquired in a complex scene is solved.

For extracting image features, acquiring the image features of the page through multiple dimensions, including acquiring the image features of multiple dimensions of a screen sharing picture; the image features of the multiple dimensions comprise at least one of color features, texture features, feature vectors, feature values, edge features, grid features, image local areas and image sub-templates, and the current page configuration file is generated based on text features and image features of the screen sharing picture.

In this embodiment, text feature extraction is performed on the screen sharing picture, text features of the screen sharing picture are obtained, image feature extraction is performed on the screen sharing picture, image features of multiple dimensions of the screen sharing picture are obtained, and a current page configuration file is obtained by extracting the text features and the image features, so that feature extraction is realized, and conditions are created for further switching frame judgment and preset page judgment.

In one embodiment, the performing, based on the preset page configuration file, preset page judgment and switch frame judgment on the current page configuration file, and generating a current page judgment result includes:

Specifically, when the preset page judgment and the switching frame judgment are performed on the screen sharing picture, by the switching frame judgment, we can judge the type of the switching frame, wherein a specific judgment algorithm is as follows: the method comprises the steps of carrying out preset page judgment on a screen sharing picture based on a preset page configuration file, judging whether the screen sharing picture is in a preset page, and judging whether the screen sharing picture is in a page switching state if the screen sharing picture is not in the preset page, wherein the specific judgment method comprises the following steps: (1) If the ambiguity score in the image is below the set threshold, and the sharpness of some regions in the cut image is significantly higher than other regions, then the switching action applied to the desktop is considered to have occurred. (2) If the horizontal gradient in the image has larger change, the number of gradient abrupt changes is more than or equal to 2, the relative distance between the two abrupt changes is calculated, and if the relative distance is lower than a set threshold value, the switching between applications is considered to occur. If the relative distance is greater than the set threshold, then a switch between tab pages within the application is deemed to have occurred. (3) If a large change in the horizontal gradient occurs in the image, but the number of gradient abrupt changes is less than 2, then a switch between application intra-label pages is considered to occur.

If the screen sharing picture is in the page switching state, judging whether the screen sharing picture is in the intra-application switching, the inter-application switching or the inter-application and homepage switching, if the screen sharing picture is in the inter-application and homepage switching, further judging whether the screen sharing picture is in the homepage, if the screen sharing picture is not in the preset page and page switching state, judging whether the screen sharing picture is in the homepage, wherein the judging method for judging whether the screen sharing picture is in the homepage is as follows: and counting the grid number of the image by utilizing the grid characteristics of the image, and considering that the current page of the user is a mobile phone homepage when the grid number is higher than a set threshold value and the size of each grid is similar, otherwise, the screen sharing picture is not in the homepage.

In this embodiment, the preset page judgment is performed on the screen sharing picture based on the preset page configuration file, whether the screen sharing picture is on the preset page is judged, if the screen sharing picture is not on the preset page, whether the screen sharing picture is on the page switching state is judged, if the screen sharing picture is on the page switching state, whether the screen sharing picture is on the intra-application switching, the inter-application switching or the inter-application and home page switching is judged, if the screen sharing picture is on the inter-application and home page switching is further judged, if the screen sharing picture is not on the preset page and the page switching state, whether the screen sharing picture is on the home page is judged, real-time judgment on the screen sharing picture is realized, meanwhile, monitoring on the application and the page switching is realized, and excessive collection of user screen recording information is avoided.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a screen recording device for realizing the screen recording method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in one or more embodiments of the screen recording device provided below may refer to the limitation of the screen recording method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 7, there is provided a screen recording apparatus 700, comprising: a page setting module 701, a configuration obtaining module 702, a sending and receiving module 703, and a picture recording module 704, wherein:

The page setting module 701 is configured to set a page to be recorded in the target application as a preset page; the target application is an application that needs to be recorded.

The configuration obtaining module 702 is configured to obtain a preset page configuration file based on feature extraction of the preset page and the target application.

The sending and receiving module 703 is configured to send a screen sharing picture to a background streaming server in real time, and receive a current page judgment result sent by the background streaming server; and the current page judgment result is generated after the background streaming server judges the preset page and switches frames of the screen sharing picture based on the preset page configuration file.

And the picture recording module 704 is configured to record the screen sharing picture based on the current page judgment result.

In one embodiment, the picture recording module 704 is specifically configured to: if the screen sharing picture is a preset page, recording the screen of the current page; if the screen sharing picture is switched in the application, prompting the user to return to the page to be recorded; if the screen sharing picture is switched between applications, prompting a user to enter a target application; if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording; if the current page is in the switch between the application and the homepage and the current page is not in the homepage, terminating the screen recording.

In one embodiment, the picture recording module 704 is further configured to: if the screen sharing picture is not in a preset page and page switching state and the screen sharing picture is in a homepage, prompting a user to return to a target application for recording; if the screen sharing picture is not in the preset page and page switching state, and the screen sharing picture is not in the homepage, ending the screen recording.

In one embodiment, as shown in fig. 8, there is provided a screen recording apparatus 800 comprising: a configuration loading module 801, a feature extraction module 802, a page judging module 803 and a result sending module 804, wherein:

The configuration loading module 801 is configured to load a preset page configuration file, and receive a screen sharing picture sent by the front-end screen recording application in real time;

The feature extraction module 802 is configured to perform feature extraction on the screen sharing picture, and obtain a current page configuration file containing page features;

The page judging module 803 is configured to perform preset page judgment and switch frame judgment on the current page configuration file based on the preset page configuration file, and generate a current page judging result;

And a result sending module 804, configured to send the current page judgment result to the front-end screen recording application.

In one embodiment, the feature extraction module 802 is specifically configured to: extracting text features of the screen sharing picture to obtain the text features of the screen sharing picture; the text features are regular expressions and/or coordinate range points of keywords which can be matched by the background; extracting image features of the screen sharing picture to obtain image features of multiple dimensions of the screen sharing picture; the image features of the multiple dimensions comprise at least one of color features, texture features, feature vectors, feature values, edge features, grid features, image local areas and image sub-templates; and generating a current page configuration file based on the text characteristics and the image characteristics of the screen sharing picture.

In one embodiment, the page determining module 803 is specifically configured to: carrying out preset page judgment on the screen sharing picture based on the preset page configuration file, and judging whether the screen sharing picture is in a preset page or not; if the screen sharing picture is not in the preset page, judging whether the screen sharing picture is in a page switching state or not; if the screen sharing picture is in a page switching state, judging that the screen sharing picture is in intra-application switching, inter-application switching or application and homepage switching; if the screen sharing picture is switched between the application and the homepage, further judging whether the screen sharing picture is in the homepage or not; and if the screen sharing picture is not in the preset page and page switching state, judging whether the screen sharing picture is in a homepage or not.

The above-mentioned screen recording device can be implemented by all or part of the modules by software, hardware and their combination. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 9. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a screen recording method.

It will be appreciated by persons skilled in the art that the architecture shown in fig. 9 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:

In one embodiment, the processor when executing the computer program further performs the steps of: if the screen sharing picture is a preset page, recording the screen of the current page; if the screen sharing picture is switched in the application, prompting the user to return to the page to be recorded; if the screen sharing picture is switched between applications, prompting a user to enter a target application; if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording; if the current page is in the switch between the application and the homepage and the current page is not in the homepage, terminating the screen recording.

In one embodiment, the processor when executing the computer program further performs the steps of: if the screen sharing picture is not in a preset page and page switching state and the screen sharing picture is in a homepage, prompting a user to return to a target application for recording; if the screen sharing picture is not in the preset page and page switching state, and the screen sharing picture is not in the homepage, ending the screen recording.

In one embodiment, the processor when executing the computer program further performs the steps of: loading a preset page configuration file, and receiving a screen sharing picture sent by a front-end screen recording application in real time; extracting the characteristics of the screen sharing picture to obtain a current page configuration file containing page characteristics; performing preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating a current page judgment result; and sending the current page judging result to the front-end screen recording application.

In one embodiment, the processor when executing the computer program further performs the steps of: extracting text features of the screen sharing picture to obtain the text features of the screen sharing picture; the text features are regular expressions and/or coordinate range points of keywords which can be matched by the background; extracting image features of the screen sharing picture to obtain image features of multiple dimensions of the screen sharing picture; the image features of the multiple dimensions comprise at least one of color features, texture features, feature vectors, feature values, edge features, grid features, image local areas and image sub-templates; and generating a current page configuration file based on the text characteristics and the image characteristics of the screen sharing picture.

In one embodiment, the processor when executing the computer program further performs the steps of: carrying out preset page judgment on the screen sharing picture based on the preset page configuration file, and judging whether the screen sharing picture is in a preset page or not; if the screen sharing picture is not in the preset page, judging whether the screen sharing picture is in a page switching state or not; if the screen sharing picture is in a page switching state, judging that the screen sharing picture is in intra-application switching, inter-application switching or application and homepage switching; if the screen sharing picture is switched between the application and the homepage, further judging whether the screen sharing picture is in the homepage or not; and if the screen sharing picture is not in the preset page and page switching state, judging whether the screen sharing picture is in a homepage or not.

According to the computer equipment, the page to be recorded in the target application is set as the preset page, the feature extraction is carried out based on the preset page and the target application, the preset page configuration file is obtained, the screen sharing picture is sent to the background streaming server in real time, the current page judgment result sent by the background streaming server is received, the screen is recorded on the screen sharing picture based on the current page judgment result, the screen recording is carried out only when the current page is in the preset page through carrying out the preset page judgment and the switching frame judgment on the current page, the monitoring on the application and the page switching is realized, and the excessive acquisition of the screen recording information of a user is avoided.

In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of: if the screen sharing picture is a preset page, recording the screen of the current page; if the screen sharing picture is switched in the application, prompting the user to return to the page to be recorded; if the screen sharing picture is switched between applications, prompting a user to enter a target application; if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording; if the current page is in the switch between the application and the homepage and the current page is not in the homepage, terminating the screen recording.

In one embodiment, the computer program when executed by the processor further performs the steps of: if the screen sharing picture is not in a preset page and page switching state and the screen sharing picture is in a homepage, prompting a user to return to a target application for recording; if the screen sharing picture is not in the preset page and page switching state, and the screen sharing picture is not in the homepage, ending the screen recording.

In one embodiment, the computer program when executed by the processor further performs the steps of: loading a preset page configuration file, and receiving a screen sharing picture sent by a front-end screen recording application in real time; extracting the characteristics of the screen sharing picture to obtain a current page configuration file containing page characteristics; performing preset page judgment and switching frame judgment on the current page configuration file based on the preset page configuration file, and generating a current page judgment result; and sending the current page judging result to the front-end screen recording application.

In one embodiment, the computer program when executed by the processor further performs the steps of: extracting text features of the screen sharing picture to obtain the text features of the screen sharing picture; the text features are regular expressions and/or coordinate range points of keywords which can be matched by the background; extracting image features of the screen sharing picture to obtain image features of multiple dimensions of the screen sharing picture; the image features of the multiple dimensions comprise at least one of color features, texture features, feature vectors, feature values, edge features, grid features, image local areas and image sub-templates; and generating a current page configuration file based on the text characteristics and the image characteristics of the screen sharing picture.

In one embodiment, the computer program when executed by the processor further performs the steps of: carrying out preset page judgment on the screen sharing picture based on the preset page configuration file, and judging whether the screen sharing picture is in a preset page or not; if the screen sharing picture is not in the preset page, judging whether the screen sharing picture is in a page switching state or not; if the screen sharing picture is in a page switching state, judging that the screen sharing picture is in intra-application switching, inter-application switching or application and homepage switching; if the screen sharing picture is switched between the application and the homepage, further judging whether the screen sharing picture is in the homepage or not; and if the screen sharing picture is not in the preset page and page switching state, judging whether the screen sharing picture is in a homepage or not.

According to the storage medium, the page to be recorded in the target application is set as the preset page, the feature extraction is carried out based on the preset page and the target application, the preset page configuration file is obtained, the screen sharing picture is sent to the background streaming server in real time, the current page judgment result sent by the background streaming server is received, the screen is recorded on the screen sharing picture based on the current page judgment result, and the screen recording is carried out only when the current page is in the preset page through carrying out the preset page judgment and the switching frame judgment on the current page, so that the monitoring on the application and the page switching is realized, and the excessive acquisition of the screen recording information of a user is avoided.

The user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A screen recording method, characterized in that it is applied to a terminal, the method comprising:

the method comprises the steps of sending a screen sharing picture captured frame by frame to a background streaming server in real time, and receiving a current page judgment result sent by the background streaming server; the current page judging result comprises a preset page judging result and a switching action type judging result, the background streaming server judges the preset page of the screen sharing picture based on the preset page configuration file, and in the case that the screen sharing picture is a switching frame, the switching action type of the switching frame is judged and generated according to at least one of the ambiguity score of the switching frame, the change condition of the horizontal gradient of the switching frame, the similarity of the switching frame and the next frame and the image characteristics of the switching frame; wherein, the step of carrying out preset page judgment on the screen sharing picture refers to judging whether the screen sharing picture is a preset page or not;

recording the screen sharing picture based on the current page judging result;

The recording the screen sharing picture based on the current page judging result comprises the following steps:

2. The method of claim 1, wherein the recording the screen sharing frame based on the current page determination result further comprises:

3. A screen recording method, applied to a server, comprising:

Loading a preset page configuration file, and receiving a screen sharing picture which is captured frame by frame and sent in real time by a front-end screen recording application;

Performing preset page judgment on the current page configuration file based on the preset page configuration file, and performing switching action type judgment of the switching frame according to at least one of a fuzzy degree score of the switching frame, a change condition of a horizontal gradient of the switching frame, similarity of the switching frame and a next frame and image characteristics of the switching frame when the screen sharing picture is the switching frame; the preset page configuration file is generated by extracting features based on a preset page and a target application, wherein the preset page is a page to be recorded in the target application, and the target application is an application to be recorded; the preset page judgment means to judge whether the screen sharing picture is a preset page or not;

Generating a current page judgment result comprising a preset page judgment result and a switching action type judgment result;

Sending the current page judgment result to the front-end screen recording application to instruct the front-end screen recording application to record the screen of the screen sharing picture based on the current page judgment result;

The recording the screen sharing picture based on the current page judging result comprises the following steps: if the screen sharing picture is a preset page, recording the screen of the current page; if the screen sharing picture is switched in the application, prompting the user to return to the page to be recorded; if the screen sharing picture is switched between applications, prompting a user to enter a target application; if the current page is switched between the application and the homepage and the current page is at the homepage, prompting the user to return to the target application for screen recording; if the current page is in the switch between the application and the homepage and the current page is not in the homepage, terminating the screen recording.

4. The method of claim 3, wherein the extracting features of the screen sharing frame to obtain a current page profile including page features comprises:

5. The method according to claim 4, wherein the method further comprises:

6. A screen recording apparatus, deployed in a terminal, the apparatus comprising:

The sending and receiving module is used for sending the screen sharing pictures captured frame by frame to the background streaming server in real time and receiving the current page judgment result sent by the background streaming server; the current page judging result comprises a preset page judging result and a switching action type judging result, the background streaming server judges the preset page of the screen sharing picture based on the preset page configuration file, and in the case that the screen sharing picture is a switching frame, the switching action type of the switching frame is judged and generated according to at least one of the ambiguity score of the switching frame, the change condition of the horizontal gradient of the switching frame, the similarity of the switching frame and the next frame and the image characteristics of the switching frame; wherein, the step of carrying out preset page judgment on the screen sharing picture refers to judging whether the screen sharing picture is a preset page or not;

the picture recording module is used for recording the screen sharing picture based on the current page judging result;

The picture recording module is specifically configured to:

7. The apparatus of claim 6, wherein the device comprises a plurality of sensors,

The screen recording module is further configured to prompt a user to return to a target application for recording a screen if the screen sharing screen is not in a preset page and page switching state and the screen sharing screen is in a home page;

8. A screen recording apparatus deployed in a server, the apparatus comprising:

The configuration loading module is used for loading a preset page configuration file, and receiving a screen sharing picture captured frame by real-time transmission of a front-end screen recording application;

The page judging module is used for carrying out preset page judgment on the current page configuration file based on the preset page configuration file, and judging the switching action type of the switching frame according to at least one of the ambiguity score of the switching frame, the change condition of the horizontal gradient of the switching frame, the similarity of the switching frame and the next frame and the image characteristic of the switching frame when the screen sharing picture is the switching frame; the preset page configuration file is generated by extracting features based on a preset page and a target application, wherein the preset page is a page to be recorded in the target application, and the target application is an application to be recorded; the preset page judgment means to judge whether the screen sharing picture is a preset page or not;

the page judging module is further used for generating a current page judging result comprising a preset page judging result and a switching action type judging result;

the result sending module is used for sending the current page judging result to the front-end screen recording application so as to instruct the front-end screen recording application to record the screen of the screen sharing picture based on the current page judging result;

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 5.