CN116744046A

CN116744046A - Method, device, equipment and storage medium for protecting screen display content

Info

Publication number: CN116744046A
Application number: CN202310515342.6A
Authority: CN
Inventors: 杜玮; 何沂娟
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-05-08
Filing date: 2023-05-08
Publication date: 2023-09-12

Abstract

The application discloses a method, a device, equipment and a storage medium for protecting screen display content, and belongs to the technical fields of Internet and computers. The method comprises the following steps: determining at least one control to be processed with content to be protected from at least one control included in a user interface; creating an aggregation control comprising to-be-protected content of at least one to-be-processed control; performing video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control; in the process of displaying the user interface, displaying other controls except the control to be processed in at least one control in the user interface, and playing the video in the user interface through a player; and responding to the recording operation of the screen display content, acquiring a recorded image of the screen display content, wherein the recorded image comprises a user interface and the video played by the player is hidden. The method is helpful for avoiding the occurrence of the content to be protected in the recorded image obtained by the screen capturing and recording operation of the user interface.

Description

Method, device, equipment and storage medium for protecting screen display content

Technical Field

The present application relates to the field of internet and computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for protecting screen display content.

Background

Because the screen display content in the user interface includes some content to be protected, when performing operations such as screenshot or screen recording on the screen display content, it is necessary to avoid that the content to be protected appears in the recorded image of the screen display content.

In the related art, in the process of generating a recorded image of the screen display content, a terminal device adds a watermark in the recorded image of the screen display content, and reflects account information representing the recording operation of the screen display content through the watermark, so that the source tracing is performed according to the watermark in the leaked recorded image under the condition that the content to be protected is leaked.

However, the recorded image of the on-screen content obtained by the method still contains the content to be protected, so that the security of displaying the content to be protected on the user interface is poor.

Disclosure of Invention

The application provides a method, a device, equipment and a storage medium for protecting screen display content. The technical scheme is as follows:

according to an aspect of an embodiment of the present application, there is provided a method for protecting content displayed on a screen, the method including:

Determining at least one control to be processed with content to be protected from at least one control included in a user interface;

creating an aggregation control comprising the content to be protected of the at least one control to be processed;

performing video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control;

displaying other controls except the control to be processed in the at least one control in the user interface in the process of displaying the user interface, and playing the video in the user interface through a player;

and responding to the recording operation of the screen display content, acquiring a recorded image of the screen display content, wherein the recorded image comprises the user interface and the video played by the player is hidden.

According to an aspect of an embodiment of the present application, there is provided a protection apparatus for on-screen content, the apparatus including:

the control acquisition module is used for determining at least one control to be processed with the content to be protected from at least one control included in the user interface;

the control aggregation module is used for creating an aggregation control comprising the content to be protected of the at least one control to be processed;

The video generation module is used for carrying out video conversion processing on the aggregation control and generating a video for simulating the display content of the aggregation control;

the interface display module is used for displaying other controls except the control to be processed in the at least one control in the user interface in the process of displaying the user interface, and playing the video in the user interface through a player;

and the image acquisition module is used for responding to the recording operation aiming at the user interface and acquiring a recorded image of the screen display content, wherein the recorded image comprises the user interface and the video played by the player is hidden.

According to an aspect of an embodiment of the present application, there is provided a terminal device including a processor and a memory, the memory storing a computer program loaded and executed by the processor to implement the method of protecting on-screen content as described above.

According to an aspect of an embodiment of the present application, there is provided a computer-readable storage medium having stored therein a computer program loaded and executed by a processor to implement the method of protecting on-screen content as described above.

According to an aspect of an embodiment of the present application, there is provided a computer program product including a computer program stored in a computer-readable storage medium, from which a processor reads and executes the computer program to implement the method of protecting on-screen content as described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that at least:

the method comprises the steps of generating an aggregation control based on at least one control to be processed, and generating a video according to the aggregation control so as to play the video in a user interface, and simulating the display content of the at least one control to be processed in the user interface. In the process of recording the screen display content, the terminal equipment hides the video in the player, so that the content to be protected does not appear in the recorded image, the leakage of the content to be protected is avoided, and the safety of the content to be protected is improved.

In addition, under the condition that the aggregation control comprises a plurality of to-be-processed controls, videos corresponding to the aggregation control are generated to replace videos corresponding to the plurality of to-be-processed controls respectively, so that the number of videos to be generated and the number of players used for playing the videos are reduced, occupation of system resources of the terminal equipment by the method is reduced, blocking of the terminal equipment due to generation or playing of the videos is avoided, and the use smoothness of the terminal equipment is improved.

Drawings

FIG. 1 is a schematic illustration of an implementation environment for an embodiment of the present application;

FIG. 2 is a flow chart of a method for protecting content on screen provided by an exemplary embodiment of the present application;

FIG. 3 is a diagram of a comparison of on-screen content and recorded images provided by an exemplary embodiment of the present application;

FIG. 4 is a diagram of on-screen content and recorded images provided in accordance with another exemplary embodiment of the present application;

FIG. 5 is an effect diagram of an aggregation control provided by an exemplary embodiment of the present application;

FIG. 6 is a diagram of a display of controls in a user interface provided in accordance with an exemplary embodiment of the present application;

FIG. 7 is a schematic diagram of a control management object provided by an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of determining a common parent control provided by an exemplary embodiment of the present application;

FIG. 9 is a schematic diagram of a screenshot protection interface provided by an exemplary embodiment of the application;

FIG. 10 is a schematic diagram of a video loading process provided by an exemplary embodiment of the present application;

FIG. 11 is a block diagram of an apparatus for protecting content on a screen provided by an exemplary embodiment of the present application;

fig. 12 is a block diagram of a terminal device according to an exemplary embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of an implementation environment for an embodiment of the present application.

The terminal device 10 may be an electronic device such as a mobile phone, a tablet computer, a PC (Personal Computer ), a game console, a multimedia playing device, a wearable device, an intelligent voice interaction device, an intelligent home appliance, a car-mounted terminal, etc.

In one example, the terminal device 10 has a target application running therein, the target application having a User Interface including a UI (User Interface) control. By performing display processing on the user interface control, display content in the user interface can be realized. Because some user interface controls for displaying the content to be protected (i.e., the controls to be processed in the following embodiments) exist in the user interface controls, during the recording operations of screenshot, screen recording, etc. on the screen display content, it is necessary to avoid the content to be protected from appearing in the recorded image as much as possible, so as to reduce the possibility of leakage of the content to be protected.

In one example, the target application is office software, and the target application is used for waiting for protection of content line management for organization information, project information, file information of an enterprise (or organization, unit, etc.).

In addition, the target application program may also be a cloud storage application program, a multimedia presentation application program, a video application program, a shopping application program, a social application program, an interactive entertainment application program, a browser application program, a content sharing application program, a virtual reality application program, an augmented reality application program, and the like, which is not limited in the embodiment of the present application. In addition, for different application programs, the corresponding functions may be different, so that the types of the content to be protected are not completely the same, and may be configured in advance according to actual requirements, which is not limited by the embodiment of the present application.

The server 20 is used to provide background services for clients of target applications in the terminal device 10. For example, the server 20 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, secure service content distribution network, (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platform, but is not limited thereto. The server 20 has at least a data receiving function and a calculating function.

In one example, where the terminal device 10 and the server 20 are in direct communication connection, a user using the target application may send a content protection request to the server 20 via the terminal device 10, where the content protection request is for requesting processing of at least one control to be processed for having content to be protected, so as to avoid that the content to be protected appears in the recorded image of the on-screen content. The server 20 instructs to start the content protection function to at least one terminal device 10 in which the target application is installed, according to the content protection request.

Fig. 2 is a flowchart of a method for protecting content displayed on a screen according to an exemplary embodiment of the present application. The execution body of the method may be, for example, the terminal device 10 in the implementation environment of the solution described in fig. 1, and the execution body may be, for example, a client of a target application running on the terminal device 10. As shown in fig. 2, the method may include the following steps (210-250):

at step 210, at least one control to be processed having content to be protected is determined from at least one control included in the user interface.

In some embodiments, at least one control belongs to a user interface control, i.e., a UI control. The UI control has display content. Optionally, the display content types include, but are not limited to, at least one of: text, pictures, video and audio. Wherein the picture information includes a still picture and a moving picture.

For example, the at least one control includes a control for displaying text information that characterizes the name of a file in a list of files displayed by the user interface. For another example, the at least one control includes a control for displaying a picture, the control being for displaying a picture stored in the gallery.

In some embodiments, the display content may be static content or dynamic content; the dynamic content comprises dynamic pictures and dynamic words. For example, the at least one control includes a control for displaying dynamic content.

Optionally, the dynamic picture is composed of a plurality of image frames, and the display effect of the dynamic picture is realized by playing the plurality of image frames through the control. Alternatively, dynamic text refers to the display effect of text content that changes over time. For example, the font size of the text content changes with time, the text color of the text content changes with time, or the display position of the text changes with time.

In some embodiments, the at least one control includes a control to be processed; the to-be-processed control is a UI control for displaying to-be-protected content. Optionally, the content to be protected is called sensitive information, and the control to be processed is called sensitive information control. The control to be processed is processed, which is equivalent to the desensitization processing of sensitive information.

In some embodiments, the content to be protected refers to the display content which is not desired to be obtained from the user interface by means of screenshot, screen recording and the like, that is, in order to promote the safety of the display of the content to be protected, the content to be protected should be able to be normally displayed in the user interface, but should not appear in the recorded image of the screen display content; the recorded image is obtained through screenshot or screen recording means.

Corresponding to the content to be protected is unprotected content, which refers to the display content that can appear in the recorded image corresponding to the user interface. Optionally, the control for displaying the unprotected content is called a non-sensitive control, and among at least one control included in the user interface, other controls except the control to be processed are all non-sensitive controls.

In some embodiments, the type of content to be protected includes at least one of: text, pictures, video and audio. For example, for a certain file list, the file list includes at least one file identification information, where the at least one file identification information is to-be-protected content, and a control for displaying the file identification information is to-be-protected control; wherein the file identification information includes at least one of: file name, file creation date, file type identification, file icon, etc.

For another example, for a multimedia gallery, the content to be protected refers to a first picture in the multimedia gallery, the control for displaying the first picture is a control to be processed, and the first picture may be any picture in the multimedia gallery.

In some embodiments, the at least one control to be processed is a control that is different from each other: that is, for any two of the at least one to-be-processed controls, the display positions of the two to-be-processed controls in the user interface do not completely overlap, or the to-be-protected contents of the two to-be-processed controls are different.

Optionally, the content to be protected is set by a user, and the content to be protected set by different users is not identical, that is, for the same user interface in the target application program, the user interfaces are not identical in the corresponding to the to-be-processed controls in different terminal devices.

In one example, the terminal device sets a first picture in the multimedia gallery as the content to be protected in response to the content to be protected setting operation. Content setting operations to be protected include, but are not limited to: click operation, long press operation, slide operation, key operation, and the like for the first picture.

Alternatively, the content to be protected is set by the administrator of the target application, i.e. the content to be protected is the same for any user using the target application. For example, the target application belongs to a resource sharing application, and after the administrator starts the content protection function, in the process of recording the screen display content in the target application, the terminal device determines, based on the content protection function, which controls in the user interface are to-be-protected controls having to-be-protected content.

In some embodiments, the terminal device includes a control management object, where the control management object is used to manage at least one control to be processed; and the terminal equipment determines at least one control to be processed through the control management object. Optionally, the control management object includes all the pending controls involved in the user interface.

Optionally, because the controls included in different user interfaces are not identical, each user interface corresponds to one control management object, and all the to-be-processed controls included in the user interface are registered in the control management object corresponding to the user interface, or all the to-be-processed controls related to the target application program are registered in the control management object. Optionally, the control management object is referred to as a screenshot management class.

For example, the terminal device determines at least one control to be processed from at least one control included in the user interface, including: and the terminal equipment selects at least one control to be processed from the control to be processed registered in the control management object.

At step 220, an aggregated control is created that includes the content to be protected that the at least one control to be processed has.

In some embodiments, the aggregation control includes at least one to-be-processed control with to-be-protected content that means: the aggregation control is used for displaying the content to be protected, which is respectively included by at least one control to be processed. That is, the display content of the aggregation control includes display content corresponding to each of the at least one to-be-processed control.

In some embodiments, the aggregate control includes at least one control to be processed.

Optionally, the terminal device creates the aggregation control based on all of the at least one pending control. Assuming that the terminal equipment determines n to-be-processed controls from at least one control included in the user interface, the aggregation control includes n to-be-processed controls, and n is a positive integer.

Optionally, the terminal device creates the aggregation control based on a part of the at least one to-be-processed control. The terminal equipment determines n to-be-processed controls from at least one control included in the user interface; the terminal equipment creates an aggregation control 1 based on m to-be-processed controls in the n to-be-processed controls; the terminal equipment creates an aggregation control 2 based on the other to-be-processed controls except m to-be-processed controls in the n to-be-processed controls, wherein m is a positive integer smaller than n.

Under the condition that the terminal equipment creates more than one aggregation control based on at least one to-be-processed control, the to-be-processed controls respectively included in different aggregation controls are different. The terminal device needs to perform step 230 separately for each of the more than one aggregation control.

Optionally, in order to reduce the creation of the aggregation control and the occupation of system resources of the terminal device in the subsequent processing of the aggregation control, there is an upper limit on the number of the aggregation controls generated based on at least one to-be-processed control, for example, the terminal device generates (n/2-1) at most (n/2-1) aggregation controls based on n to-be-processed controls, that is, each aggregation control includes at least two to-be-processed controls.

Optionally, the relative positions of the n pending controls in the aggregated control are consistent with the relative positions (which may also be referred to as distribution positions) of the n pending controls in the user interface. It should be noted that in the embodiment provided by the present application, the to-be-processed controls are not displayed on the user interface, and the relative positions of the n to-be-processed controls in the user interface referred to in the foregoing description may be determined according to the position and size information corresponding to the n to-be-processed controls, and it is not necessary to determine the relative positions of the n to-be-processed controls in the user interface by displaying the n to-be-processed controls in the user interface.

In some embodiments, the display size of the aggregate control is greater than the display size respectively corresponding to any one of the to-be-processed controls. Optionally, the display size of the aggregation control can cover the display size corresponding to each of the to-be-processed controls, and for the method for determining the display size of the aggregation control, please refer to the following embodiments.

In some embodiments, n pending controls are registered in the control management object, and the control management object selects m pending controls from the n pending controls, including: and the control management object selects m to-be-processed controls from the n to-be-processed controls according to the distribution positions of the n to-be-processed controls in the user interface. For example, the control management object selects m to-be-processed controls with adjacent distribution positions from n to-be-processed controls, generates an aggregate control including to-be-protected contents of the m to-be-processed controls respectively, n is a positive integer, and m is a positive integer smaller than or equal to n.

Since a plurality of pending controls are typically included in the user interface, an aggregate control including the plurality of pending controls is created in step 220 of the method, such that only the aggregate control needs to be correspondingly operated in subsequent steps 230-250. Compared with the method for processing each control to be processed independently, the method for protecting the screen display content simplifies the execution times of repeated steps in the execution process of the method for protecting the screen display content, is beneficial to saving system resources in terminal equipment and reduces the condition of blocking of the terminal equipment.

And 230, performing video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control.

In some embodiments, the video conversion process is used to process the aggregation control to generate video related to the aggregation control. Optionally, the video conversion process includes at least two steps of: performing picture conversion processing on the aggregation control to generate at least one picture corresponding to the aggregation control; based on at least one picture, a video is generated for simulating the display of the aggregation control, for specific steps of the process, see the following examples.

In some embodiments, the aggregate control display content includes display content corresponding to at least one control to be processed, respectively. As can be seen from the above, the aggregate control includes the content to be protected of at least one control to be processed, that is, it is assumed that at least one control to be processed is two controls to be processed, namely, a control to be processed 1 and a control to be processed respectively; the to-be-processed control 1 has to-be-protected content 1, and the to-be-processed control 2 has to-be-protected content 2, and the aggregate control display content comprises the to-be-protected control 1 and the to-be-protected control 2.

Optionally, the aggregate control display content includes at least one display style corresponding to the content to be protected, in addition to the content to be protected. The display style of the content to be protected is related to a control attribute of the control to be processed, and the control attribute is used for indicating the display style of the content to be protected on the user interface.

In some embodiments, the video for simulating the aggregate control display content includes at least one video frame. Optionally, if at least one control to be processed does not include a control to be processed for displaying dynamic information, the video includes a video frame, and the video frame is equivalent to obtaining a screenshot of the aggregation control; if at least one control to be processed comprises a first control to be processed for displaying dynamic content, the video comprises a plurality of video frames, the display content corresponding to the first control to be processed in the plurality of video frames is not identical, and in the video playing process, the display effect of the dynamic content in the first control to be processed is simulated by playing the plurality of video frames.

In some embodiments, the display size of the video frames is the same as the display size of the aggregation control, i.e., the display size of the video is the same as the display size of the aggregation control. For example, if the display size of the aggregation control is 20×100 pixels, the display size of at least one video frame included in the video is 20×100 pixels.

The video obtained by performing video conversion processing on the aggregation control can simulate the display content of the aggregation control, so that the subsequent terminal equipment can be replaced by playing the video on a user interface, and the aggregation control or at least one control to be processed is displayed on the user interface, so that the content to be protected is conveniently hidden in the subsequent recording operation process, and the specific process of the step is referred to the following embodiment.

In some embodiments, the step of performing video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control is completed through a control management object in the terminal device. After generating the aggregation control based on at least one control to be processed, the control management object generates a video corresponding to the aggregation control, and for specific content of this step, please refer to the following embodiments.

Step 240, in the process of displaying the user interface, displaying other controls except for the control to be processed in at least one control in the user interface, and playing the video in the user interface through the player.

In some embodiments, the user interface refers to any one of the display interfaces of the target application. Optionally, the terminal device displays the user interface in response to a user interface display (or switching) operation. User interface display (or switching) operations include, but are not limited to, at least one of: click operation, slide operation, key operation, touch operation, and the like.

For example, the user interface display operation is a click operation for opening the target application program. For another example, the user interface switching operation is a sliding operation for switching a new user interface, and it is assumed that a file list is displayed in the user interface, and the terminal device updates the display content of the file list in response to the user interface switching operation; before updating the display content of the file list, the terminal equipment determines at least one control to be processed, creates an aggregation control comprising at least one control to be processed, generates a video for simulating the display content of the aggregation control by performing video conversion processing on the aggregation control, and plays the video in the file list through a player.

It should be noted that, the implementation method of the user interface display (or switching) operation is determined according to actual needs, and the present application is not limited herein.

In some embodiments, a player refers to a video playback control for playing video. Optionally, the player belongs to a user interface control. After the terminal device displays the player in the user interface, the player plays the video so as to simulate the display content of at least one control to be processed. For a specific method for displaying a player in a user interface by a terminal device, please refer to the following embodiments.

Optionally, the display position of the player in the user interface is the same as the display position of the aggregation control.

In some embodiments, the terminal device displays a user interface comprising: the terminal equipment performs display processing on other controls to generate display contents respectively corresponding to the other controls; the terminal device displays the player in the user interface and plays the video through the player.

For example, the user interface comprises 10 controls, and 6 controls in the 10 controls are to-be-processed controls; in the process of displaying the user interface, the terminal equipment displays 4 other controls except 6 to-be-processed controls in the 10 controls in the user interface, and plays the video in the user interface through the player. That is, the terminal device does not need to directly display the 6 to-be-processed controls in the user interface, but plays the video through the player instead of the display content of the 6 to-be-processed controls.

In step 250, in response to the recording operation for the on-screen content, a recorded image of the on-screen content is acquired, the recorded image includes a user interface, and the video played by the player is hidden.

In some embodiments, the recording operation for on-screen content is used to record a display of a user interface. Optionally, the recording operation for the on-screen content includes, but is not limited to, at least one of: and (5) screenshot operation and screen recording operation. The recording operation for the screen display content can be carried by an operating system of the terminal equipment, can be provided by a target application program, and can be realized by other application programs except the target application program.

In some embodiments, on-screen content refers to display content in a user interface. Further, the on-screen content refers to the entire display content included in the user interface. Optionally, the on-screen content includes, but is not limited to, at least one of: and the other controls respectively correspond to the display content and the video played in the player.

In some embodiments, the recorded image is used to reflect the display content of the user interface. Optionally, the recorded image includes display contents respectively corresponding to other controls, and does not include any video frame of the video.

To assist the user in quickly recording the on-screen content, the terminal device generally allows the user to record the on-screen content through an image recording operation. Because the screen display content comprises some sensitive information (i.e. the content to be protected), if the recorded image comprises the sensitive information in the user interface, information leakage of individuals such as users, enterprises and the like is easy to cause. Therefore, when generating a recorded image of the user interface, it is necessary to subject sensitive information in the user interface to a desensitization process through the above steps.

According to the method, the display content of the to-be-processed controls for displaying the to-be-protected content is simulated through the video played in the player, and when the on-screen content is recorded, the video played in the player is hidden in the recorded image, so that the to-be-protected content respectively provided by at least one to-be-processed control is not displayed in the recorded image, and the occurrence of leakage of the to-be-protected content from the recorded image can be avoided.

In some embodiments, the terminal device determines whether the video played in the player is a video generated based on the aggregation control; if the video played in the player is the video generated based on the aggregation control, any video frame which does not comprise the video in the image is recorded; if the video played in the player is not based on the video of the aggregation control device, a certain video frame including the video in the image is recorded.

Optionally, the terminal device obtains a recorded image of the screen content in response to a recording operation for the screen content, including: the terminal equipment determines whether the video played in the player needs to be hidden (i.e. the terminal equipment determines whether the video played in the player is generated based on the aggregation control), and if the video played in the player needs to be hidden, the terminal equipment records other display contents except the player in the screen display contents; the terminal device cancels recording of the video displayed in the player.

In some embodiments, the display locations of the display content of the other controls in the recorded image are the same as the display locations of the other controls in the user interface. If the player is displayed at a first position in the user interface and the first other of the other controls is displayed at a third position in the user interface, the content related to the video is not displayed at a second position corresponding to the first position in the recorded image, and the display content of the first display control is displayed at a fourth position corresponding to the third position in the recorded image.

Optionally, the second position in the recorded image does not display content, or the second position in the recorded image has content protection information displayed therein; the content protection information is used for prompting that the first position of the user interface originally displays the content to be protected, and the content to be protected is hidden in the recorded image.

In order to avoid the influence of the protection method of the screen display content on the display effect of the screen display content in the user interface, in the process of recording the screen display content by the terminal equipment, a player is still displayed in the user interface, and a video is played by the player. Only when the terminal equipment generates a recorded image of the screen display content in the background, the video played in the player is not recorded. The method ensures that the recorded image does not contain the content to be protected, does not influence the playing of the player in the user interface, and is beneficial to reducing the influence of the method on the display effect in the user interface.

Fig. 3 is a diagram showing a comparison of on-screen content and recorded images provided by another exemplary embodiment of the present application.

The control for displaying the picture in fig. 3 belongs to a control to be processed, and after the user interface is subjected to screenshot, the picture is not displayed in the recorded image.

Fig. 4 is a diagram of a comparison of on-screen content and recorded images provided by an exemplary embodiment of the present application.

The file information in the file list in fig. 4 belongs to the content to be protected, the control for displaying the file information belongs to the control to be processed, and after the screen display content is subjected to screenshot, the file information in the file list is not included in the recorded image.

In summary, the method generates an aggregation control based on at least one control to be processed, and generates a video according to the aggregation control so as to play the video in the user interface, and simulates the display content of the at least one control to be processed in the user interface. In the process of recording the screen display content, the terminal equipment hides the video in the player, so that the content to be protected does not appear in the recorded image, the leakage of the content to be protected is avoided, and the safety of the content to be protected is improved.

The following describes the method of generating the aggregation control by several embodiments.

In some embodiments, the terminal device creates an aggregation control including content to be protected possessed by at least one control to be processed, including: the terminal equipment determines the position and size information of the aggregation control according to the position and size information of at least one control to be processed; and the terminal equipment creates the aggregation control according to the position and the size information of the aggregation control.

In some embodiments, the position information of the control to be processed is used to indicate a display position of the control to be processed in the user interface. Optionally, the position information includes horizontal position information (x-axis) and vertical position information (y-axis), and the position information of the control to be processed uniquely indicates a display position of the control to be processed in the user interface.

For example, horizontal position information is used to characterize the horizontal position coordinates of the upper left corner of the control to be processed, and vertical position information is used to characterize the vertical position coordinates of the lower left corner of the control to be processed. For another example, the horizontal position information is used to characterize the horizontal position coordinates of the center point of the control to be processed, and the vertical position information is used to characterize the vertical position coordinates of the center point of the control to be processed.

In some embodiments, the size information of the control to be processed is used to characterize the coverage of the control to be processed in the user interface. Optionally, the content information to be protected of the control to be processed is displayed in the coverage area of the control.

In one example, the control to be processed is a polygon, and the size information of the control to be processed includes width information and height information; the width information is used for representing the span of the to-be-processed control in the horizontal direction of the user interface, and the height information is used for representing the span of the to-be-processed control in the vertical direction of the user interface.

It should be noted that, the size and the position information are set according to actual needs, and the numerical range of the size and the position information of each control is not limited in the application.

In some embodiments, the terminal device determines the position and size information of the aggregation control according to the position and size information of the at least one control to be processed, including: the terminal equipment determines the position information of the aggregation control according to the position information of at least one control to be processed; and the terminal equipment determines the size information of the aggregation control according to the size information of at least one control to be processed.

Optionally, the size information of the aggregate control is greater than the size information of any one of the controls to be processed. Further, the size information of the aggregation control is larger than or equal to the sum of the size information corresponding to at least one control to be processed.

Optionally, the terminal device determines minimum horizontal position information from horizontal position information corresponding to at least one control to be processed respectively, and determines horizontal position information of the aggregation control according to the minimum horizontal position information; the terminal equipment determines minimum vertical position information from the vertical position information corresponding to at least one control to be processed respectively, and determines the vertical position information of the aggregation control according to the minimum vertical position information.

Next, taking an example of determining the horizontal position information of the aggregation control, a description will be given of a determination process of the position information of the aggregation control. Assuming that the minimum horizontal position information in the horizontal position information corresponding to the at least one control to be processed is equal to 20, the horizontal position information of the aggregation control is equal to 20 minus a horizontal edge width, and the horizontal edge width is used for limiting the minimum distance between the edge of the aggregation control and the edge of the at least one control to be processed.

In other embodiments, the terminal device determines the position and size information of the aggregation control according to the position and size information of the at least one control to be processed, including: the terminal equipment determines the position and size information of the minimum circumscribed rectangle of the at least one control to be processed according to the position information of the at least one control to be processed; and the terminal equipment determines the position and the size information of the aggregation control according to the position and the size information of the minimum circumscribed rectangle and the edge width.

The minimum bounding rectangle of at least one control to be processed refers to a rectangular region that can include the at least one control to be processed described above. For example, the at least one to-be-processed control means x controls, the smallest circumscribed rectangle of the x to-be-processed controls can include the smallest rectangle of the x to-be-processed controls, and x is a positive integer. The edge width is 0 or any positive number. The aggregation control comprises at least one control to be processed, and the minimum distance between the edge of the at least one control and the edge of the aggregation control is controlled through the edge width, so that the phenomenon that the aggregation control cannot cover all the controls to be processed, and display content of a certain control to be processed in a video generated later is incomplete is avoided.

Optionally, the size information of the aggregation control is equal to the size information of the minimum bounding rectangle plus twice the edge width, the horizontal position information of the aggregation control is equal to the horizontal position information of the minimum bounding rectangle minus the edge width, and the vertical position information of the aggregation control is equal to the vertical position information of the minimum bounding rectangle minus the edge width.

In some embodiments, the terminal device determines the edge width according to a first control in the at least one to-be-processed control, wherein the first control is a control adjacent to the minimum circumscribed rectangle in the at least one to-be-processed control, and the terminal device determines a second control adjacent to the first control from at least one control included in the user interface; the terminal equipment determines the gap distance between the first control and the second control according to the position and the size information respectively corresponding to the first control and the second control; the terminal device determines an edge width according to the gap distance, wherein the value of the edge width is smaller than the value of the gap distance. And determining the edge width according to the gap distance between the to-be-processed control and other controls, so that partial overlapping of the aggregation control and other controls is reduced, the interaction between the aggregation control and other controls is reduced, the terminal equipment can independently display the aggregation control and other controls, and the calculated amount of the terminal equipment in the process of displaying the other controls and the player is reduced.

In some embodiments, the aggregate control includes at least one control to be processed and a transparent region. Optionally, the aggregation control is a transparent area except for the area occupied by at least one control to be processed; the transparent area is an area which is not filled with color and does not display content, and the transparent area does not correspond to other overlapped display content to cause shielding.

By setting the transparent area in the aggregation control, the video generated based on the aggregation control also comprises the transparent area, which is beneficial to avoiding shielding of display contents of other controls in the process of playing the video by using a player.

FIG. 5 is an effect diagram of an aggregation control provided by an exemplary embodiment of the present application.

The aggregation control in fig. 5 comprises a control to be processed 1, a control to be processed 2 and a control to be processed 3. As can be seen from fig. 5, the aggregation control is not adjacent to the edge of the control to be processed 1, the aggregation control is not adjacent to the edge of the control to be processed 2, and the aggregation control is not adjacent to the edge of the control to be processed 3, that is, there is an edge width between the aggregation control and the edge including the control to be processed, and other areas except the control to be processed 1, the control to be processed 2 and the control to be processed 3 in the aggregation control are transparent areas.

By setting the edge width between the aggregation control and the to-be-processed control included by the aggregation control, the aggregation control can completely include the to-be-processed control, so that the video generated based on the aggregation control can completely display the display content of the to-be-processed control, and the method is also beneficial to avoiding shielding of other controls of a user interface in the video playing process of a video player.

Because the transparent area exists in the aggregation control, if the overlapping area exists between the transparent area in the aggregation control and the adjacent control, multiple times of rendering are needed in the process of displaying the user interface on the terminal equipment, and unnecessary calculation amount is increased. By setting the edge width smaller than the gap distance, the overlapping area between the video in the user interface and the adjacent control can be avoided, the calculation amount in the process of displaying the user interface by the terminal equipment can be reduced, and the energy consumption of the terminal equipment can be reduced.

In some embodiments, the terminal device creates an aggregation control according to the position and size information of the aggregation control, including: the terminal equipment creates a blank control according to the position and the size information of the aggregation control; and the terminal equipment respectively adds at least one control to be processed into the blank controls to obtain an aggregation control.

Optionally, the terminal device determines the distribution of the at least one control to be processed in the blank control according to the position and the size information corresponding to the at least one control to be processed respectively; wherein the distribution of the at least one control to be processed in the blank interface is the same as the distribution of the at least one control to be processed in the user interface.

In some embodiments, the process of generating an aggregated control is performed by a control management object in the terminal device. The control management object is registered with at least one control, and the control management object acquires the position and size information of at least one control to be processed; the control management object determines the position and the size information of the aggregation control according to the position and the size information of at least one control to be processed; and the control management object generates an aggregation control according to the position and size information of the aggregation control.

Optionally, after generating the aggregation control, the control management object generates at least one picture corresponding to the aggregation control by using the picture conversion frame, and generates a video for simulating the display content of the aggregation control based on the at least one picture through the video conversion frame.

The method enables the distribution condition of at least one control to be processed in the aggregation control to be consistent with the display condition of at least one control to be processed on the user interface, and enables the display effect of the video generated subsequently to be the same as the display effect of at least one control to be processed in the user interface. And under the condition that the display effect of the user interface is not affected, protecting the content to be protected of at least one control to be processed respectively in the picture recording process.

In the process of displaying the user interface, the terminal device plays the video by using the player. In some embodiments, the player's position and size information is the same as the position and size information of the aggregation control. Optionally, after determining the position and the size of the aggregation control, the terminal device takes the position and the size information of the aggregation control as the position and the size information of the player; the terminal device creates the player according to the position and size information of the player so as to play the video through the player in the display process of the user interface.

By setting the position and size information of the player to be the same as the position and size information of the aggregation control, the size of the video (the number of pixels in the horizontal direction and the number of pixels in the vertical direction of the video image frame can be understood) does not need to be scaled in the video playing process, so that the display effect of at least one control to be processed in the video is close to the real display effect of at least one control to be processed, and the display effect of the user interface caused by the display of the video in the user interface is reduced.

Video needs to be played through the player during the process of displaying the user interface, so that the player needs to be created and managed before the user interface is displayed, and the management process of the player is described in the following through several embodiments.

In some embodiments, the method for protecting the screen display content further comprises: the terminal equipment determines at least one public parent control of the to-be-processed control from a control tree of the user interface, wherein the control tree is used for carrying out hierarchical management on at least one control included in the user interface in a tree structure; the terminal device adds the player to the common parent control to display the player when the common parent control is loaded.

In some embodiments, the control tree includes a plurality of controls with different levels, and there may be no inclusion relationship between the controls of the same level, and there may be no inclusion relationship between the controls of different levels. For example, for any two adjacent two levels included in the control tree, there are two controls with parent-child relationships; the hierarchy of the parent control is higher than that of the child control, the parent control comprises the child control, the parent control is provided with at least one child control, and one child control can only be contained by one parent control.

In some embodiments, the control tree includes a root control, the root control being the highest in the hierarchy in the control tree, the root control including other controls in the control tree.

In some embodiments, the common parent control of at least one control to be processed refers to a control comprising the at least one control to be processed in a control tree, and the control with the lowest hierarchy. For a specific process of determining the common parent control by the terminal device, please refer to the following embodiment.

In some embodiments, a view manager is provided in the terminal device, the view manager is configured to generate display content of the user interface, the view manager includes a plurality of controls related to the user interface, and the plurality of controls include: a control to be processed and other controls other than the control to be processed. I.e. sensitive information controls and non-sensitive information controls comprised in the view manager. Optionally, the control tree is maintained by a view manager. Maintenance of the control tree by the view manager includes, but is not limited to, at least one of: adding new controls to the control tree, and deleting controls from the control tree.

In some embodiments, when the user interface needs to be displayed, the view controller performs display processing on a plurality of controls included in the control tree, generating the user interface.

It can be known from the above that at least one control to be processed is registered in the control management object, and the process of determining the common parent control of the at least one control to be processed is executed by the control management object, and the control management object determines the common parent control of the at least one control to be processed in the control tree. That is, the common parent control of at least one control to be processed is looked up by the control management object from the control tree maintained by the view controller.

Optionally, in the case that the user interface needs to be displayed, the terminal device instructs the view manager to display the controls included in the control tree; before the view manager performs display processing on the controls included in the control tree, the view manager notifies the control management object; after the control management object receives the notification of the view manager, the common parent control of at least one to-be-processed control registered in the control management object is determined from the control tree.

Optionally, after determining the public parent control, the view controller adds the player to the public parent control, so that when the user interface needs to be displayed, the display processing is performed on the player on the public parent control through the view controller, and the player for playing the video appears in the user interface.

FIG. 6 is a diagram of a display of controls in a user interface provided in accordance with an exemplary embodiment of the present application.

The at least one control to be processed includes control 1, control 2, and control 3 in fig. 6. The video played by the player comprises display contents respectively corresponding to a control 1, a control 2 and a control 3, and after recording operation (screenshot operation) of the screen display contents is carried out, the obtained recorded image does not comprise playing frames in the video.

By adding the player to the public father control, the player can be normally displayed in the user interface, and the method has small influence on the hierarchy among the controls in the control tree, so that the influence of the player on the generation process of the display content of the original user interface is reduced.

In the method, at least one control to be processed is included in one aggregation control, and display content corresponding to the at least one control to be processed is simulated on a user interface by playing videos generated based on the aggregation control.

Because the display positions of the at least one to-be-processed control in the user interface are different, as the display content of the user interface is changed, a state transition control can appear in the at least one to-be-processed control, and the state transition control is a control of which the display state in the user interface is changed between display and non-display. The player is added to the public father control, so that the situation that the terminal equipment cancels playing of the video in the user interface after the state transition control (the control which is converted from the display state to the non-display state) appears in at least one to-be-processed control is avoided, and the corresponding display contents of the k to-be-displayed controls in the user interface disappear is caused as long as k to-be-displayed controls exist in at least one to-be-processed control, and video can be played in the user interface through the player, wherein k is a positive integer.

In the following, specific steps for determining a common parent control are described in terms of several embodiments.

In some embodiments, the terminal device determines, from a control tree of the user interface, a common parent control of the at least one pending control, including: the terminal equipment determines at least one parent control corresponding to the to-be-processed control respectively from the control tree, wherein the parent control at least comprises the to-be-processed control; and under the condition that the father controls corresponding to the at least one to-be-processed control are the same control, the terminal equipment determines the same control as the public father control.

In some embodiments, the determining, by the terminal device, a parent control corresponding to at least one to-be-processed control from the control tree includes: for a first child control in the at least one to-be-processed control, the terminal device searches a parent control corresponding to the first child control from the control tree, wherein the first child control is any control in the at least one to-be-processed control.

Optionally, the parent control corresponding to the to-be-processed control refers to: the control tree comprises the control to be processed and the control with the lowest hierarchy. The hierarchy of the to-be-processed control in the control tree is adjacent to the hierarchy of the parent control corresponding to the to-be-classified control, and the hierarchy of the parent control in the control tree is higher than the hierarchy of the to-be-processed control. For example, a certain control to be processed is at layer 4 in the control tree, and the parent control corresponding to the control to be processed is at layer five in the control tree.

Optionally, the terminal device determines, through the control management object, parent controls corresponding to at least one to-be-processed control respectively from the control tree maintained by the view controller.

FIG. 7 is a schematic diagram of a control management object provided by an exemplary embodiment of the present application. The screenshot management class in fig. 7 is a control management object. In fig. 7, the control management object registers that the control 1, the control 2 and the control 3, that is, the control 1, the control 2 and the control 3 all have contents to be protected. And determining parent controls corresponding to the control 1, the control 2 and the control 3 in the control management object control tree.

In some embodiments, after obtaining the parent controls corresponding to the at least one to-be-processed control respectively, the terminal device determines whether the parent controls corresponding to the at least one to-be-processed control respectively are the same parent control.

In one case, if the parent controls corresponding to the at least one to-be-processed controls are the same controls, that is, the same controls include the at least one to-be-processed controls. For example, the at least one control to be processed includes: the method comprises the steps that a control 1 to be processed and a control 2 to be processed are adopted, a parent control corresponding to the control 1 to be processed is a control A, a parent control corresponding to the control 2 to be processed is a control A, and a public parent control of the control 1 to be processed and the control 2 to be processed is a control A.

Because the hierarchy of the parent control corresponding to the to-be-processed controls is adjacent to the hierarchy of the controls, the same control meets the condition of the public parent control, and the same control is the public parent control of the at least one to-be-processed control. That is, in this case, after determining the parent controls corresponding to at least one to-be-processed control respectively, the terminal device can directly use the parent control corresponding to the to-be-processed control as the public parent control.

In another case, the parent control corresponding to the at least one to-be-processed control is a plurality of different controls, and a method for determining the common parent control of the at least one to-be-processed control in the control tree in this case is described below through several embodiments.

In some embodiments, the terminal device determines a common parent control of at least one control from a control tree of the user interface, including: the terminal equipment determines father controls corresponding to at least one control respectively from the control tree; under the condition that the father control corresponding to at least one control is a plurality of different controls, the terminal equipment creates a control set comprising the plurality of different controls; for a first control and a second control in any two different levels in the control set, the level of the first control is higher than the level of the second control, if the first control does not comprise the second control, the terminal equipment determines a father control of the second control from the control tree, and replaces the second control in the control set by the father control of the second control; if the first control comprises the second control, the terminal equipment deletes the second control from the control set; for any two third controls and fourth controls in the same hierarchy in the control set, determining a fifth control from the control tree, wherein the fifth control comprises the third control and the fourth control; the terminal equipment uses the fifth control to replace a third control and a fourth control in the control set; and the terminal equipment repeats the steps until one control remains in the control set, and determines one control as a public parent control.

For a method for determining parent controls corresponding to at least one control respectively, please refer to the previous embodiment, and details are not described herein.

In some embodiments, the parent control corresponding to the at least one control is a plurality of different controls, which means that at least two different parent controls exist in the parent control corresponding to the at least one control. For example, the at least one control to be processed includes: the control system comprises a to-be-processed control 1, a to-be-processed control 2 and a to-be-processed control 3, wherein a parent control corresponding to the to-be-processed control 1 is a control A, a parent control corresponding to the to-be-processed control 2 is a control B, and a parent control corresponding to the to-be-processed control 3 is a control C, and the controls corresponding to the to-be-processed control 1, the to-be-processed control 2 and the to-be-processed control 3 are a plurality of different controls respectively.

Optionally, the plurality of different parent controls includes the following types: parent controls belonging to different levels and parent controls belonging to the same level. The terminal equipment cuts down a plurality of different father controls according to the structure of the control tree, and finally determines the public father control of at least one control from the control tree according to the plurality of different father controls. It should be noted that the plurality of different parent controls may or may not include a common parent control (the terminal device searches for the common parent control in a higher layer of the control tree according to the plurality of different parent controls).

In some embodiments, a terminal device creates a control set comprising a plurality of different controls, the control set for temporarily storing the plurality of different controls in determining a common parent control. Optionally, in the process of determining the public parent control, the terminal device may add a control to the control set, or may delete a control from the control set. The level of the newly added control in the control set in the control tree is higher than the level of at least one control in the control set in the control tree. It should be noted that, in the process that the terminal device has the common parent control, the storage control for temporarily storing the intermediate control (for example, the parent space corresponding to at least one to-be-processed control respectively) can be understood as a control set, and the control set can be implemented by any structure. After determining the public parent control, the terminal device adds a player to the public parent control through the view controller, optionally, in order to ensure that the playing effect of the video is similar to the display effect respectively corresponding to at least one control, the position and size information of the player are the same as the position and size information of the aggregation control. For the determination of the size and position information of the aggregation control, please refer to the above embodiment, and details are not described herein.

FIG. 8 is a schematic diagram of a determination of a common parent control provided by an exemplary embodiment of the present application. The control tree species comprises a parent control 1, a parent control 2, a parent control 3, a control 1, a control 2 and a control 3. The relationships of the above controls in the control tree are as follows, the parent control of the control 1 is the parent control 1, the parent control of the control 2 is the parent control 2, the parent control of the control 3 is the parent control 2, the parent control of the parent control 1 is the parent control 3, and the parent control of the parent control 2 is the parent control 3. In fig. 8, control 1, control 2 and control 3 are registered in a screenshot management class (also referred to as a control management object), that is, control 1, control 2 and control 3 are all to-be-processed controls. The control management object determines common father controls of the control 1, the control 2 and the control 3 from the control tree, and the common father controls comprise: the control management object is determined from the control tree: the parent control of the control 1 is a parent control 1, the parent control of the control 2 is a parent control 2, the parent control of the control 3 is a parent control 2, and the parent control of the parent control 1 is a parent control 3, namely, the parent controls respectively corresponding to the control 1, the control 2 and the control 3 are a plurality of different controls. According to the content in the above embodiment, the control management object creates a control set, and the control set is (parent control 1, parent control 2); for a parent control 1 and a parent control 2 belonging to the same hierarchy in the control set, a control management object is determined from a control tree: the parent control of the parent control 1 is the parent control 3, the parent control of the parent control 2 is the parent control 3, namely, the parent control 3 comprises the parent control 1 and the parent control 2, the control management object adds the parent control 3 into the control set, removes the parent control 1 and the parent control 2 from the control set, and the control set is changed into (the parent control 3). Because at this time, one control remains in the control set, that is, the common parent control of control 1, control 2, and control 3 is parent control 3.

The following describes the process of determining the public parent control by the terminal device by combining the two cases:

step a: under the condition that the control set comprises at least two different levels of controls, for a first control and a second control in any two levels of controls, the terminal equipment judges whether the first control comprises the second control or not; and if the first control comprises the second control, the terminal equipment deletes the second control from the control set. If the first control does not comprise the second control, the terminal equipment deletes the second control from the control set, and adds the father control corresponding to the second control to the control set.

Step b: under the condition that the control set comprises at least two controls with the same level, for any two third controls and fourth controls with the same level, the terminal equipment determines a fifth control from the control tree, wherein the fifth control comprises the third control and the fourth control, and the terminal equipment deletes the third control and the fourth control from the control set and adds the fifth control into the control set.

Optionally, in this step: the terminal equipment determines a father control of the third control and a father control of the fourth control from the control tree; if the father control of the third control and the father control of the fourth control are the same control, the terminal device takes the father control of the third control as a fifth control. If the father control of the third control and the father control of the fourth control are different controls, the terminal equipment adds the father control of the third control and the father control of the fourth control into the aggregation control, and deletes the third control and the fourth control from the aggregation control; the terminal equipment regards the father control of the third control as a new third control, and regards the father control of the fourth control as a new fourth control; the terminal equipment performs the steps of determining the father control of the new third control from the control tree and determining the father control of the new fourth control until the fifth control is determined from the control tree.

And the terminal equipment repeatedly executes the step a and the step b until one control remains in the control set, and takes the one control as a public control of at least one control. The step of determining the common parent control is performed by a control management object in the terminal device.

It should be noted that the execution sequence of the step a and the step b may be exchanged. For example, during a loop of determining the common parent control, the terminal device first performs step a, and then performs step b. For example, during a loop of determining the common parent control, the terminal device first performs step b, and performs step a. The application is not limited to the order of execution of steps a and b during the cycle.

Optionally, after determining the common parent control, the control management object notifies the view controller of the common parent control corresponding to the at least one to-be-processed control, and the view manager adds the player to the common parent control so as to play the video through the player. For details of this step, please refer to the above embodiments, and detailed descriptions thereof are omitted herein.

Under the condition that the control set comprises at least two controls with different levels, the terminal equipment can determine whether an inclusion relationship exists between at least two controls in any two different levels, and delete the control with the lower level in the included controls, so that the determined public parent control comprises the at least one control to be processed, and the control with the lowest level in the control tree is the public parent control, and the accuracy of the determined public parent control is improved.

In some embodiments, after the terminal device performs video conversion processing on the aggregation control and generates a video for simulating the display content of the aggregation control, the method further includes: the terminal device removes at least one control to be processed from the control tree.

And the terminal equipment removes the at least one control to be processed from the control tree, so that the at least one control to be processed disappears from the control tree. Optionally, the terminal device only removes at least one control to be processed from the control tree, and does not remove other controls included in the control tree.

In some embodiments, the step of removing, by the view controller of the terminal device, the at least one to-be-processed control from the control tree, the view controller removing the at least one to-be-processed control from the control tree after the control management object determines a common parent control of the at least one to-be-processed control from the control tree; when the user interface needs to be displayed, the view controller performs display processing on the controls included in the control tree, so that the controls included in the control tree are displayed on the user interface.

In the method, the display content of at least one control to be processed in a user interface is simulated by using a player to play video. If deletion processing is not performed on at least one to-be-processed control included in the control tree, the terminal equipment directly displays the at least one to-be-processed control on the user interface. On one hand, the processing pressure of the terminal equipment in the display processing process of the controls in the control tree is increased. On the other hand, the terminal equipment responds to the recording operation of the screen display content, and although the video in the player is hidden in the recorded image, the user interface also displays the display content respectively corresponding to at least one to-be-processed control, so that the to-be-protected content appears in the recorded image. By deleting at least one control to be processed from the control tree, the computing pressure of the terminal equipment is reduced, leakage of the content to be protected from the recorded image is avoided, and desensitization processing of the content with protection (namely sensitive information) is realized.

In some embodiments, the terminal device performs video conversion processing on the aggregation control, and generates a video for simulating display content of the aggregation control, including: the terminal equipment performs picture conversion processing on the aggregation control to obtain at least one picture corresponding to the aggregation control, wherein the picture comprises display content of at least one control to be processed; and the terminal equipment generates a video for simulating the display content of the aggregation control according to at least one picture.

In some embodiments, in the case that the at least one control to be processed does not include a control for displaying dynamic content, the terminal device only needs to generate a picture corresponding to the aggregation control, where the picture corresponding to the aggregation control includes at least one control to be processed to respectively correspond to the display content. That is, the pictures corresponding to the aggregation control include at least one to-be-protected content respectively possessed by the to-be-processed controls.

In some embodiments, in the case that at least one control to be processed includes a control for displaying dynamic content, the terminal device needs to generate a plurality of pictures corresponding to the aggregation control, and generates a video corresponding to the aggregation control through the plurality of pictures corresponding to the aggregation control, so that the dynamic content in the control to be processed is reflected in the process of playing the video.

For example, at least one control to be processed includes a control for displaying a dynamic picture, the dynamic picture includes 3 picture frames, the terminal device generates 3 pictures corresponding to the aggregation control, and the controls for displaying the dynamic picture respectively correspond to different display contents in the 3 pictures. By the method, the dynamic content can be represented through the picture, and the generated video can completely simulate the dynamic content of the control to be processed.

In some embodiments, the size information of the picture corresponding to the aggregation control is the same as the size information of the aggregation control, and the picture corresponding to the aggregation control includes at least one display content and a transparent area respectively corresponding to the control. Optionally, the image corresponding to the aggregation control is a transparent area except for the image area for displaying the content corresponding to at least one control. By including the transparent area in the picture corresponding to the aggregation control, the method can avoid the subsequent display of the video related to the aggregation control from blocking other normally displayed user interface controls, and reduce the influence of the protection method of the screen display content on the display effect of the user interface.

In some embodiments, the terminal device generates at least one picture corresponding to the aggregation control through the picture conversion framework. Optionally, the picture conversion framework is provided by an operating system of the terminal device, for example a Core Graphics framework in the ios system. Optionally, the terminal device invokes the picture conversion framework to generate at least one picture corresponding to the aggregation control.

In some embodiments, the terminal device generates a video including at least one picture according to at least one picture corresponding to the aggregation control. Optionally, the terminal device inputs the at least one picture into a video conversion framework, and the video conversion framework is used for processing the at least one picture to generate a video including the at least one picture. The video conversion framework may be a self-contained framework in an operating system of the terminal device, such as an AVFoundation framework in the ios system.

In some embodiments, the terminal device generates a video corresponding to a picture in a case that the at least one control to be processed does not include a control for displaying dynamic content. Alternatively, a video generated from one picture is referred to as a single-frame video.

In some embodiments, in a case that the at least one control to be processed includes a control for displaying dynamic content, the terminal device generates videos corresponding to the plurality of pictures. As can be seen from the above, the display effects of the controls for displaying the dynamic content in each picture are not completely the same, and the terminal device sorts the pictures according to the display order of the display effects of the controls in the dynamic content, and generates a video including the pictures according to the sorted pictures.

And generating a video based on the pictures corresponding to the aggregation control, so that the display content of the video is the same as the display content of the aggregation control, and the simulation effect of simulating the display content of at least one control to be processed through the video is more real.

The method of implementing the display content of at least one control to be processed in a user interface is described below by way of an example.

In some embodiments, the terminal device comprises a control management object and a view controller, and a binding relationship exists between the control management object and the view controller.

Step1, when the user interface needs to be displayed, the view controller sends notification information to the control management object, wherein the notification information is used for prompting the control management object to generate an aggregation control of at least one to-be-processed control.

At least one control to be processed refers to a control registered in a control management object. Optionally, at least one control to be processed is set by a user. For example, the terminal device determines a control with the content to be protected as a control to be processed in response to an instruction operation for the content to be protected; and the terminal equipment registers at least one control to be processed to the control management object. Registering the control to be processed to the control management object means that attribute information of the control to be processed is stored in the control management object, so that the control management object can determine the control to be processed from the control tree through the attribute information corresponding to the control to be processed. For example, the attribute information includes the name, number, and the like of the control to be processed.

Optionally, the at least one control to be processed may be set by a manager of the target application, at least one control to be processed set by the manager is stored in the server, the terminal device obtains the at least one control to be processed through the server, and registers the obtained at least one control to be processed in the control management object. In some embodiments, the method for protecting the content of the screen display provided by the method has an on state and an off state in the target application program, and when the method for protecting the content of the screen display is in the on state, the terminal device starts to execute from the step 1. When the protection method of the screen display content is in a closed state, the terminal equipment does not execute the steps from the step 1, and in this case, the terminal equipment directly displays the display content of the control to be protected on the user interface; the recorded image generated by the terminal equipment in response to the recording method of the screen display content also comprises display content corresponding to the image to be processed, which is equivalent to the fact that the plurality of controls included in the user interface do not have the control to be processed. Alternatively, the on-screen content protection method has an on state and an off state in the target application, and may be set by any user, or by an enterprise administrator.

FIG. 9 is a schematic diagram of a screenshot protection interface provided by an exemplary embodiment of the application.

In fig. 9, the administrator opens the "screenshot protection" option to enable the on-screen content protection method to be in an on state in the target application.

Step2, traversing a control tree of the view manager according to the at least one to-be-processed control by the control management object, and determining a public parent control of the at least one to-be-processed control from the control tree. For details of this step, please refer to the above embodiments, and detailed descriptions thereof are omitted herein.

Step3, the control management object creates an aggregation control comprising the content to be protected of at least one control to be processed respectively according to the position and the size information of the at least one control to be processed. The aggregation control comprises the at least one control to be processed, the distribution condition of the at least one control to be processed in the aggregation control is determined according to the position and the size information of the at least one control to be processed, and the specific content of the process is described in the above embodiments.

Step4, the terminal equipment performs video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control. Specifically, the control management object generates at least one picture corresponding to the aggregation control by using the Core Graphics framework. And under the condition that at least one control to be processed does not comprise a control for displaying dynamic content, only one picture corresponding to the aggregation control is needed to be generated. And the control management object processes at least one picture corresponding to the aggregation control by using the AVFoundation framework to generate a video comprising the at least one picture.

Step5, the view controller adds a player for playing video to the common parent control.

Step6, the view controller removes at least one control to be processed from the control tree.

It should be noted that the steps of steps 2-Step5 may be flexibly changed, for example, step3 may be performed before Step2, step6 may be performed at any time after Step4, and steps 1-6 do not represent a limitation on the timing of each Step included in the method.

In some cases, in order to reduce the occupation of system resources of the terminal device by generating the video and adding the player in the common parent control, at least one control to be processed refers to a plurality of controls to be processed. Optionally, the at least one control to be processed refers to all sensitive information controls referred to by the user interface.

In some embodiments, the recorded image acquisition method further comprises: and in the case that the occurrence frequency of at least one control to be processed in the user interface is higher than a first threshold value, the terminal equipment stores the video and in the case that the at least one control to be processed is included in the other user interface, the terminal equipment plays the video in the other user interface through a player in the process of displaying the other user interface.

In some embodiments, another user interface is different from the user interface with the player and other controls displayed in step 240.

Since at least one control to be processed may repeatedly appear in different user interfaces of the target application, in order to avoid that the terminal device repeatedly performs steps 210 to 230 when displaying different user interfaces, the calculation amount of the terminal device is increased. The terminal equipment stores the video after the video is generated for the first time, and under the condition that the terminal equipment needs to display another user interface, if the control management object corresponding to the other interface comprises at least one control to be processed, the terminal equipment does not need to generate the aggregation control comprising at least one control to be processed again, the terminal equipment directly acquires the video stored in the cache, and plays the video in the other user interface of the player.

In some embodiments, the frequency of occurrence of at least one control to be processed in the user interface refers to the number of occurrences of at least one control to be processed in a different user interface, optionally, the frequency of occurrence of the control to be processed in the user interface is determined by the type of control display operation. For example, a certain to-be-processed control can correspond to 3 control display operations, and then the frequency of occurrence corresponding to the to-be-processed control is equal to 3, that is, any one of the 3 control display operations can make the control be displayed in the user interface.

Optionally, the more the types of control display operations corresponding to the controls, the greater the frequency of occurrence of the controls in the user interface, the less the types of control display operations corresponding to the controls, and the less the frequency of occurrence of the controls in the user interface.

In some embodiments, the frequency of occurrence of the at least one control to be processed in the user interface refers to a display duration of the at least one control to be processed in the user interface. The longer the display duration of the control to be processed on the user interface is, the higher the appearance frequency of the control to be processed is, the shorter the display duration of the control to be processed on the user interface is, and the lower the appearance frequency of the control is.

In some embodiments, the first threshold is preset. Optionally, the first threshold is a positive integer.

In some embodiments, the terminal device divides the at least one control to be processed into a first control group and a second control group; the first control group comprises at least one to-be-processed control with the occurrence frequency being greater than or equal to a first threshold value, and the second control group comprises at least one to-be-processed control with the occurrence frequency being less than the first threshold value; the method comprises the steps that a first aggregation control comprising any one of the to-be-processed controls in a first control group is created by the terminal equipment, a second aggregation control comprising any one of the to-be-processed controls in a second control group is created, video conversion processing is respectively carried out on the first aggregation control and the second aggregation control by the terminal equipment, and a first video for simulating display content of the first aggregation control and a second video for simulating display content of the second aggregation control are generated; in the process of displaying the user interface, the terminal equipment plays the first video through the first player and plays the second video through the second player on the user interface, and displays display contents respectively corresponding to other controls except at least one to-be-processed control in the plurality of controls.

Optionally, the video of the recorded image related to the image to be processed is hidden, i.e. neither the first video nor the second video is present in the recorded image. Optionally, after the user interface is displayed, the first video is stored by the terminal device, and the second video is deleted by the terminal device, so as to reduce the occupation of the buffer space of the terminal device by the stored video.

It should be noted that the above-mentioned process of generating the first video and the second video refers to all the processes included in steps 220-230, and will not be described herein. In this case, the terminal device stores the first video, and in the case where the to-be-processed control in the first control group is included in the other user interface, the terminal device plays the first video in the other user interface through the player.

By caching the video, occupation of the repeated video generated to system resources is reduced, and the phenomenon that the terminal equipment is blocked is avoided.

When the user interface needs to be displayed, the view controller performs display processing on the controls in the control tree to obtain the user interface. The following describes a method for a player to play video in a user interface by several embodiments.

In some embodiments, the terminal device plays the video in the user interface through the player, including: the terminal equipment generates a false invalid path according to a storage path of the video in the terminal equipment, wherein the false invalid path refers to a path which cannot be analyzed correctly due to a path format error; generating video information comprising fake encryption information corresponding to the video according to the fake invalid path, wherein the video information is used for indicating a player to hide the video when a recording operation is detected; the terminal equipment corrects the path format errors existing in the false invalid paths to obtain false valid paths; the terminal equipment uses the false effective path to create a video downloading task to load the video; and the terminal equipment plays the video through the player.

In some embodiments, a false invalid path is used to indicate the wrong video storage path. Alternatively, the false invalid path refers to a uniform resource locator of a format error. In some embodiments, the false invalid path includes a protocol identification and an access path, wherein the protocol identification of the false invalid path is the wrong communication protocol name and the access path is a storage path of the video in the terminal device.

Optionally, the terminal device rewrites the existing protocol name to generate an erroneous communication protocol name. For example, the terminal device adds a prefix before the existing protocol name, resulting in an erroneous communication protocol name.

In some embodiments, the terminal device generates a false invalid path according to a stored path of the video in the terminal device, including: the terminal equipment rewrites according to the existing protocol name to obtain the wrong communication protocol name; the terminal equipment combines the wrong communication protocol name with the storage path of the video in the terminal equipment to obtain a false invalid path.

The false effective path refers to an access address without format errors, and optionally, the terminal device replaces the wrong communication protocol name in the false ineffective path to obtain the false effective path.

In some embodiments, the false effective path includes the correct communication protocol name and the stored path of the video in the terminal device.

In some embodiments, the terminal device triggers the player to generate a path resolution error through a false invalid path; the terminal equipment sends a false invalid path to a play manager through a player, and the play manager is used for managing the play of at least one video; the terminal equipment generates a key address for acquiring a fake encryption key of the video according to the fake invalid path through the play manager; and the terminal equipment generates video information which falsifies the video into the encrypted video according to the key address.

In some embodiments, after generating the false invalid path, the terminal device sends the false invalid path to the player; the player parses the false invalid path. Because of format errors in the false invalid path, the player fails to analyze the false invalid path, and the player sends the false invalid path to a play control manager in the terminal equipment.

After receiving the false invalid path, the play control manager also needs to analyze the false invalid path, the play control manager also fails to analyze the false invalid path, and determines that the false invalid path is forged by the terminal equipment according to the wrong communication protocol name included in the false invalid path, and then the play control manager calls a first proxy method according to the false invalid path; wherein the first proxy method is used for generating video information according to the false invalid path so as to inform the terminal equipment according to the video information when the screen content is recorded: it is necessary to conceal the video in the recorded image.

Optionally, the video information is of a data type of m3u8 (Extended Moving Picture Experts Group Audio Layer, uniform Resource Locator, uniform resource locator of the extended motion picture expert compressed standard audio layer 3), and the video information includes: the EXT-X-KEY field includes a KEY address.

In some embodiments, the key address is a path that is not accessible in the presence of a format error. Optionally, the terminal device uses the false invalid path as a key address. When the player obtains the key of the video according to the key address in the video information, the key address needs to be resolved. Because of the format error of the key address, the player fails to parse the key address. The player sends the secret key address to the play controller; the play controller analyzes the key address, and the play controller fails to analyze the key address; the playback controller generates a blank key that is used to forge the encryption key for the video. The terminal equipment determines the video as an encrypted video according to the blank secret key; after the terminal equipment detects the screenshot and recording operation, the system service is started to hide the video.

After the video information is generated, the terminal equipment generates a false effective path according to the false ineffective path, the terminal equipment provides the false effective path for the player, and after the player receives the false effective path, the false effective path is analyzed, and because the format of the false effective path is correct, the false effective path can be successfully analyzed by the player. After the player successfully parses the virtual effective path, the terminal device creates a video download task loaded video using the virtual effective path, and for details of this process, please refer to the following examples.

The following describes the loading process of the video download task by several embodiments.

In some embodiments, the terminal device creates a video download task loaded video using a false active path, comprising: the terminal device triggers the player to request the download manager to create a download task through the false effective path; under the condition that a download list of the download manager is free, the terminal equipment informs the resource manager to download the video according to the false effective path through the download manager; the terminal equipment determines a storage path according to the false effective path through a resource manager; the terminal device loads the video from the storage path through the resource manager.

In some embodiments, a download manager is used to create and manage at least one download task. Optionally, a download list is maintained in the download manager, the download list including at least one incomplete download task. If the download list is free, the download manager can create a new download task and add the new download task to the download list; if the download list is not free, a new download task is created after waiting for the download list control to be free.

In some embodiments, the process of creating a video download task is accomplished by a player and a download manager. Continuing with the above process, after the player successfully parses the false effective path, the player notifies the download manager to create a download task. If the download list is free, the download manager generates a video download task; if the download list is not free, the download manager waits for the download list to be free, and then generates a download task.

In some embodiments, the resource manager is configured to obtain the content to be loaded from the internet or the terminal device. After the video download task is created, the download manager sends a false effective path to the resource manager, the resource manager determines a storage path of the video in the terminal device according to the false effective path, and the resource manager loads the video from the storage path.

Optionally, the determining, by the terminal device, a storage path according to the false effective path through the resource manager includes: and the terminal equipment removes the communication protocol name in the false effective path to obtain the storage path of the video in the terminal equipment. In some embodiments, the protocol identification in the false invalid path is used to indicate the wrong communication protocol name and the protocol identification in the false valid path is used to indicate the correct communication protocol name.

By modifying the communication protocol name, the conversion between the false invalid path and the false effective path is realized in a simpler way, which is beneficial to reducing the amount of time for the terminal equipment to generate the false invalid path and the false effective path. Meanwhile, the form of the wrong communication protocol name in the false invalid path can be agreed with the play controller, so that the false invalid path can be conveniently identified by the play controller, video information is generated according to the false invalid path, and the video is falsified into online video to be protected through the video information, so that the video played in the player is hidden and displayed in the recording operation process of the terminal equipment.

Fig. 10 is a schematic diagram of a video loading process provided by an exemplary embodiment of the present application.

Step 1: the terminal device needs to display a user interface in response to a user operation.

Step 2: and the terminal equipment generates a false invalid path according to the storage path of the video in the terminal equipment, wherein the false invalid path refers to a path which cannot be resolved correctly due to the error of the path format. Optionally, the protocol identification in the false invalid path is used to indicate the wrong communication protocol name. Alternatively, the false invalid path is a URL (Uniform Resource Locator ) with a format error, and may also be a false invalid URL.

For example, if the storage path of the video in the terminal device is "path.to.local.video", and the wrong communication protocol name is "testHttp", the terminal device combines the wrong communication protocol name and the storage path of the video in the terminal device to obtain a false failure URL: testHttp:// path.to.local.video1.

Step 3: the terminal device generates video information for falsifying the video into an encrypted video according to the false invalid path, the video information being used to indicate a hidden video when a recording operation is detected. Optionally, the video information is of the m3u8 data type.

Wherein, step 3 comprises the following sub-steps:

step 3.1: triggering the player to generate path analysis errors through false invalid paths; the virtual invalid path is sent by the player to a play manager, which is used to manage the play of the at least one video.

Specifically, when the player receives the false invalidation URL, the player invokes a first proxy method for obtaining video resources, where the first proxy method is used to modify the URL again, that is, send the false invalidation path to the play manager through the player. The first proxy method is registered in the play control manager.

Step 3.3: generating a key address for acquiring a fake key of the video according to the fake invalid path through the play manager; video information is generated that falsifies the video into an encrypted video based on the key address.

When the play manager receives the call of the first proxy method, the play manager analyzes whether the false failure URL belongs to the terminal equipment false URL; if the false failure URL is forged by the terminal equipment, the play manager generates video information, such as m3u8 data, according to the false failure URL.

The video information needs to include a key address, and the play manager generates the key address according to the false failure URL and generates the video information according to the key address.

Step 4: the terminal equipment corrects the path format errors existing in the false invalid paths to obtain false valid paths.

Optionally, the terminal device corrects the wrong communication protocol name in the false invalid path into the correct communication protocol name, so as to obtain the false valid path.

For example, the terminal device replaces the wrong communication protocol name in the false failure URL with the correct communication protocol name, such as modifying the false failure URL "testHttp:// path.to.local.video1" to the false valid URL "http:// path.to.local.video1".

Step 5: the terminal device creates a video download task to load the video using the false active path.

Wherein, step 5 comprises the following sub-steps:

Step 5.1: triggering the player to request the download manager to create a download task through the false active path.

After the player receives the false effective URL, the player adds a download task through a download manager, and the download manager judges whether the downloaded list is free or not. If the downloaded list is free, continuing to execute the step 5.3; if the download list is not free, the download list needs to be waited for to be free.

Step 5.3: and under the condition that the download list of the download manager is free, notifying the resource manager to load the video according to the false effective path through the download manager.

Optionally, the download manager invokes a second proxy method in the resource controller for uploading video from the terminal device through the resource controller. The second proxy method comprises the steps of 5.5 and 5.6.

Step 5.5: the storage path is determined by the resource manager from the false effective path. Specifically, the resource manager receives the call of the second proxy method and starts to load the video.

Step 5.6: the download manager sets the download task to wait for the video asset manager to load to avoid other controllers for downloading video attempting to acquire video from the false inactive path.

Step 5.7: video is loaded from the storage path through the resource manager.

Step 6: the video loading is completed.

Step 7: the terminal device plays the video in the user interface through the player, and the terminal device deletes the task for downloading the video in the downloading list.

According to the method, the DRM (Digital Rights Management) video is created by the video stored in the terminal equipment in a pseudo manner, and meanwhile, a blank key is transmitted into the m3u8 file, so that a system of the terminal equipment can identify the video as the DRM video, and therefore, in the process of screenshot and screen recording, a protection mechanism of the DRM video is triggered, the content to be protected is hidden, and the security of the content to be protected in the screen display content is improved.

The following are examples of the apparatus of the present application that may be used to perform the method embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the method of the present application.

Fig. 11 shows a block diagram of a manipulation device for virtual articles according to an exemplary embodiment of the present application. The apparatus may be implemented as all or part of a terminal device by software, hardware or a combination of both. The apparatus 1100 may include: a control acquisition module 1110, a control aggregation module 1120, a video generation module 1130, an interface display module 1140, and an image acquisition module 1150.

The control acquisition module 1110 is configured to determine at least one control to be processed with content to be protected from at least one control included in the user interface.

And the control aggregation module 1120 is used for creating an aggregation control comprising the content to be protected of the at least one control to be processed.

And the video generation module 1130 is used for performing video conversion processing on the aggregation control to generate a video for simulating the display content of the aggregation control.

An interface display module 1140, configured to display, in the user interface, other controls except the control to be processed in the at least one control, and play the video in the user interface through a player.

An image obtaining module 1150, configured to obtain a recorded image of the on-screen content in response to a recording operation for the user interface, where the recorded image includes the user interface and the video played by the player is hidden.

In some embodiments, the control aggregation module 1120 is configured to determine position and size information of the aggregation control according to the position and size information of the at least one control to be processed; and creating the aggregation control according to the position and the size information of the aggregation control.

In some embodiments, the player's location and size information is the same as the aggregation control's location and size information.

In some embodiments, the apparatus 1100 further comprises: the parent control determining module is used for determining a public parent control of the at least one control to be processed from a control tree of the user interface, and the control tree is used for carrying out hierarchical management on the at least one control included in the user interface in a tree structure; and adding the player to the public parent control to display the player when the public parent control is loaded.

In some embodiments, the parent control determining module is configured to determine parent controls that respectively correspond to the at least one to-be-processed control from the control tree, where the parent controls at least include the to-be-processed control; under the condition that the father controls corresponding to the at least one to-be-processed control are the same control, determining the same control as the public father control; creating a control set comprising a plurality of different controls under the condition that the father controls respectively corresponding to the at least one to-be-processed control are the plurality of different controls; for a first control and a second control in any two different levels in the control set, wherein the level of the first control is higher than that of the second control, if the first control does not comprise the second control, determining a father control of the second control from the control tree, and replacing the second control in the control set by using the father control of the second control; if the first control comprises the second control, deleting the second control from the control set; for any two third controls and fourth controls in the control set in the same level, determining a fifth control from the control tree, wherein the fifth control comprises the third control and the fourth control; replacing the third control and the fourth control in the control set by using the fifth control; repeating the steps until one control remains in the control set, and determining the one control as the public parent control.

In some embodiments, the apparatus 1100 further comprises: and the control deleting module is used for removing the at least one control to be processed from the control tree.

In some embodiments, the video generating module 1130 is configured to perform a picture conversion process on the aggregation control to obtain at least one picture corresponding to the aggregation control, where the picture includes display content of the at least one control to be processed; and generating the video for simulating the display content of the aggregation control according to the at least one picture.

In some embodiments, the apparatus further comprises a video caching module for storing the video if the frequency of occurrence of the at least one control to be processed in the user interface is above a first threshold; and in the case that the at least one control to be processed is included in the other user interface, playing the video in the other user interface through a player in the process of displaying the other user interface.

In some embodiments, the interface display module 1140 includes: the path generation module is used for generating a false invalid path according to the storage path of the video in the terminal equipment, wherein the false invalid path refers to a path which cannot be resolved correctly due to a path format error; an information falsification module for generating video information falsifying the video into an encrypted video according to the false invalid path, wherein the video information is used for indicating to hide the video when the recording operation is detected; the path correction module is used for correcting the path format errors existing in the false invalid path to obtain a false valid path; the video loading module is used for creating a video downloading task to load the video by using the false effective path; and the video playing module is used for playing the video through the player.

In some embodiments, the video loading module is configured to trigger, through the false valid path, the player to request a download manager to create a download task; under the condition that a download list of the download manager is free, notifying a resource manager to load the video according to a false effective path by the download manager; determining, by the resource manager, the storage path from the false effective path; and loading the video from the storage path through the resource manager.

In some embodiments, the protocol identification in the false invalid path is used to indicate a wrong communication protocol name and the protocol identification in the false valid path is used to indicate a correct communication protocol name.

In some embodiments, the information falsification module is configured to trigger the player to generate a path analysis error through the false invalid path; sending the false invalid path to a play manager through the player, wherein the play manager is used for managing the play of at least one video; generating, by the play manager, a key address for obtaining a fake key of the video according to the fake invalid path; and generating video information which falsifies the video into an encrypted video according to the key address.

It should be noted that, in the apparatus provided in the foregoing embodiment, when implementing the functions thereof, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be implemented by different functional modules, that is, the content structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the apparatus and the method embodiments provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the apparatus and the method embodiments are detailed in the method embodiments and are not repeated herein. The beneficial effects of the device provided in the foregoing embodiments are described with reference to the method side embodiments, and are not repeated herein.

Fig. 12 is a block diagram showing a structure of a terminal device according to an exemplary embodiment of the present application.

The terminal device 1200 may be an electronic device such as a cell phone, tablet computer, game console, electronic book reader, multimedia playing device, wearable device, PC, etc. The terminal device is used for implementing the method for protecting the screen display content provided in the embodiment. The terminal device may be the terminal device 10 shown in fig. 1. Specifically, the present application relates to a method for manufacturing a semiconductor device.

In general, the terminal apparatus 1200 includes: a processor 1201 and a memory 1202.

Processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 1201 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1201 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1201 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and rendering of content required to be displayed by the display screen. In some embodiments, the processor 1201 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1202 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 1202 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1202 stores a computer program that is loaded and executed by processor 1201 to implement the method of protecting on-screen content provided by the method embodiments described above.

In some embodiments, the terminal device 1200 may further optionally include: a peripheral interface 1203, and at least one peripheral. The processor 1201, the memory 1202, and the peripheral interface 1203 may be connected by a bus or signal lines. The individual peripheral devices may be connected to the peripheral device interface 1203 via buses, signal lines, or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1204, a display 1205, audio circuitry 1206, and a power supply 1207.

It will be appreciated by those skilled in the art that the structure shown in fig. 12 is not limiting and that more or fewer components than shown may be included or certain components may be combined or a different arrangement of components may be employed.

The embodiment of the application also provides a computer readable storage medium, and the storage medium stores a computer program, and the computer program is loaded and executed by a processor to realize the method for protecting the screen display content provided by the method embodiments.

The computer readable medium may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM (Random Access Memory ), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash Memory or other solid state Memory technology, DVD (Digital Video Disc, high density digital video disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the ones described above.

The embodiment of the application also provides a computer program product, which comprises a computer program, wherein the computer program is stored in a computer readable storage medium, and a processor reads and executes the computer program from the computer readable storage medium to realize the method for protecting the screen display content provided by the method embodiments.

It should be understood that references herein to "a plurality" are to two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and principles of the application are intended to be included within the scope of the application.

Claims

1. A method for protecting content on a screen, the method comprising:

2. The method of claim 1, wherein the creating an aggregate control that includes the content to be protected that the at least one control to be processed has comprises:

determining the position and size information of the aggregation control according to the position and size information of the at least one control to be processed;

and creating the aggregation control according to the position and the size information of the aggregation control.

3. The method of claim 2, wherein the player's location and size information is the same as the aggregation control's location and size information.

4. The method according to claim 1, wherein the method further comprises:

determining a public parent control of the at least one control to be processed from a control tree of the user interface, wherein the control tree is used for carrying out hierarchical management on the at least one control included in the user interface in a tree structure;

and adding the player to the public parent control to display the player when the public parent control is loaded.

5. The method of claim 4, wherein the determining the common parent control of the at least one pending control from the control tree of the user interface comprises:

determining parent controls corresponding to the at least one to-be-processed control respectively from the control tree, wherein the parent controls at least comprise the to-be-processed control;

under the condition that the father controls corresponding to the at least one to-be-processed control are the same control, determining the same control as the public father control;

creating a control set comprising a plurality of different controls under the condition that the father controls respectively corresponding to the at least one to-be-processed control are the plurality of different controls;

for a first control and a second control in any two different levels in the control set, wherein the level of the first control is higher than that of the second control, if the first control does not comprise the second control, determining a father control of the second control from the control tree, and replacing the second control in the control set by using the father control of the second control; if the first control comprises the second control, deleting the second control from the control set;

For any two third controls and fourth controls in the control set in the same level, determining a fifth control from the control tree, wherein the fifth control comprises the third control and the fourth control; replacing the third control and the fourth control in the control set by using the fifth control;

repeating the steps until one control remains in the control set, and determining the one control as the public parent control.

6. The method of claim 4, wherein the video conversion process is performed on the aggregation control, and after generating the video for simulating the display content of the aggregation control, the method further comprises:

and removing the at least one control to be processed from the control tree.

7. The method of claim 1, wherein the video conversion process for the aggregation control to generate a video for simulating the display content of the aggregation control comprises:

performing picture conversion processing on the aggregation control to obtain at least one picture corresponding to the aggregation control, wherein the picture comprises display content of the at least one control to be processed;

and generating the video for simulating the display content of the aggregation control according to the at least one picture.

8. The method according to claim 1, wherein the method further comprises:

storing the video if the frequency of occurrence of the at least one control to be processed in the user interface is higher than a first threshold;

and in the case that the at least one control to be processed is included in the other user interface, playing the video in the other user interface through a player in the process of displaying the other user interface.

9. The method of claim 1, wherein playing the video in the user interface by a player comprises:

generating a false invalid path according to a storage path of the video in the terminal equipment, wherein the false invalid path refers to a path which cannot be resolved correctly due to a path format error;

generating video information for forging the video into an encrypted video according to the false invalid path, wherein the video information is used for indicating to hide the video when the recording operation is detected;

correcting the path format errors existing in the false invalid paths to obtain false valid paths;

creating a video download task to load the video using the false effective path;

And playing the video through the player.

10. The method of claim 9, wherein creating a video download task to load the video using the false active path comprises:

triggering the player to request a download manager to create a download task through the false effective path;

under the condition that a download list of the download manager is free, notifying a resource manager by the download manager to load the video according to the false effective path;

determining, by the resource manager, the storage path from the false effective path;

the video is loaded from the storage path through the resource manager.

11. The method of claim 9, wherein the protocol identification in the false invalid path is used to indicate a wrong communication protocol name and the protocol identification in the false valid path is used to indicate a correct communication protocol name.

12. The method of claim 9, wherein generating video information that counterfeits the video to an encrypted video from the false invalid path comprises:

triggering the player to generate path analysis errors through the false invalid paths;

Sending the false invalid path to a play manager through the player, wherein the play manager is used for managing the play of at least one video;

generating, by the play manager, a key address for obtaining a fake key of the video according to the fake invalid path;

and generating video information which falsifies the video into an encrypted video according to the key address.

13. A device for protecting content on a screen, the device comprising:

14. A terminal device, characterized in that it comprises a processor and a memory, in which a computer program is stored, which computer program is loaded and executed by the processor to implement the method of protecting on-screen content according to any of claims 1 to 12.

15. A computer readable storage medium, wherein a computer program is stored in the storage medium, the computer program being loaded and executed by a processor to implement the method of protecting on-screen content according to any one of claims 1 to 12.

16. A computer program product, characterized in that it comprises a computer program stored in a computer readable storage medium, from which a processor reads and executes the computer program for implementing the method of protecting on-screen content according to any of claims 1 to 12.