CN111767554B

CN111767554B - Screen sharing method and device, storage medium and electronic equipment

Info

Publication number: CN111767554B
Application number: CN202010484148.2A
Authority: CN
Inventors: 张荣春
Original assignee: Oppo Chongqing Intelligent Technology Co Ltd
Current assignee: Oppo Chongqing Intelligent Technology Co Ltd
Priority date: 2020-06-01
Filing date: 2020-06-01
Publication date: 2024-02-20
Anticipated expiration: 2040-06-01
Also published as: CN111767554A

Abstract

The embodiment of the application discloses a screen sharing method, a device, a storage medium and electronic equipment, wherein the method comprises the following steps: detecting a screen sharing instruction aiming at a display interface, acquiring all display layers corresponding to the display interface, determining that at least one display layer in all display layers exists in a safe layer library, copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state, and carrying out shielding processing on the interface frame image and sharing. By adopting the embodiment of the application, the risk of privacy disclosure in the screen sharing process can be reduced, and the safety of screen sharing is improved.

Description

Screen sharing method and device, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a screen sharing method, a device, a storage medium, and an electronic apparatus.

Background

Along with the rapid development of communication technology, the application scene of the terminal (such as a mobile phone) is wider and wider, the basic communication and entertainment requirements of people are met, and meanwhile, richer resource storage and application expansion are provided, and the screen sharing function is one of the most widely used and frequent functions.

Currently, the screen sharing function of the terminal can be social sharing based on a social platform (such as an instant messaging platform), the terminal can record an interface frame image corresponding to a display interface on a screen, and then the interface frame image recorded on the screen is shared on the social platform; or, the user can establish communication connection with other terminals (such as televisions) through the terminal (such as a mobile phone), and then share the interface frame images corresponding to the display interfaces on the terminal to the other terminals in real time.

Disclosure of Invention

The embodiment of the application provides a screen sharing method, a device, a storage medium and electronic equipment, which can reduce the risk of privacy disclosure in the screen sharing process and improve the safety of screen sharing. The technical scheme of the embodiment of the application is as follows:

in a first aspect, an embodiment of the present application provides a screen sharing method, where the method includes:

detecting a screen sharing instruction aiming at a display interface, and acquiring all display layers corresponding to the display interface;

determining that at least one display layer in all display layers exists in a security layer library, and copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state;

And carrying out shielding treatment on the interface frame images and sharing the interface frame images.

In a second aspect, an embodiment of the present application provides a screen sharing device, where the device includes:

the image layer acquisition module is used for detecting a screen sharing instruction aiming at a display interface and acquiring all display image layers corresponding to the display interface;

the frame image copying module is used for determining that at least one display layer in all display layers exists in a safety layer library, copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state;

and the processing sharing module is used for carrying out shielding processing on the interface frame images and sharing the interface frame images.

In a third aspect, embodiments of the present application provide a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.

In a fourth aspect, embodiments of the present application provide an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

The technical scheme provided by some embodiments of the present application has the beneficial effects that at least includes:

in one or more embodiments of the present application, a terminal detects a screen sharing instruction for a display interface, acquires all display layers corresponding to the display interface, then determines that at least one display layer in all display layers exists in a secure layer library, copies an interface frame image corresponding to the display interface, where the interface frame image is a frame image to be shared in a screen sharing state, and performs shielding processing on the interface frame image and shares the interface frame image. Through the fact that all display layers contained in the display interface of the terminal are matched with the security layer library when the screen sharing is carried out on the display interface of the terminal based on the pre-established security layer library, namely, the terminal determines that at least one display layer in all display layers exists in the security layer library, privacy content can be related to the display interface, and the shielding treatment is carried out on the interface frame image corresponding to the display interface, so that privacy leakage risk in the screen sharing process can be reduced, and the security of the screen sharing is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a screen sharing method provided in an embodiment of the present application;

fig. 2 is a flowchart of another screen sharing method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a screen sharing device according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a processing sharing module according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a second shielding unit according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of another screen sharing device according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an operating system and user space provided by an embodiment of the present application;

FIG. 9 is an architecture diagram of the android operating system of FIG. 7;

FIG. 10 is an architecture diagram of the IOS operating system of FIG. 7.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present application, it is to be understood that the terms "comprise" and "have," and any variations thereof, are intended to cover non-exclusive inclusions, unless otherwise specifically defined and defined. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context. Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "or", describing the association relationship of the association object, means that there may be three relationships, for example, a or B, may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

In the related art, in the process of screen sharing of the interface frame image corresponding to the display interface, privacy content of a user may exist on the display interface, for example, the display interface relates to a bank card account number and a password of the user, and privacy content such as identity information and account balance of the user exists on the display interface, and at this time, if the screen sharing is directly performed on the frame image corresponding to the display interface, security risk of privacy content leakage exists; to avoid this, in the related art, a security attribute is set for a display interface related to personal privacy content in a terminal application, for example, taking an operating system of the terminal as an android system, and in a development or a later use process of the display interface of the application, the application interface is set as a security attribute. For example, in a bank login interface, in order to prevent an interface frame image related to an account password from being shared out in the process of inputting the bank password, the bank login interface is set to be a secure attribute, and then in the screen sharing process, the interface frame image corresponding to the bank login interface is invisible to the outside, i.e. the interface frame image corresponding to the bank login interface is not shared out, or the shared interface frame image is a black display picture. However, in this way, many display interfaces related to the private content of the user on the terminal are not all provided with security attributes in the development process of a developer or in the later terminal use process, so that security risks of privacy disclosure exist in the screen sharing process, and security in the screen sharing process is reduced.

Furthermore, in the screen sharing process, if the user relates to a scene of inputting characters by using the input method application on the display interface, such as inputting a password when logging in a bank, inputting some private information by using the input method application, and the like, in the related art, since the input method application relates to a use field Jing Jiaoduo of the user (even the screen sharing process also relates to the inputting of the characters), the safety attribute is not set for the input method interface generally.

The present application is described in detail with reference to specific examples.

In one embodiment, as shown in fig. 1, a screen sharing method is specifically proposed, which may be implemented by a computer program and may be executed on a screen sharing device based on von neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application. Wherein, the screen sharing device in the embodiment of the application can be a terminal,

the terminal may be an electronic device with a screen sharing function, including but not limited to: wearable devices, handheld devices, personal computers, tablet computers, vehicle-mounted devices, smart phones, computing devices, or other processing devices connected to a wireless modem, etc. Terminal devices in different networks may be called different names, for example: a user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent or user equipment, a cellular telephone, a cordless telephone, a personal digital assistant (personal digital assistant, PDA), a terminal device in a 5G network or a future evolution network, and the like.

Specifically, the screen sharing method includes:

step S101: and detecting a screen sharing instruction aiming at a display interface, and acquiring all display layers corresponding to the display interface.

The display page may generally include an interface for displaying display elements such as text, pictures, animation, tables, icons, etc., and in practical application, the display page on the terminal may generally be a display page corresponding to a certain application program.

The screen sharing instruction is an instruction or a requirement for instructing the terminal to perform screen sharing, and can be understood as a code for instructing the terminal to perform a "screen sharing operation on a display interface" or some control of realizing a function. In practical application, when detecting a screen sharing instruction of a user aiming at a display interface, the terminal responds to the screen sharing instruction and executes a code for performing screen sharing operation or a certain control of function realization on the display interface, and all display layers corresponding to the display interface of the terminal are acquired first.

The display layers may be, for example, films containing text or graphics, and the like, stacked together in a sequence of Zhang An sheets, which combine to form the final effect of the display interface. When rendering and displaying the display interface, the terminal can accurately position the elements to be displayed on the display interface through displaying the layer, and can add corresponding texts, pictures, tables, plug-ins and the like into the layer, and can nest the layer in the layer. In practical application, the display layers are layers forming the terminal display interface, the number of the display layers can be one or more, and in the display process of the display interface, a plurality of display layers are overlapped to finally form the terminal display interface.

In a specific implementation scenario, the terminal has a screen sharing (i.e. display interface sharing) function, such as a wifi display screen sharing function, and the terminal display interface is projected to other terminals for display through the HDMI function. The screen sharing function can facilitate the content and form of multimedia information communication among different terminal users, and it can be understood that the content information can be used as resources for sharing communication with terminals of other users as long as the content information can be displayed on a display interface of a terminal screen. Such as sharing display content of an application display interface on a terminal or an application game interface on a terminal, sharing a video display interface played on a terminal, and so on.

In a specific implementation scenario, the screen sharing of the terminal to the current display interface may be implemented through a screen recording function (i.e. screen recording), and when a user of the terminal needs to share an operation process on the terminal and some display interfaces of the terminal with users of other terminals, the screen recording function of the terminal is usually started to record the display interfaces of the terminal, so as to obtain a corresponding video file containing a frame image corresponding to the display interfaces. The screen recording function can be a screen recording function in a terminal system or a screen recording function of a screen recording application program installed in the terminal; the video file includes at least the screen during operation. Optionally, the video file further includes environmental background audio collected during the screen recording process or audio played in the terminal.

Specifically, the terminal may provide a control or a function option corresponding to the screen sharing function for the user, when the user needs to share the display interface of the terminal, for example, in a screen recording manner on the display interface, the terminal may monitor the control or the function option corresponding to the screen sharing function of the display interface, and when the terminal detects a screen sharing instruction input by the user for the control or the function option corresponding to the screen sharing function of the display interface, the terminal responds to the screen sharing instruction, and executes a code for performing a screen sharing operation on the display interface or implementing a certain control of the function, so as to first obtain all display layers corresponding to the display interface of the terminal. Specifically, the terminal may obtain the layer identifiers of all the display layers corresponding to the display interface.

The screen sharing instruction may be input on a current display interface of the terminal, for example, sliding, clicking, double clicking or long pressing a screen sharing control icon, pressing a physical key corresponding to a preset screen sharing function, shaking the terminal to input an operation of the screen sharing function in a gesture sense mode, or the like, or may be a screen sharing instruction sent to the terminal through communication connection between other intelligent devices, for example, a screen sharing instruction sent to the terminal through an infrared remote controller, a laser pen, an electronic pointer, or the like.

In particular, the terminal may have a touch screen, which may be a touch screen capable of implementing a single touch function, for example: the capacitive touch screen and the electromagnetic touch screen can also be touch screens capable of simultaneously realizing capacitive induction, electromagnetic induction and infrared induction. When a user touches a touch screen on a terminal through a finger, capacitance parameters of the touch position change, and the touch frame determines the touch position of the finger on a display interface of the terminal according to the change of the capacitance; or when the finger touches the touch frame, the infrared receiving end is blocked from receiving infrared signals, the touch frame determines the touch position of the finger on the display interface according to the blocked infrared signals, and the terminal can detect a screen sharing instruction input by a user when a control corresponding to a screen sharing function or a function option is triggered in the display interface by identifying a code corresponding to a logic control instruction corresponding to the touch position.

Optionally, the screen sharing instruction input by the user on the terminal display interface may be an icon of a screen sharing control selected by an external device mouse connected with the terminal; the voice signal of the screen sharing input by the user is collected through the voice collecting device of the terminal, such as a voice input of a small assistant, which helps me to share the screen.

It should be noted that, the manner of the screen sharing instruction input by the user for the display page may be one or more of the above, which is not limited herein.

Specifically, after detecting a screen sharing instruction input by a user for a display page, the terminal responds to the screen sharing instruction, in the process of screen sharing of the display page, all display layers corresponding to the display interface can be obtained, whether the display interface contains content related to privacy security of the user (such as bank card information, personal account information, health information and other personal privacy information) or not in the process of determining the screen sharing based on the display layers, and specifically, the terminal can obtain layer identifiers of all the display layers.

Wherein the layer identifier may be understood as indication information for uniquely characterizing a layer of the display interface, and typically the layer identifier may be a uniquely characterized layer id, for example, numbers 1, 2, 3, etc. representing ids; key characters that may be uniquely characterized, such as a, b, c, etc.; may be key strings that uniquely characterize the layers, such as pth_a, pth_b, pth_c; etc. Further, for each layer included in the display interface of the current terminal, a layer identifier corresponds to each layer identifier, where the layer identifier may be an identifier set by default in the system or a customized identifier. For example, the LAYER identifier may be defined by a developer during the development of the display interface programming, for example, the LAYER corresponding to one sidebar display element may be represented by layer_time_system-DOCK, and the LAYER corresponding to one TOOLBAR display element may be represented by layer_time_mark_TOOLbar.

In practical applications, the terminal may acquire the layer identifiers of all display layers by calling a related layer identifier acquiring function, and one way may be to acquire all layers corresponding to the display interface by dumpsys SurfaceFlinger function call, for example, through an adb command line: adb shell dumpsys SurfaceFlinger; one way may be that the prv_getindixoflayer () function call may obtain the identity of each layer; one way may be a getWindow (). GetDecoreView () function call, which determines the layer identity of all display layers by listening for the return value of return getAllChildViews (); etc.

Step S102: and determining that at least one display layer in all display layers exists in a security layer library, and copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state.

The security layer library comprises a plurality of security layers, wherein the security layers can be stored by default when the terminal leaves the factory, and can be added to the security layer library by self definition when a maintainer of the terminal detects the security privacy of each display interface of the common application in the later use of the terminal, and the security layer library can be understood as a database of each security layer corresponding to the display interface of the terminal approved by the relevant maintainer or administrator.

Specifically, after the terminal obtains all display layers corresponding to the display interface, based on the security layer library, security privacy detection is automatically performed on all display layers, that is, in all display layers, whether display layers consistent with all security layers in the security layer library exist or not is judged:

1. if one display layer is not a security layer in the security layer library in all display layers, determining that the display interface does not contain content related to user security privacy, and if the terminal directly shares the display interface, the security risk of user privacy leakage does not exist, and at the moment, the terminal directly shares the screen with an interface frame image generated after copying the display image of the display interface.

2. If at least one display layer in all display layers is a security layer in the security layer library, it is determined that the display interface includes content related to user security privacy, for example, the display interface may be a password input interface (such as a bank card password input), that is, there is a layer corresponding to account password input (such as a layer corresponding to an input box for password input). At this time, if the terminal directly shares the screen with the display interface, that is, if the terminal directly shares the screen with the interface frame image generated after the terminal copies the display screen, there is a security risk of privacy disclosure.

Specifically, the security layer library stores security layer identifiers for uniquely characterizing the security layers, and it is understood that the process of determining that at least one display layer of the all display layers exists in the security layer library by the terminal is performed based on the layer identifiers of all display layers, that is, whether at least one layer identifier exists as a security layer identifier in the security layer library in the layer identifiers of all display layers, and if so, determining that the display interface contains content related to user security privacy; otherwise, if the content does not exist, the display interface is determined to not contain the content related to the safety privacy of the user.

Further, the security layer library is a database storing security layer identifiers corresponding to the security layers, and the security layer library can be set according to an actual application environment, for example, the security layer library can be set in a local storage space of the terminal, so that the terminal can conveniently and quickly compare the security layer library after acquiring the layer identifiers of all display layers; for example, the security layer library may be disposed on a server that establishes communication connection with the terminal, where the security layer library is disposed on the server, and can save resources (such as storage space) of the terminal when the space occupied by the security layer library is large, after the terminal obtains the layer identifiers of all display layers, the terminal may send a detection request for the layer identifiers of all display layers to the server, and the server receives the detection request to perform correspondence, automatically performs security privacy detection on all display layers based on the security layer library, and then sends a detection result to the terminal, where the terminal can measure the risk of directly sharing the display interface according to the detection result.

Specifically, after determining that at least one display layer in the all display layers exists in the secure layer library, the terminal generally includes content related to user security privacy, and first copies a display image of the display interface to obtain an interface frame image corresponding to the display interface.

Step S103: and carrying out shielding treatment on the interface frame images and sharing the interface frame images.

In a specific embodiment, after determining that at least one display layer in the all display layers exists in the secure layer library and copying the interface frame image corresponding to the display interface, in order to avoid related privacy content (such as a bank card password, user short message content, etc.) on the current display interface, before performing occlusion processing on the interface frame image corresponding to the display interface, the terminal may determine an occlusion area for performing occlusion processing first, and when determining that the display interface includes content related to user security privacy based on the method in step S102, the terminal may use all display content of the interface frame image as an occlusion area for which occlusion processing is required, that is, all image areas of the interface frame image are occlusion areas, and then perform occlusion processing on the occlusion areas.

In a specific embodiment, a plurality of display elements are generally included in an interface frame image corresponding to a display interface, for example, the interface frame image includes display elements such as characters, pictures, input frames, icons, tables and the like on the display interface, a target display element related to privacy content generally only belongs to a part of the interface frame image, a terminal can take an area corresponding to the target display element related to privacy content as an occlusion area, then the occlusion area is subjected to occlusion processing, and a display area which does not belong to the occlusion area in the interface frame image is not occluded, so that accurate occlusion related to user privacy content can be realized.

The method for determining the shielding area corresponding to the privacy content in the interface frame image comprises the following steps: when the terminal renders and displays the display interface, the terminal can be generally regarded as a film (i.e., display layer) containing display elements such as characters or graphics, and one Zhang An film is sequentially stacked together to form the display interface, i.e., the display interface corresponds to a plurality of display layers, such as a display layer for displaying text, a display layer for displaying pictures, a display layer for displaying interface controls, and a display layer for displaying input frames (such as passwords and account number input frames), in the step S102, the terminal can determine the security layer belonging to the safe layer library in all the display layers, the terminal can acquire the layer information of the corresponding security layer, the layer information generally contains the display position information of the display element related to the privacy content in the interface, for example, a certain security layer is the display layer corresponding to the input frame on the display interface, and at this time, the terminal can acquire the layer information of the display layer, then can reach the display position information corresponding to the display element (input frame) based on the layer information, and can understand that the terminal can determine the display element corresponding to the privacy content on the basis of the display position information in the above-mentioned step S102, the terminal can acquire the layer information corresponding to the target display element on the display layer, and can do not belong to the target region of the display region as the target region.

A method for determining an occlusion region corresponding to private content in an interface frame image: privacy content detection may be performed on display elements of the interface frame image, so as to determine occlusion regions corresponding to the display elements related to the privacy content.

If the text region corresponding to the privacy display element of the user is determined, the terminal can acquire text information related to the interface frame image by adopting a text recognition method based on Optical Character Recognition (OCR) on the interface frame image, and perform key character recognition on the text information, so that key characters related to the text information are determined, and the region where the key characters are located in the interface frame image is used as a shielding region;

for another example, determining an image area corresponding to the privacy display element of the user, the terminal may perform image recognition on the interface frame image, thereby determining a target image area related to the privacy of the user in the interface frame image, and taking the target image area as an occlusion area, where the method for performing image recognition on the interface frame image may be that the interface frame image is input into a pre-trained image recognition model, and the occlusion area corresponding to the output interface frame image, for example, the image recognition model may be implemented based on one or more of LR (Logistic Regression, logistic regression model), SVM (Support Vector Machine ), decision tree, naive bayesian classifier, CNN (Convolutional Neural Network ), RNN (Recurrent Neural Networks, recurrent neural network), and the like, and training the initial image recognition model based on the image sample data with the occlusion area marked, so as to obtain a trained image recognition model.

The shielding processing can be that the terminal can perform interface rendering on the shielding area according to a preset rendering color.

The preset rendering color can be understood as a color preset by the terminal for realizing shielding of the content related to the privacy of the user in the shielding area, and after the interface rendering is completed on the shielding area, the original display content of the shielding area is not visible to the user. For example, the preset rendering color may be black, blue, red, etc., in the implementation, when the terminal performs interface rendering on the shielding region according to the preset rendering color, the terminal determines pixel points corresponding to the shielding region in the interface frame image first, and then performs one-to-one rendering on the pixel points according to the preset rendering color.

The terminal can superimpose and display preset shielding images at the shielding area, the preset shielding images can be understood as images preset by the terminal for shielding the content related to the privacy of a user in the shielding area, the terminal can set a shielding image library which comprises a plurality of shielding images, the sizes of the shielding images are generally different, the terminal can obtain preset shielding images matched with the area size in the shielding image library based on the area size of the shielding area, and then superimpose and display the preset shielding images at the shielding area, for example, the terminal can newly establish a shielding image layer on an interface frame image, and superimposedly display the preset shielding images at the position indicated by the shielding area on the shielding image layer; or when the terminal displays the preset shielding image in a superposition manner at the shielding area, the terminal can perform image processing on the preset shielding image based on the area size of the shielding area, such as equal-scale zooming, image clipping and the like.

An occlusion process may be that a terminal may blur a corresponding image at the occlusion region. Specifically, after determining the blocking area of the interface frame image, the terminal performs blurring processing on the blocking area on the interface frame image, where the blurring processing may be gaussian blurring processing, central blurring processing, or the like, and the purpose of the blurring processing is to reduce the difference between pixels between every two pixels of the blocking area, so as to achieve the effect of visual blurring at the blocking area, and the terminal obtains the interface frame image after the blurring processing of the blocking area, where the blocking area of the interface frame image after the blurring processing has an effect similar to that of frosted glass.

Specifically, after the terminal performs shielding processing on the interface frame image, a processed interface frame image is obtained, then screen sharing is performed on the interface frame image, a specific sharing mode is determined based on the actual application environment of the terminal, for example, if the screen sharing of the terminal is based on a display screen sharing function and an HDMI function, a terminal display interface is projected to other terminals for display, which is also called screen sharing, and at this time, the terminal shares the interface frame image to be processed with other terminals through communication connection. For another example, if the screen sharing of the terminal is implemented based on the screen recording function (i.e. screen recording), the terminal may store the processed interface frame image as a frame of video image corresponding to the screen recording function in a video file manner. Further, after the screen recording is finished, the terminal can share the video file to other terminals.

In the embodiment of the application, a terminal detects a screen sharing instruction for a display interface, acquires all display layers corresponding to the display interface, then determines that at least one display layer in all display layers exists in a safe layer library, copies an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state, and performs shielding processing and sharing on the interface frame image. Through the fact that all display layers contained in the display interface of the terminal are matched with the security layer library when the screen sharing is carried out on the display interface of the terminal based on the pre-established security layer library, namely, the terminal determines that at least one display layer in all display layers exists in the security layer library, privacy content can be related to the display interface, and the shielding treatment is carried out on the interface frame image corresponding to the display interface, so that privacy leakage risk in the screen sharing process can be reduced, and the security of the screen sharing is improved.

Referring to fig. 2, fig. 2 is a flowchart illustrating another embodiment of a screen sharing method according to the present application. Specific:

step S201: and acquiring at least one application interface corresponding to the application, and detecting privacy content of all display layers corresponding to the application interfaces of the applications.

The application is a commonly used application program on the terminal, the application program can be an application self-contained in a terminal operating system and can be a third party application, and the third party application refers to an application self-contained in a non-terminal operating system developed by a third party and comprises some applications, applets, plug-ins and the like developed by the third party. In this embodiment, the third party application may be understood as an application other than an application of the operating system preset system of the terminal, such as an application program of email, instant messaging, e-commerce, and the like.

The application interface is a display interface for man-machine interaction, which is displayed on the terminal after the application is started, the application interface can generally include at least one of display elements such as display text, picture, animation, form, icon, etc., the process of displaying the application interface by the terminal is generally a rendering display process of all display layers included in the application interface, it can be understood that in practical application, the display layers are layers forming the application interface, the number of the display layers can be one or more, and in general, in the process of displaying the application interface, the application interface is finally formed by superposing a plurality of display layers.

In the embodiment of the application, the private content detection is mainly performed on each display layer, and the private content includes private information and private video, for example, fitting of one or more of business secret files and video, authentication information, bank account, mail, address book contact, personal picture, personal account, password login content, personal health information, chat information and the like. The private content may also include user-defined content. By performing private content detection on all display layers of the application interface, i.e. determining the layer containing the private content, the layer containing the private content can be generally called a security layer, i.e. a layer related to the privacy security of the user. Such as a display layer corresponding to a user bank password input box, display content corresponding to a user personal account display column, and the like.

Specifically, the terminal detects the private content of the display layer, namely, detects the text content of the display layer, specifically, acquires text information in the display layer, carries out key character recognition on the text information, thereby determining whether key characters are involved in the text information, wherein the key characters are preset for the private content, a key character library can be set for the private content, key character matching is carried out on the text information based on the key character library in specific implementation, whether the text information contains key characters in the key character library is judged, and when the text information contains the key characters, the terminal determines that the display layer corresponding to the text information is a target layer containing the private content; or, the terminal may detect the private content of the display layer, for example, detect whether the display layer contains the private content of the user (such as a user image) or not, specifically, the terminal may perform image privacy detection on the image content in the display layer, so as to determine whether the image content in the display layer relates to the image of the user privacy, for example, input the display layer into a pre-trained image privacy detection model, output an image privacy detection result corresponding to the display layer, and determine whether the display layer contains the private content of the user based on the image privacy detection result, and when determining whether the display layer contains the private content of the user, determine that the display layer corresponding to the text information is the target layer containing the private content.

Optionally, the image privacy detection model may be implemented based on fitting one or more of a convolutional neural network (Convolutional Neural Network, CNN) model, a deep neural network (Deep Neural Network, DNN) model, a cyclic neural network (Recurrent Neural Networks, RNN) model, an embedded (ebedding) model, a gradient lifting decision tree (Gradient Boosting Decision Tree, GBDT) model, a logistic regression (Logistic Regression, LR) model, and the like, and meanwhile, an error back propagation algorithm is introduced to optimize based on the existing neural network model, so that the recognition accuracy of the initial image privacy detection model based on the neural network model may be improved. In the training stage of the initial image privacy detection model, a large amount of image sample data corresponding to the display image layer can be obtained in advance, the initial image privacy detection model is trained based on the marked image sample data, and the trained image privacy detection model can be obtained.

Step S202: and determining target layers containing private contents in all display layers corresponding to the application interface, and adding the target layers into a security layer library.

Specifically, after collecting at least one application interface corresponding to an application, the terminal detects private content on all display layers corresponding to the application interfaces of the applications, determines a target layer containing the private content in all display layers corresponding to the application interface based on a private content detection result, and then adds the target layer to a secure layer library. In this embodiment of the present application, based on dimensions of display layers of a display interface, privacy content detection is performed on all layers of the display interface of a terminal commonly used application in advance, so that a target layer containing privacy content is determined, after the target layer is added to a secure layer library, detection may be conveniently performed on the display layer corresponding to the display interface and the secure layer library when the terminal is in a screen sharing state in a later use process of the terminal, and when the display layer is determined to be a layer in the secure layer library, the terminal may determine that the display interface contains content of user security privacy, for example, the display interface may be a password input interface (such as a bank card password input), that is, there is a layer corresponding to account password input (such as a layer corresponding to an input frame for password input).

Specifically, in practical application, the security layer library stores security layer identifiers for uniquely characterizing the security layers (such as target layers), and it is understood that the terminal determines target layers including private content in all display layers corresponding to the application interface, and adds the layer identifiers corresponding to the target layers to the security layer library.

Further, the acquiring an application interface corresponding to at least one application, performing privacy content detection on all display layers corresponding to the application interfaces of the applications, determining a target layer containing privacy content in all display layers corresponding to the application interfaces, and adding the target layer to a secure layer library may be performed on a server which establishes communication connection with a terminal, and the server may be a server which is included in a secure service provider, where the application secure service provider may be a certain antivirus software, a certain secure manager application, a certain application cleaning master, and the like, and performing privacy content detection on all display layers through a display layer risk detection service provided by the secure service provider. The terminal can synchronize the security layer libraries on the server regularly or in real time, so that the latest security layer library on the server is obtained, the security layer libraries are arranged in the local storage space of the terminal, and the terminal can conveniently and quickly compare the security layer libraries after obtaining the layer identifiers of all display layers; or the security layer library may be disposed on a server that establishes communication connection with the terminal, where the security layer library is disposed on the server, and can save resources (such as storage space) of the terminal when the space occupied by the security layer library is large, and after the terminal obtains the layer identifiers of all display layers, the terminal may send a detection request for the layer identifiers of all display layers to the server, and the server receives the detection request to perform correspondence, automatically performs security privacy detection on all display layers based on the security layer library, and then sends a detection result to the terminal.

Step S203: and detecting a screen sharing instruction aiming at a display interface, and acquiring all display layers corresponding to the display interface.

See step S101, and will not be described here.

Step S204: determining that at least one display layer of the total display layers exists in the secure layer library.

See step S102, which is not described here.

Step S205: when the input method application is detected to be called on the display interface, copying an interface frame image corresponding to the input method application, and carrying out shielding treatment on the interface frame image corresponding to the input method application.

The input method application is software which can be operated on a terminal for realizing text input, and is also called input method software, an input method platform, an input method framework or an input method system. The input method is generally defaulted to have a certain coding mode, such as pinyin coding in a Chinese input method, namely a so-called pinyin input method. The software names of the input methods are the code names of the input methods, such as a Wang code five-stroke character input method and a super-strong two-stroke input method.

In the embodiment of the application, when a user needs to input privacy contents such as corresponding characters, numbers and the like based on actual use conditions, for example, an account password login interface needs to input characters, numbers and the like, the process needs to call an input method application by a terminal to realize the user to input the characters, and in the process, because all layers of a display interface corresponding to the input method application are usually not present in a layer security library (because the input method application user uses more frequently, each terminal application scene can be involved, a related layer of the display interface corresponding to the input method application is usually not added to the layer security library in the layer security library creation stage), if the display interface corresponding to the input application is subjected to screen sharing, if the sequence of inputting the characters (characters, numbers and the like) in the input method is memorized, the privacy content of the user is at risk.

At this time, the terminal may monitor the display interface after the "determining that at least one display layer of the all display layers exists in the secure layer library" and load the display interface corresponding to the input method application when the input method application is called at the display interface is monitored, where a loading manner is: loading an input method interface of an input method application on a current display interface, wherein the input method interface and the current display interface are displayed in a superposition manner, the input method interface is visually displayed in a part of a display area of a terminal display interface, the display area except for the display input method displays page display contents before the input method is called, for example, the input method is displayed in a lower area of the terminal display interface, and the upper area displays page display contents (such as a login frame, character login information and the like) before the input method is called, and one loading manner is that: the input method interface of the input method application is loaded on the current display interface, the input method interface is displayed on the terminal display interface entirely, namely, only display contents related to the input method are displayed on the terminal display interface, and at the moment, page display contents before the input method is called are not displayed on the terminal display interface generally.

The terminal can copy the display screen corresponding to the input method application, and then obtain an interface frame image corresponding to the display interface, namely the interface frame image corresponding to the input method application, in this embodiment of the present application, the related screen sharing scene is generally that the interface frame image after the display screen is copied is shared, for example, the frame interface frame image is saved in a video file form in a screen recording manner to facilitate sharing, and is sent to an electronic device (such as other terminals) for screen sharing in a screen sharing manner, for example, when the copied interface frame image is obtained, the frame image cannot be directly used for screen sharing including the interface frame image of the input method application at this time, and the terminal can perform shielding processing on the interface frame image corresponding to the input method application, so that the security risk of privacy leakage caused by directly carrying out screen sharing on the obtained interface frame image including the input method application is avoided.

In order to avoid relevant privacy content (such as a bank card password input on the input method, a short message content input on the input method, and the like) on a display interface corresponding to the current input method, before the terminal performs shielding processing on an interface frame image corresponding to the display interface, the terminal can determine a shielding area for performing the shielding processing first, can take all display content of the interface frame image as the shielding area for which the shielding processing is required, namely, all image areas of the interface frame image are shielding areas, and then performs shielding processing on the shielding area; or, the region corresponding to the input method of the interface frame image can be used as a shielding region, and then the region where the input method is located is shielded.

The step S102 may refer to the process of shielding the interface frame image, and will not be described herein.

Step S206: and copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state.

See step S102, which is not described here.

Step S207: and determining all display pictures of the interface frame images as shielding areas.

According to some embodiments, after determining that at least one display layer in the all display layers exists in the secure layer library and copying the interface frame image corresponding to the display interface, in order to avoid related privacy content (such as a bank card password, a user short message content, and the like) on the current display interface, before performing occlusion processing on the interface frame image corresponding to the display interface, the terminal may determine an occlusion area for performing the occlusion processing first, and the terminal may use all display content of the interface frame image as an occlusion area required to perform the occlusion processing, that is, all image areas of the interface frame image are occlusion areas.

Step S208: and determining an occlusion region corresponding to the privacy content in the interface frame image.

Specifically, the terminal may perform risk detection on the display element of the interface frame image, then determine a risk display element, and use a display area corresponding to the risk display element as a shielding area.

The risk display element may be understood as a display element related to the private content, and typically, the risk display element may be a display element such as a text, a picture, an input box, an icon, a table, a control, and the like related to the private content.

Specifically, the terminal may train a risk detection model for risk detection of the display element, input the interface frame image as a model to the risk detection model by the terminal to perform model identification, and output a risk detection result of the interface frame image, where the risk detection result includes a risk display element in the interface frame image and a display area corresponding to the risk display element on the display interface. And the terminal can take the display area corresponding to the risk display element as a shielding area based on the risk detection result.

In practical application, the terminal may create an initial risk detection model, and may acquire all or part of sample images from an existing image database (such as an existing CUHKPQ gallery, an AVA gallery, etc.) and/or acquire sample images (i.e., a large number of interface frame images) captured in a practical environment by using a device with a photographing function, and intercept interface frame images in a practical application environment by using a screen capturing function of the terminal, where the interface frame images are used as sample images. The method comprises the steps of pre-obtaining a large number of sample images, preprocessing the sample images, wherein the preprocessing comprises the processing procedures of digitalization, geometric transformation, normalization, smoothing, restoration enhancement and the like, eliminating irrelevant information in the sample images, extracting image features, inputting the image features into an initial risk detection model for training, and obtaining a trained risk detection model.

Alternatively, the risk detection model may be a deep learning based image detection method, such as a Cascade convolutional neural network (Cascade Convolutional Neural Network, cascade CNN), a fast regional convolutional neural network (Faster Regions with CNN features, fast RCNN), a RetinaFace image detection model based on a conventional object detection network (RetinaNet), and so on. Compared with the traditional image detection method (such as a rigid template method), the characteristics extracted by the deep neural network have stronger robustness and description capability.

It should be noted that, in the embodiment of the present application, the risk detection model may be one or more of the fitting of the foregoing, which is not specifically limited herein.

Step S209: and obtaining the shielding level corresponding to the shielding area, and determining the shielding processing mode corresponding to the shielding level.

Specifically, the terminal performs risk detection on the display elements of the interface frame image, then determines risk display elements, takes a display area corresponding to the risk display elements as a shielding area, can acquire shielding levels corresponding to the shielding area, and then determines a shielding processing mode with the shielding levels according to the shielding levels.

A method for acquiring the shielding level corresponding to the shielding area comprises the following steps: the terminal can determine the shielding level based on the area of the shielding area, and it can be understood that the larger the area of the shielding area is, the more the user privacy content related to the shielding area is, the higher the shielding level corresponding to the shielding area is, the mapping relation between the shielding level and the shielding processing mode is stored in advance, and the shielding processing mode corresponding to the shielding level can be determined based on the mapping relation.

For example, the area of the area corresponding to the shielding region may be represented according to the area ratio of the shielding region to the interface frame image, and the terminal may set in advance: the shielding area with the area ratio below 0.5 corresponds to the shielding treatment mode A; the shielding area with the area ratio between 0.5 and 0.7 corresponds to the shielding treatment mode B; and the shielding region with the area ratio between 0.7 and 1 corresponds to the C shielding processing mode, and if the terminal determines that the shielding region occupies the area ratio of the interface frame image to be 0.65, the terminal performs shielding processing on the shielding region according to the B shielding processing mode.

A method for acquiring the shielding level corresponding to the shielding area comprises the following steps: the terminal may determine based on the type of risk display element of the occlusion region, and the types of common risk display elements may be: the terminal may set different or the same occlusion levels for different types of display elements, such as setting a text type to an a-level, a picture type to a B-level, an input box type to an a-level, and so on. The terminal pre-stores the mapping relation between the shielding level and the shielding processing mode, and based on the mapping relation, the shielding processing mode corresponding to the shielding level can be determined.

A method for acquiring the shielding level corresponding to the shielding area comprises the following steps: the terminal may determine the occlusion level corresponding to the occlusion region based on the privacy content type contained in the occlusion region, e.g., the terminal may use a file, picture, video, etc. related to the trade secret as an a privacy type, authentication information (e.g., a short message authentication code) as a B privacy type, password login content, a bank or financial account as a C privacy type, a personal life photo or picture D as a privacy type. The terminal pre-stores the mapping relation between the shielding level and the shielding processing mode, and based on the mapping relation, the shielding processing mode corresponding to the shielding level can be determined.

A method for acquiring the shielding level corresponding to the shielding area comprises the following steps: the terminal can be pre-trained with an occlusion level determining model, the occlusion level determining model can be the same model as the risk detection model, when the occlusion level determining model is trained, the image layer sample data is marked (namely, the occlusion level is marked) based on a large amount of image sample data corresponding to the interface frame images obtained in advance, and the initial occlusion level determining model is trained based on the marked image sample data, so that the trained occlusion level determining model can be obtained. The terminal inputs the interface frame image into the shielding level determining model for model identification, and can output the detection result of the interface frame image, wherein the detection result can comprise the shielding level of the interface frame image, the terminal stores the mapping relation between the shielding level and the shielding processing mode in advance, and the shielding processing mode corresponding to the shielding level can be determined based on the mapping relation.

The shielding processing mode can be that interface rendering is carried out on the shielding area according to preset rendering colors; a preset shielding image can be displayed in a superimposed mode at the shielding area; the blurring process can be performed on the corresponding image at the shielding area; displaying preset prompt characters (such as privacy is related to the current area, shielding treatment is performed) at the shielding area, and setting the background of the characters to be in an opaque state; it may be to acquire the previous frame history interface frame image (image not containing the privacy content), intercept the picture of the partial display area of the history interface frame image to cover the current interface frame image, and so on.

Step S210: and carrying out shielding treatment on the shielding region based on the shielding treatment mode and sharing the shielding region.

See step S103, and will not be described here.

In the embodiment of the application, a terminal detects a screen sharing instruction for a display interface, acquires all display layers corresponding to the display interface, then determines that at least one display layer in all display layers exists in a safe layer library, copies an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state, and performs shielding processing and sharing on the interface frame image. Through the fact that all display layers contained in the display interface are matched with the security layer library when the screen sharing is carried out on the display interface of the terminal based on the pre-established security layer library, namely, the terminal determines that at least one display layer in all display layers exists in the security layer library, the display interface can be determined to relate to privacy content, and the shielding treatment is carried out on the interface frame image corresponding to the display interface, so that privacy leakage risk in the screen sharing process can be reduced, and the security of the screen sharing is improved; and under the condition that at least one display layer in all display layers exists in the security layer library, if the input method application is detected to be called on the display interface, the terminal can conduct shielding processing on the interface frame image corresponding to the input method application, so that the risk of privacy leakage caused by calling the input method application in the screen sharing process can be reduced; and when the interface frame image is subjected to shielding treatment, all pictures of the interface frame image can be subjected to shielding treatment, or shielding areas corresponding to privacy contents in the interface frame image are shielded, so that shielding treatment modes are enriched, and the intelligent degree of screen sharing shielding treatment modes is improved.

The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Fig. 3 is a schematic structural diagram of a screen sharing device according to an exemplary embodiment of the present application. The screen sharing device may be implemented as all or part of the device by software, hardware, or a combination of both. The device 1 comprises a layer acquisition module 11, a frame image copying module 12 and a screen sharing module 13.

The layer acquisition module 11 is configured to detect a screen sharing instruction for a display interface, and acquire all display layers corresponding to the display interface;

the frame image copying module 12 is configured to determine that at least one display layer in the all display layers exists in a secure layer library, copy an interface frame image corresponding to the display interface, where the interface frame image is a frame image to be shared in a screen sharing state;

and the processing sharing module 13 is used for carrying out shielding processing on the interface frame images and sharing the interface frame images.

Optionally, as shown in fig. 6, the apparatus 1 further includes:

the privacy detection module 14 is configured to collect at least one application interface corresponding to an application, and detect privacy content for all display layers corresponding to the application interfaces of the applications;

And the secure layer library adding module 15 is configured to determine a target layer containing private content in all display layers corresponding to the application interface, and add the target layer to the secure layer library.

Optionally, the frame image copying module 12 is further configured to:

when the input method application is detected to be called on the display interface, copying an interface frame image corresponding to the input method application;

the process sharing module 13 is further configured to:

and determining target layers containing private contents in all display layers corresponding to the application interface, and adding the target layers into a security layer library.

Optionally, as shown in fig. 4, the process sharing module 13 includes:

a first shielding unit 131, configured to determine that all display pictures of the interface frame image are shielding areas, and perform shielding processing on the shielding areas;

and the second shielding unit 132 is configured to determine a shielding area corresponding to the privacy content in the interface frame image, and perform shielding processing on the shielding area.

Optionally, as shown in fig. 5, the second shielding unit 132 is specifically configured to:

a risk element determining subunit 1321, configured to perform risk detection on a display element of the interface frame image, and determine a risk display element;

The occlusion region determining subunit 1322 is configured to take a display region corresponding to the risk display element as an occlusion region.

Optionally, the process sharing module 13 is specifically configured to:

performing interface rendering on the shielding area according to a preset rendering color; or alternatively, the first and second heat exchangers may be,

superposing and displaying a preset shielding image at the shielding area; or alternatively, the first and second heat exchangers may be,

and carrying out blurring processing on the corresponding image at the shielding area.

It should be noted that, when the screen sharing device provided in the foregoing embodiment performs the screen sharing method, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the screen sharing device and the screen sharing method embodiment provided in the above embodiments belong to the same concept, which embody detailed implementation procedures in the method embodiment, and are not described herein again.

The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

The embodiment of the present application further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executed by the processor, where the specific execution process may refer to the specific description of the embodiment shown in fig. 1-2, and details are not repeated herein.

The present application further provides a computer program product, where at least one instruction is stored, where the at least one instruction is loaded by the processor and executed by the processor, where the specific execution process may refer to the specific description of the embodiment shown in fig. 1-2, and details are not repeated herein.

Referring to fig. 7, a block diagram of an electronic device according to an exemplary embodiment of the present application is shown. An electronic device in the present application may include one or more of the following components: processor 110, memory 120, input device 130, output device 140, and bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected by a bus 150.

Processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and lines to connect various portions of the overall electronic device, perform various functions of the electronic device 100, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in at least one hardware form of digital signal processing (digital signal processing, DSP), field-programmable gate array (field-programmable gate array, FPGA), programmable logic array (programmable logic Array, PLA). The processor 110 may integrate one or a combination of several of a central processing unit (central processing unit, CPU), an image processor (graphics processing unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented solely by a single communication chip.

The memory 120 may include a random access memory (random Access Memory, RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system, including an Android system-based deep development system, an IOS system developed by apple corporation, including an IOS system-based deep development system, or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the electronic device in use, such as phonebooks, audiovisual data, chat log data, and the like.

Referring to FIG. 8, the memory 120 may be divided into an operating system space in which the operating system is running and a user space in which native and third party applications are running. In order to ensure that different third party application programs can achieve better operation effects, the operating system allocates corresponding system resources for the different third party application programs. However, the requirements of different application scenarios in the same third party application program on system resources are different, for example, under the local resource loading scenario, the third party application program has higher requirement on the disk reading speed; in the animation rendering scene, the third party application program has higher requirements on the GPU performance. The operating system and the third party application program are mutually independent, and the operating system often cannot timely sense the current application scene of the third party application program, so that the operating system cannot perform targeted system resource adaptation according to the specific application scene of the third party application program.

In order to enable the operating system to distinguish specific application scenes of the third-party application program, data communication between the third-party application program and the operating system needs to be communicated, so that the operating system can acquire current scene information of the third-party application program at any time, and targeted system resource adaptation is performed based on the current scene.

Taking an operating system as an Android system as an example, as shown in fig. 9, a program and data stored in the memory 120 may be stored in the memory 120 with a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360 and an application layer 380, where the Linux kernel layer 320, the system runtime library layer 340 and the application framework layer 360 belong to an operating system space, and the application layer 380 belongs to a user space. The Linux kernel layer 320 provides the underlying drivers for various hardware of the electronic device, such as display drivers, audio drivers, camera drivers, bluetooth drivers, wi-Fi drivers, power management, and the like. The system runtime layer 340 provides the main feature support for the Android system through some C/c++ libraries. For example, the SQLite library provides support for databases, the OpenGL/ES library provides support for 3D graphics, the Webkit library provides support for browser kernels, and the like. Also provided in the system runtime library layer 340 is a An Zhuoyun runtime library (Android run) which provides mainly some core libraries that can allow developers to write Android applications using the Java language. The application framework layer 360 provides various APIs that may be used in building applications, which developers can also build their own applications by using, for example, campaign management, window management, view management, notification management, content provider, package management, call management, resource management, location management. At least one application program is running in the application layer 380, and these application programs may be native application programs of the operating system, such as a contact program, a short message program, a clock program, a camera application, etc.; and can also be a third party application program developed by a third party developer, such as a game application program, an instant messaging program, a photo beautification program, a screen sharing program and the like.

Taking an operating system as an IOS system as an example, the programs and data stored in the memory 120 are shown in fig. 10, the IOS system includes: core operating system layer 420 (Core OS layer), core service layer 440 (Core Services layer), media layer 460 (Media layer), and touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes an operating system kernel, drivers, and underlying program frameworks that provide more hardware-like functionality for use by the program frameworks at the core services layer 440. The core services layer 440 provides system services or program frameworks required by the application, such as a Foundation (Foundation) framework, an account framework, an advertisement framework, a data storage framework, a network connection framework, a geographic location framework, a sports framework, and the like. The media layer 460 provides an interface for applications related to audiovisual aspects, such as a graphics-image related interface, an audio technology related interface, a video technology related interface, an audio video transmission technology wireless play (AirPlay) interface, and so forth. The touchable layer 480 provides various commonly used interface-related frameworks for application development, with the touchable layer 480 being responsible for user touch interactions on the electronic device. Such as a local notification service, a remote push service, an advertisement framework, a game tool framework, a message User Interface (UI) framework, a User Interface UIKit framework, a map framework, and so forth.

Among the frameworks illustrated in fig. 10, frameworks related to most applications include, but are not limited to: the infrastructure in core services layer 440 and the UIKit framework in touchable layer 480. The infrastructure provides many basic object classes and data types, providing the most basic system services for all applications, independent of the UI. While the class provided by the UIKit framework is a basic UI class library for creating touch-based user interfaces, iOS applications can provide UIs based on the UIKit framework, so it provides the infrastructure for applications to build user interfaces, draw, process and user interaction events, respond to gestures, and so on.

The manner and principle of implementing data communication between the third party application program and the operating system in the IOS system may refer to the Android system, which is not described herein.

The input device 130 is configured to receive input instructions or data, and the input device 130 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 are a touch display screen for receiving a touch operation thereon or thereabout by a user using a finger, a touch pen, or any other suitable object, and displaying a user interface of each application program. Touch display screens are typically provided on the front panel of an electronic device. The touch display screen may be designed as a full screen, a curved screen, or a contoured screen. The touch display screen can also be designed to be a combination of a full screen and a curved screen, and a combination of a special-shaped screen and a curved screen, which is not limited in the embodiment of the application.

In addition, those skilled in the art will appreciate that the configuration of the electronic device shown in the above-described figures does not constitute a limitation of the electronic device, and the electronic device may include more or less components than illustrated, or may combine certain components, or may have a different arrangement of components. For example, the electronic device further includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (wireless fidelity, wiFi) module, a power supply, and a bluetooth module, which are not described herein.

In the embodiment of the present application, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is an operating system of the electronic device. The operating system may be an android system, an IOS system, or other operating systems, which embodiments of the present application do not limit.

The electronic device of the embodiment of the application may further be provided with a display device, where the display device may be various devices capable of implementing a display function, for example: cathode ray tube displays (cathode ray tubedisplay, CR), light-emitting diode displays (light-emitting diode display, LED), electronic ink screens, liquid crystal displays (liquid crystal display, LCD), plasma display panels (plasma display panel, PDP), and the like. A user may utilize a display device on electronic device 101 to view displayed text, images, video, etc. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality ) device, an automobile, a data storage device, an audio playing device, a video playing device, a notebook, a desktop computing device, a wearable device such as an electronic watch, electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, an electronic article of clothing, etc.

In the electronic device shown in fig. 7, where the electronic device may be a terminal, the processor 110 may be configured to invoke the screen sharing application stored in the memory 120, and specifically perform the following operations:

In one embodiment, before executing the detection of the screen sharing instruction for the display interface, the processor 110 further performs the following operations:

collecting at least one application interface corresponding to an application, and detecting privacy content of all display layers corresponding to the application interfaces of the applications;

In one embodiment, after performing the determining that at least one display layer of the total display layers is present in the secure layer library, the processor 110 further performs the following operations:

and carrying out shielding treatment on the interface frame image corresponding to the input method application.

In one embodiment, the processor 110, when executing the occlusion processing for the interface frame image, specifically executes the following operations:

determining all display pictures of the interface frame images as shielding areas, and carrying out shielding treatment on the shielding areas; or alternatively, the first and second heat exchangers may be,

and determining an occlusion region corresponding to the privacy content in the interface frame image, and carrying out occlusion treatment on the occlusion region.

In one embodiment, when the determining the occlusion region corresponding to the private content in the interface frame image is performed, the processor 110 specifically performs the following operations:

and detecting risks of display elements of the interface frame image, determining risk display elements, and taking a display area corresponding to the risk display elements as a shielding area.

In one embodiment, before performing the occlusion processing on the occlusion region, the processor 110 further performs the following operations:

acquiring a shielding level corresponding to the shielding area, and determining a shielding processing mode corresponding to the shielding level;

And carrying out shielding treatment on the shielding region based on the shielding treatment mode.

In one embodiment, when the processor 110 performs the interface occlusion processing on the occlusion region, the following steps are specifically performed:

It is clear to a person skilled in the art that the technical solutions of the present application may be implemented by means of software or hardware. "Unit" and "module" in this specification refer to software or hardware capable of performing a specific function, either alone or in combination with other components, such as Field programmable gate arrays (Field-ProgrammaBLE Gate Array, FPGAs), integrated circuits (Integrated Circuit, ICs), etc.

It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, such as the division of the units, merely a logical function division, and there may be additional manners of dividing the actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be performed by hardware associated with a program that is stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.

The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.

Claims

1. A screen sharing method, the method comprising:

determining that at least one display layer of the all display layers exists in a secure layer library;

when the input method application is detected to be called on the display interface, copying an interface frame image corresponding to the input method application, and carrying out shielding treatment on the interface frame image corresponding to the input method application;

copying an interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state;

carrying out shielding treatment on the interface frame images and sharing the interface frame images;

wherein, the shielding processing for the interface frame image includes: and taking an area corresponding to the privacy content in the interface frame image as a shielding area, and shielding the shielding area, wherein the area corresponding to the privacy content is a display area corresponding to a risk display element, and the risk display element is determined after risk detection is carried out on the display element of the interface frame image.

2. The method of claim 1, wherein prior to detecting the screen sharing instruction for the display interface, further comprising:

3. The method according to claim 1, wherein the masking the interface frame image comprises:

4. A method according to claim 3, wherein determining an occlusion region corresponding to private content in the interface frame image comprises:

5. The method of claim 3, wherein prior to the occlusion processing of the occlusion region, further comprising:

6. A method according to claim 3, wherein said interface occlusion processing of said occlusion region comprises:

7. A screen sharing apparatus, the apparatus comprising:

the frame image copying module is used for determining that at least one display layer in all display layers exists in a safe layer library, copying an interface frame image corresponding to an input method application when the input method application is detected to be called on the display interface, carrying out shielding treatment on the interface frame image corresponding to the input method application, and copying the interface frame image corresponding to the display interface, wherein the interface frame image is a frame image to be shared in a screen sharing state;

The processing sharing module is used for carrying out shielding processing on the interface frame images and sharing the interface frame images; the processing sharing module is specifically configured to take an area corresponding to the privacy content in the interface frame image as a blocking area, perform blocking processing on the blocking area, wherein the area corresponding to the privacy content is a display area corresponding to a risk display element, and the risk display element is determined after risk detection is performed on the display element of the interface frame image.

8. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps of any one of claims 1 to 6.

9. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-6.