WO2022247664A1

WO2022247664A1 - Graphical interface display method, electronic device, medium, and program product

Info

Publication number: WO2022247664A1
Application number: PCT/CN2022/092872
Authority: WO
Inventors: 杜奕全; 邓勐; 杨晓; 李凯
Original assignee: 华为技术有限公司
Priority date: 2021-05-28
Filing date: 2022-05-13
Publication date: 2022-12-01
Also published as: CN115408708A

Abstract

The present disclosure relates to a graphical interface display method, an electronic device, a computer-readable storage medium, and a computer program product. According to the described graphical interface display method, a user interface (UI) is displayed on a screen of an electronic device; a protection operation associated with the UI is detected; and in response to detecting the protection operation, enabling content that is associated with at least part of the UI to be unable to be normally displayed on the screen. According to the foregoing method, user information can be protected, allowing user information to not be viewed by an unauthorized user, thereby enhancing the security of an electronic device and significantly improving user experience.

Description

Graphical interface display method, electronic device, medium and program product

technical field

The present disclosure generally relates to the field of information technology, and more particularly relates to a graphical interface display method, electronic equipment, computer-readable storage medium, and computer program products.

Background technique

With the popularization of electronic devices, information security and privacy protection have attracted more and more attention. For example, typical information protection methods include app lock, private space, artificial intelligence (AI) to hide notification content, etc. However, traditional user information protection methods cannot effectively prevent information from being peeped, and how to further improve information security is still the key to improving user experience.

Contents of the invention

According to some embodiments of the present disclosure, a graphical interface display method, an electronic device, a medium, and a program product are provided, which can protect user information so that the user information cannot be viewed by unauthorized users, thereby enhancing the security of the electronic device, And significantly improved user experience.

In a first aspect of the present disclosure, a graphical interface display method is provided. According to the graphical interface display method of the first aspect, a user interface UI is displayed on the screen of the electronic device; a protection operation associated with the UI is detected; and in response to detecting the protection operation, making at least a part of the UI The associated content does not display properly on said screen. In this way, user information can be protected so that the user information cannot be viewed by unauthorized users, thereby enhancing the security of the electronic device and significantly improving user experience.

In some implementations, the content associated with at least a portion of the UI includes at least one of: content displayed by at least a portion of the UI, and content accessed via at least a portion of the UI. In this way, content currently displayed by the UI and/or content further accessed via the content currently displayed by the UI can be protected, thereby enhancing security of the electronic device and significantly improving user experience.

In some implementations, the protection operation includes: an element protection operation for protecting UI elements in the UI, and the element protection operation prevents the content associated with the UI element from being displayed normally, or protects the UI element The UI is protected by an interface protection operation, and the interface protection operation prevents at least a part of the content of the UI from being displayed normally. In this way, user information can be protected at the granularity of the UI or at the granularity of UI elements in the UI, thereby enhancing the security of the electronic device and significantly improving user experience.

In some implementations, the protection operation is the element protection operation, and preventing the content from being displayed normally includes: determining a target UI element corresponding to the element protection operation based on the element protection operation; The content associated with the target UI element cannot be displayed normally. In this way, the target UI element can be easily determined and protected, thereby enhancing the flexibility of information protection and improving user experience.

In some implementations, the element protection operation includes a control display operation and a selection operation, and the determining the target UI element corresponding to the element protection operation based on the element protection operation includes: based on the control display operation, in Displaying on the screen an element protection control corresponding to at least one UI element included in the UI; and based on the selection operation, determining the UI element corresponding to the element protection control on which the selection operation acts as the target UI elements. In this manner, the target UI element can be intuitively selected through the element protection control, and the target UI element can be protected, thereby enhancing the flexibility of information protection, thereby improving user experience.

In some implementations, preventing the content associated with the target UI element from being displayed normally includes: determining a mode in which the target UI element is to be protected, and the mode indicates at least one of the following: making the content blurring, and preventing access to the content via the target UI element; and disabling normal display of the content associated with the target UI element based on the pattern. In this way, different protection modes can be applied to different UI elements, thereby enhancing the flexibility of information protection, thereby improving user experience.

In some implementation manners, preventing the content associated with the target UI element from being displayed normally includes: blurring the content. For example, obscuring the content may include obscuring content displayed by the target UI element itself and/or obscuring content accessed via the target UI element. In this manner, unauthorized users cannot clearly view the content that the user desires to protect.

In some implementations, preventing the content associated with the target UI element from being displayed normally includes: preventing access to the content via the target UI element. For example, preventing access to content via the target UI element may include making the target UI element inoperable by a user, and/or making the target UI element no longer responsive to user manipulation. In this manner, unauthorized users are rendered inaccessible to content that the user desires to protect.

In some implementations, the protection operation is the element protection operation, and the method further includes: detecting an unprotect operation on the target UI element; and in response to detecting the unprotect operation, interacting with The content associated with the target UI element is normally displayed. In this way, the protection of the content can be released, enabling the user to view desired content.

In some implementations, enabling the normal display of the content associated with the target UI element includes: verifying the identity of the user who initiates the unprotecting operation; The described content associated with the element is displayed normally. In this manner, an authorized user whose authentication is successful is enabled to view desired content.

In some implementation manners, the protection operation is the interface protection operation, and making the all contents of the UI unable to be displayed normally includes: blurring the all contents of the UI. In this manner, unauthorized users cannot clearly view the content that the user desires to protect.

In some implementations, the protection operation is the interface protection operation, and the method further includes: detecting an unprotect operation acting on the UI; in response to detecting the unprotect operation, determining the target area indicated by the location where the unprotect operation occurred; and causing the target area to be displayed normally. In this way, the protection of the content can be released, enabling the user to view desired content.

In some implementations, the unprotecting operation is a click operation, the UI includes at least one window, and determining the target area includes: determining the target window where the click operation occurs in the at least one window as the target area. In this way, the protection of the clicked window can be released, so that the user can easily view his desired content by clicking.

In some implementations, the unprotecting operation is an area drawing operation, and determining the target area includes: determining a part of the UI surrounded by the area drawing operation as the target area. For example, a user can draw a target area by a predetermined shape. Alternatively, the user can freely draw the target area arbitrarily. In this way, it is possible to release the protection of the part in the circle, so that the user can easily view his desired content through the area drawing operation.

In some implementation manners, the unprotecting operation is a sliding wiping operation, and determining the target area includes: determining a part of the UI where the sliding wiping operation occurs as the target area. In this way, the protection of the wiped portion can be released, so that the user can easily view his desired content through the wiping operation.

In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor and a memory storing instructions, wherein the instructions cause the electronic device to execute any method according to the first aspect and its implementation manner when executed by the processor.

In a third aspect of the present disclosure, a computer readable storage medium is provided. The computer-readable storage medium stores instructions, wherein the instructions cause the electronic device to execute any method according to the first aspect and its implementations when executed by the processor.

In a fourth aspect of the present disclosure, a computer program product is provided. The computer program product comprises instructions, wherein the instructions, when executed by the processor, cause the electronic device to perform any method according to the first aspect and implementations thereof.

Description of drawings

The features, advantages and other aspects of various implementations of the present disclosure will become more apparent with reference to the following detailed description when taken in conjunction with the accompanying drawings. Several implementations of the present disclosure are shown here by way of illustration and not limitation, in the accompanying drawings:

1A to 1B show schematic diagrams of a hardware structure and a software structure of an electronic device that can implement an embodiment of the present disclosure.

FIG. 2 shows a block diagram of another electronic device in which embodiments of the present disclosure may be implemented.

3A-3I illustrate schematic diagrams of a first example of a UI associated with an element protection operation, according to some embodiments of the present disclosure.

4A-4L illustrate schematic diagrams of a second example of a UI associated with an element protection operation, according to some embodiments of the present disclosure.

5A-5P illustrate diagrams illustrating a third example of a UI associated with an element protection operation according to some embodiments of the present disclosure.

6A-6D show diagrams of a fourth example of UI associated with interface protection operations according to some embodiments of the present disclosure.

7A-7E show diagrams of a fifth example of UI associated with an interface protection operation according to some embodiments of the present disclosure.

8A-8E are diagrams showing a sixth example of UI associated with an interface protection operation according to some embodiments of the present disclosure.

9A-9E show diagrams of a seventh example of UI associated with an interface protection operation according to some embodiments of the present disclosure.

10A-10E are diagrams showing an eighth example of UI associated with an interface protection operation according to some embodiments of the present disclosure.

Fig. 11 shows a flowchart of a graphical interface display method according to an embodiment of the present disclosure.

Fig. 12 shows a schematic diagram of a system architecture according to an embodiment of the present disclosure.

Fig. 13 shows a schematic diagram of a partial sharpening architecture according to an embodiment of the present disclosure.

Figure 14 shows a schematic diagram of physical elements according to an embodiment of the disclosure.

Detailed ways

Some example implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some example implementations of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the example implementations set forth herein. Rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As used herein, the term "comprise" and its variants mean open inclusion, ie "including but not limited to". The term "or" means "and/or" unless otherwise stated. The term "based on" means "based at least in part on". The terms "embodiments" and "some embodiments" mean "at least some embodiments." The terms "first", "second" and other descriptions are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit that "first" and "second" are different types.

The term "UI" used in this article refers to the interface where the user interacts with the application program or the operating system and exchanges information, which realizes the conversion between the internal form of information and the form acceptable to the user. For example, the UI of an application program is source code written in a specific computer language such as java and extensible markup language (XML). , such as pictures, text, buttons and other UI elements.

In some embodiments, the attributes and contents of the UI elements in the UI are defined by tags or nodes. For example, XML specifies the UI elements included in the UI through nodes such as <Textview>, <ImgView>, and <VideoView>. A node corresponds to a UI element or attribute in the UI. After the node is parsed and rendered, it is presented as the content visible to the user. In addition, the UI of many applications, such as hybrid applications, usually includes web pages. A web page can be understood as a special UI element embedded in the application UI. A web page is a source code written in a specific computer language, such as hypertext markup language (hyper text markup language, HTML), cascading style sheets (cascading style sheets) , CSS), java script (JavaScript, JS), etc., the source code of the webpage can be loaded and displayed as user-identifiable content by a browser or a webpage display component similar in function to the browser. The specific content contained in the webpage is also defined by the tags or nodes in the source code of the webpage. For example, HTML defines the elements and attributes of the webpage through <p>, <img>, <video>, and <canvas>.

The term "UI element" used in this article includes but is not limited to: window (window), scroll bar (scrollbar), table view (tableview), button (button), menu bar (menu bar), text box (text box) , navigation bar, toolbar (toolbar), image (image), static text (tatictext), component (Widget) and other visual UI elements.

In some processes described in the embodiments of the present disclosure, a plurality of operations or steps appearing in a specific order are included, but it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present disclosure Or execute in parallel, the sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these procedures may include more or fewer operations, and these operations or steps may be performed sequentially or in parallel, and these operations or steps may be combined.

As mentioned above, with the popularity of electronic devices, information security and privacy protection have drawn more and more attention. Traditionally, there are various information protection methods to protect information security and privacy, such as application lock, private space, AI hidden notification content, etc. However, these information protection methods have their own drawbacks. For example, the application lock can only prevent unauthorized users from entering the application to view the content of the application, the protection scenario is single, and it cannot prevent the leakage of application content in real time. In addition, the application lock can only protect the content of the entire application, but cannot individually protect the content of a certain control or page in the application. The private space can only prevent unauthorized users from entering the private space, the protection scene is single, and it cannot prevent the leakage of the private space in real time. In addition, AI hiding notification content can only protect message notification scenarios, but cannot protect other scenarios.

It can be seen that the traditional information protection methods all have the defect of a single protection scenario, which cannot meet the needs of users. For example, when a user uses an electronic device in the subway, the content displayed on the electronic device can easily be seen by others, resulting in information leakage. This scenario cannot be solved by traditional information protection methods such as App Lock and Privacy Space.

To this end, the embodiments of the present disclosure propose a new solution for graphical interface display. Embodiments of the present disclosure relate to applying a protection operation to content associated with at least a part of the UI, so that the content cannot be displayed normally on the screen. Specifically, protection operations include element protection operations and interface protection operations. In the element protection operation, the method of protecting controls and layouts is adopted to realize the protection of partial content. For example, when a user is using a certain application, he may not want others to see a certain control or layout, in this case, the user can use element protection operations to protect these contents. After these contents are protected, specific controls or layouts within the app need to be verified by password, fingerprint, face, etc. In addition, in the interface protection operation, the electronic device only displays a part of the UI, while other parts cannot be displayed normally, such as blurring, so as to realize information protection. By displaying the partial content, the desired protected content of the electronic device can be protected from being leaked in real time, thereby enhancing the security of the electronic device and significantly improving user experience.

Some example embodiments of the present disclosure will be described below with reference to FIGS. 1A to 13 .

FIG. 1A shows a schematic diagram of a hardware structure of an electronic device 100 that can implement an embodiment of the present disclosure. As shown in Figure 1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, and a battery 142 , antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193 , a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It should be understood that the structures shown in the embodiments of the present disclosure do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present disclosure, the electronic device 100 may include more or fewer components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship among the modules shown in the embodiments of the present disclosure is only for schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present disclosure, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device 100 through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input of the battery 142 and/or the charging management module 140, and supplies power for the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 100 may be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G/6G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area network (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (Bluetooth, BT), global navigation satellite Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (long term evolution, LTE), 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. Among them, GNSS can include global positioning system (global positioning system, GPS), global navigation satellite system (global navigation satellite system, GLONASS), Beidou satellite navigation system (beidou navigation satellite system, BDS), quasi-zenith satellite system (quasi- zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 , and the application processor. The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example, moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in an external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present disclosure take a mobile operating system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .

FIG. 1B is a schematic diagram of the software structure of the electronic device 100 according to the embodiment of the present disclosure. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the operating system can be divided into four layers, which are application program layer, application program framework layer, operating system runtime (runtime) and system library, and kernel layer from top to bottom.

The application layer can consist of a series of application packages. As shown in FIG. 1B , the application package may include application programs such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in FIG. 1B, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like. A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. Content providers are used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, and the like. The view system includes visual controls, such as controls that display text, controls that display images, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a notification icon of a short message may include a view for displaying text and a view for displaying images. The phone manager is used to provide communication functions of the electronic device 100 . For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources to the application, such as localized strings, icons, images, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.

Continuing to refer to FIG. 1B , the operating system runtime includes a core library and a virtual machine. When the operating system is running, it is responsible for the scheduling and management of the operating system. The core library consists of two parts, one part is the function function that the Java language needs to call, and the other part is the core library of the operating system. The application layer and the application framework layer run in virtual machines. The virtual machine executes the Java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. A system library can include multiple function modules. For example, surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc. 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

FIG. 2 shows a block diagram of another electronic device 200 that may implement embodiments of the present disclosure. As shown in FIG. 2, electronic device 200 may be in the form of a general-purpose computing device. Components of electronic device 200 may include, but are not limited to, one or more processors or processing units 210, memory 220, storage device 230, one or more communication units 240, one or more input devices 250, and one or more output devices 260. The processing unit 210 may be an actual or virtual processor and is capable of performing various processes according to programs stored in the memory 220 . In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capability of the electronic device 200 .

Electronic device 200 typically includes a plurality of computer storage media. Such media can be any available media that is accessible to electronic device 200, including but not limited to, volatile and nonvolatile media, removable and non-removable media. Memory 220 can be volatile memory (eg, registers, cache, random access memory (RAM), nonvolatile memory (eg, read only memory (ROM), electrically erasable programmable read only memory (EEPROM) , flash memory) or some combination of them. Storage device 230 may be removable or non-removable media, and may include machine-readable media, such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (e.g., training data for training) ) and can be accessed within the electronic device 200.

The electronic device 200 may further include additional detachable/non-detachable, volatile/nonvolatile storage media. Although not shown in FIG. 2, a disk drive for reading from or writing to a removable, nonvolatile disk (such as a "floppy disk") and a disk drive for reading from a removable, nonvolatile disk may be provided. CD-ROM drive for reading or writing. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 220 may include a computer program product 225 having one or more program modules configured to execute the object editing method or process of the embodiments of the present disclosure.

The communication unit 240 enables communication with other computing devices through communication media. Additionally, the functions of the components of the electronic device 200 may be implemented in a single computing cluster or as a plurality of computing machines capable of communicating via communication links. Accordingly, electronic device 200 may operate in a networked environment using logical connections to one or more other servers, a network personal computer (PC), or another network node.

The input device 250 may be one or more input devices, such as a mouse, keyboard, trackball, and the like. Output device 260 may be one or more output devices, such as a display, speakers, printer, or the like. In an embodiment of the present disclosure, the output device 260 may include a touch screen having a touch sensor, which may receive a user's touch input. The electronic device 200 can also communicate with one or more external devices (not shown) through the communication unit 240 as required, such as storage devices, display devices, etc., and one or more devices that enable the user to interact with the electronic device 200 communicate with, or communicate with, any device (eg, network card, modem, etc.) that enables electronic device 200 to communicate with one or more other computing devices. Such communication may be performed via an input/output (I/O) interface (not shown).

It should be understood that the above electronic device 100 shown in FIG. 1 and the electronic device 200 shown in FIG. no limitations on the function and scope of the described embodiments.

Electronic devices are described above. Further, as described above, the embodiments of the present disclosure relate to applying a protection operation to content associated with at least a part of the UI, so that the content cannot be displayed normally on the screen. In some embodiments, the content associated with at least a portion of the UI is content displayed by at least a portion of the UI (e.g., a setting in a list of settings options), and/or content accessed via at least a portion of the UI (For example, the setting interface displayed after clicking the setting).

Specifically, the protection operation includes an element protection operation and an interface protection operation. In the element protection operation, the method of protecting controls and layouts is adopted to realize the protection of partial content. In addition, in the interface protection operation, the electronic device only displays a part of the UI, while other parts cannot be displayed normally, such as blurring, so as to realize information protection. For this reason, an example UI associated with an element protection operation will be described below with reference to FIGS. 3A-5P , and an example UI associated with an interface protection operation will be described with reference to FIGS. 6A-10E .

In the element protection operation, the electronic device may detect an element protection operation associated with the UI. Thus, the electronic device can determine the target UI element corresponding to the element protection operation based on the element protection operation, and prevent the content associated with the target UI element from being displayed normally. For example, the electronic device can monitor whether the user triggers an element protection operation, such as any appropriate action such as three-finger long press, three-finger slide, specific voice command, and the like. If an element protection operation is detected, the electronic device may disable content associated with at least a portion of the UI from being properly displayed on the screen.

In some embodiments, element protection operations may include control display operations and selection operations. In this case, the electronic device may display an element protection control corresponding to at least one UI element included in the UI on the screen based on the control display operation. In addition, the electronic device may determine, based on the selection operation, the UI element corresponding to the element protection control on which the selection operation acts as the target UI element. For example, the electronic device may monitor whether the user triggers a control display operation, such as any appropriate action such as three-finger long press, three-finger slide, specific voice command, and the like. If a control display operation is detected, the electronic device may display an element protection control corresponding to at least one UI element included in the UI on the screen. The electronic device may detect a selection operation acting on the element protection control, such as any appropriate action such as clicking, double-clicking, long pressing, or a specific voice command, and determine the UI element corresponding to the element protection control on which the selection operation acts as the target UI element, Thereby, the selection of the target UI element is realized. If a selection operation is detected, the electronic device may disable normal display of content associated with the target UI element.

In some embodiments, if a selection operation is detected, the electronic device may determine a mode in which to protect the target UI element. Modes may indicate to obscure content, or prevent content from being accessed via targeted UI elements. For example, in order to prevent content from being accessed via the target UI element, the user may not be able to operate the target UI element, such as graying out or hiding the target UI element, so that the user cannot click on the target UI element. Alternatively, it is also possible to make the target UI element no longer affect user operations, such as no longer displaying the content that will be displayed after clicking the target UI element, or no longer jumping to the target UI element that will be jumped to after clicking the target UI element. content. Thus, the electronic device may disable normal display of content associated with the target UI element based on the determined pattern. However, alternatively, the mode for protecting the target UI element may also be predetermined, so the electronic device does not need to further determine the mode for protecting the target UI element.

For example, if the user desires to protect a certain UI element, he can make the UI element enter the protected state by clicking the element protection control corresponding to the UI element, so that the content of the UI element, or via the UI element Accessed content cannot be displayed properly. For example, the electronic device may obscure the content of the UI element, obscure content accessed via the UI element, or render inaccessible content via the UI element. Thereby, information protection can be realized.

Further, after making the target UI element enter the protected state, the electronic device may also detect an unprotection operation acting on the target UI element. If an unprotect operation is detected, the electronic device may cause the content associated with the target UI element to be displayed normally. In some embodiments, when the electronic device detects the unprotection operation, it can also verify the identity of the user who initiates the unprotection operation. If the authentication is successful, the electronic device can cause the content associated with the target UI element to be displayed normally. For example, if a user wants to view the content associated with a UI element that has entered a protected state, he can trigger an unprotected verification interface (e.g., password, fingerprint, identity, etc.) face, etc. any appropriate verification method). After the user verification is successful, the electronic device may cause the UI element to leave the protected state, so as to normally display the content associated with the UI element. For example, the electronic device may restore clarity to the content of the UI element, restore clarity to content accessed via the UI element, or enable access to content via the UI element. In this way, information release protection can be realized.

3A-3I show schematic diagrams of a first example of UIs 300A-300I associated with element protection operations, according to some embodiments of the present disclosure. Specifically, in the example of FIGS. 3A-3I , the scene of protecting the content of the control is shown.

As shown in FIG. 3A, UI 300A may include a plurality of UI elements, such as UI element 330 and UI elements 340-1 to 340-7 (hereinafter collectively referred to as "UI elements 340"). For example, UI 300A may be a message interface of an instant messaging application, in which case UI element 330 may be a control to open a list of additional operations or a control to add a new message, while UI element 340 may be a message from a different sender. As another example, UI 300A may be a setting interface of an electronic device, in which case UI element 300 may be a control to open an additional operation list or a control to add a new setting, and UI element 340 may be a different setting option.

It should be understood that UI 300A and UI elements 330 and 340 are examples only, and in fact the UI may be any suitable interface and the UI elements may be any suitable controls. As mentioned above, the term "UI element" used in this article includes but is not limited to: window (window), scroll bar (scrollbar), table view (tableview), button (button), menu bar (menu bar), text box (text box), navigation bar, toolbar (toolbar), image (image), static text (tatictext), component (Widget) and other visual UI elements.

As shown in FIG. 3B, the electronic device may detect an element protection operation associated with the UI. As described above, element protection operations may include control display operations and selection operations. For example, the electronic device may monitor whether the user triggers a control display operation, such as any appropriate action such as three-finger long press, three-finger slide, specific voice command, and the like. If a control display operation is detected, as shown in FIG. 3C , the electronic device may display on the screen an element protection control corresponding to at least one UI element included in the UI, for example, corresponding to UI elements 340-1 to 340-7. Element protection controls 310 - 1 to 310 - 7 (hereinafter collectively referred to as “element protection controls 310 ”), and an element protection control 320 corresponding to UI element 330 . It should be understood that the shape, size, location, etc. of the element protection controls 310 and 320 are examples only, and in fact, the element protection controls may have any suitable shape, size, and location. Further, although element protection controls corresponding to UI elements 330 and 340 are shown in FIG. 3C , the number of element protection controls is merely an example. In fact, the electronic device may display more or fewer element protection controls, for example, the element protection control 320 corresponding to the UI element 330 may not be displayed, or more elements corresponding to the status bar of the electronic device (that is, the UI element at the top) may be displayed. The element protection control corresponding to the status bar showing network, power, time and other information).

As shown in FIG. 3D, the electronic device may detect a selection operation acting on an element protection control corresponding to a target UI element in at least one UI element, such as any appropriate action such as clicking, double-clicking, long-pressing, or a specific voice command. If a selection operation is detected, the electronic device may disable the normal display of the content of the targeted UI element. For example, as shown in FIG. 3E, the user desires to protect UI element 340-3, and clicks on element protection control 310-3 corresponding to UI element 340-3. Thus, as shown in FIG. 3F , the UI element 340-3 enters a protected state, so that the content of the UI element 340-3 cannot be displayed normally, for example, the content of the UI element 340-3 is blurred. Thereby, information protection can be realized. It should be understood that blurring is only an example of the failure to display normally. In fact, the content of the UI element can be prevented from being displayed normally by any appropriate means, such as covering with another image with appropriate transparency, disrupting the display order of the content of the UI element Wait.

Further, after making the target UI element enter the protected state, the electronic device may also detect an unprotection operation acting on the target UI element. If an unprotection operation is detected, the electronic device may cause the content of the target UI element to be displayed normally. In some embodiments, when the electronic device detects the unprotection operation, it can also verify the identity of the user who initiates the unprotection operation. If the authentication is successful, the electronic device can cause the content of the target UI element to be displayed normally. For example, as shown in Figure 3G, the user wants to view the content of the UI element 340-3 that has entered the protected state, and triggers the unprotection as shown in Figure 3H by long pressing, double-clicking on the UI element 340-3, etc. The verification interface (for example, verification interface such as password, fingerprint, face, etc.). After the user authentication is successful, the UI element 340-3 can be left in the protected state, so as to display the content of the UI element normally, as shown in FIG. 3I .

4A-4L show schematic diagrams of a second example of UIs 400A-400L associated with element protection operations, according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 4A-4L , a scenario of protecting content accessed via controls is shown.

In order to understand this example scenario more clearly, an example UI without the element UI entering the protected state will first be described with reference to FIGS. 4A-4C . As shown in FIG. 4A, UI 400A may include a plurality of UI elements, such as UI element 430 and UI elements 440-1 to 440-7 (hereinafter collectively referred to as "UI elements 440"). For example, UI 400A may be a message interface of an instant messaging application, in which case UI element 430 may be a control to open a list of additional operations or a control to add a new message, while UI element 440 may be a message from a different sender. As another example, UI 400A may be a setting interface of an electronic device, in which case UI element 400 may be a control for opening an additional operation list or adding a new setting, and UI element 440 may be a different setting option.

It should be understood that UI 400A and UI elements 430 and 440 are examples only, and in fact the UI may be any suitable interface and the UI elements may be any suitable controls. As mentioned above, the term "UI element" used in this article includes but is not limited to: window (window), scroll bar (scrollbar), table view (tableview), button (button), menu bar (menu bar), text box (text box), navigation bar, toolbar (toolbar), image (image), static text (tatictext), component (Widget) and other visual UI elements.

As shown in FIG. 4B , the user wants to view content accessed via UI element 430 , for example, the user wants to open a list of additional operations and clicks on UI element 430 . In this case, as shown in FIG. 4C , the electronic device displays a list of additional operations accessible via UI element 430 , such as a list of operations 1-3.

Having described the example UI in case the element UI does not enter the protected state above, the example UI in case the element UI is protected will be described below with reference to FIGS. 4D-4L .

Similar to FIG. 4A, as shown in FIG. 4D, UI 400D may include a plurality of UI elements, such as UI element 430 and UI element 440. As shown in FIG. 4E, the electronic device may detect element protection operations associated with the UI. As described above, element protection operations may include control display operations and selection operations. For example, the electronic device may monitor whether the user triggers a control display operation, such as any appropriate action such as three-finger long press, three-finger slide, specific voice command, and the like. If a control display operation is detected, as shown in FIG. 4F , the electronic device may display on the screen an element protection control corresponding to at least one UI element included in the UI, for example, corresponding to UI elements 440-1 to 440-7. Element protection controls 410 - 1 to 410 - 7 (hereinafter collectively referred to as “element protection controls 410 ”), and an element protection control 420 corresponding to UI element 430 . It should be understood that the shape, size and location, etc. of element protection controls 410 and 420 are merely examples, and in fact, the element protection controls may have any suitable shape, size and location. Further, although element protection controls corresponding to UI elements 430 and 440 are shown in FIG. 4F , the number of element protection controls is merely an example. In fact, the electronic device may display more or fewer element protection controls, for example, may not display the element protection control 420 corresponding to the UI element 430, or may display more elements related to the status bar of the electronic device (that is, the topmost element of the UI). The element protection control corresponding to the status bar showing network, power, time and other information).

As shown in FIG. 4G, the electronic device may detect a selection operation acting on an element protection control corresponding to a target UI element in at least one UI element, such as any appropriate action such as clicking, double-clicking, long-pressing, or a specific voice command. If a selection operation is detected, the electronic device may disable normal display of content accessed via the target UI element. For example, as shown in FIG. 4H , the user desires to protect the UI element 430 and clicks on the element protection control 420 corresponding to the UI element 430 . Thus, as shown in FIG. 4I , the UI element 430 enters a protected state, so that the content accessed via the UI element 430 cannot be displayed normally, for example, the UI element 430 becomes a dotted line representation, and additional operations cannot be opened via the UI element 430 list. Thereby, information protection can be realized. It should be understood that the dotted line represents only an example of the UI element entering the protected state. In fact, any appropriate method may be used to indicate that the UI element enters the protected state, such as replacing the UI element with another image, graying the UI element, etc.

Further, after making the target UI element enter the protected state, the electronic device may also detect an unprotection operation acting on the target UI element. If an unprotect operation is detected, the electronic device may cause the content accessed via the target UI element to be displayed normally. In some embodiments, when the electronic device detects the unprotection operation, it can also verify the identity of the user who initiates the unprotection operation. If the authentication is successful, the electronic device can cause the content accessed via the target UI element to display normally. For example, as shown in FIG. 4J, the user wants to view the content accessed via the UI element 430 entering the protected state, and triggers the verification of unprotection as shown in FIG. 4K by long pressing, double-clicking on the UI element 430, etc. Interface (for example, password, fingerprint, face verification interface, etc.). After successful user authentication, the UI element 430 may be caused to leave the protected state, and the list of additional operations accessed via the UI element 430 may be displayed normally, as shown in FIG. 4L .

The protection of controls is described above with reference to FIGS. 3A-4L , and the protection of layouts will be described below with reference to FIGS. 5A-5P . 5A-5P illustrate diagrams of a third example of UIs 500A-500P associated with element protection operations, according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 5A-5P , a scenario of protecting a layout accessed via a control is shown.

In order to understand this example scenario more clearly, an example UI where the element UI does not enter the protected state will first be described with reference to FIGS. 5A-5C . As shown in FIG. 5A , UI 500A may include multiple UI elements, such as controls associated with various smart devices and controls associated with Home, Mall, Smart, and My layouts or pages. It should be understood that UI 500A and the UI elements therein are merely examples, and in fact, the UI may be any suitable interface, and the UI elements may be any suitable controls. As mentioned above, the term "UI element" used in this article includes but is not limited to: window (window), scroll bar (scrollbar), table view (tableview), button (button), menu bar (menu bar), text box (text box), navigation bar, toolbar (toolbar), image (image), static text (tatictext), component (Widget) and other visual UI elements.

As shown in FIG. 5B , the user wants to view the content of the mall page, and clicks on the mall control. In this case, as shown in FIG. 5C , the electronic device displays the content of the mall page.

Having described the example UI in the case that the element UI does not enter the protected state above, the example UI in the case that the element UI is protected will be described below with reference to FIGS. 5D-5P .

Similar to FIG. 5A , as shown in FIG. 5D , UI 500D may include multiple UI elements, such as controls associated with various smart devices and controls associated with Home, Mall, Smart and My layouts or pages. As shown in FIG. 5E, the electronic device may detect element protection operations associated with the UI. As described above, element protection operations may include control display operations and selection operations. For example, the electronic device may monitor whether the user triggers a control display operation, such as any appropriate action such as three-finger long press, three-finger slide, specific voice command, and the like. If the control display operation is detected, as shown in FIG. 5F, the electronic device may display on the screen an element protection control corresponding to at least one UI element included in the UI, for example, corresponding to the home control, the mall control, the smart control and the personal control respectively. The controls correspond to element protection controls 510-540. It should be understood that the shape, size, and location, etc. of the element protection controls 510-540 are examples only, and in fact, the element protection controls may have any suitable shape, size, and location. Further, although element protection controls respectively corresponding to the home control, mall control, smart control and my control are shown in FIG. 5F , the number of element protection controls is just an example. In fact, the electronic device may display more or fewer element protection controls, for example, may not display the element protection control 510 corresponding to the home control, or may display more controls related to various smart devices (for example, the TV in the living room) ) corresponds to the element protection control.

As shown in FIG. 5G , the electronic device may detect a selection operation acting on an element protection control corresponding to a target UI element in at least one UI element, such as any appropriate action such as click, double click, long press, specific voice command, and the like. If a selection operation is detected, the electronic device may disable normal display of content accessed via the target UI element. For example, as shown in FIG. 5H , the user desires to protect the content accessed via the mall control, and clicks on the element protection control 520 corresponding to the mall control. Thus, as shown in FIG. 51 , the content accessed via the mall control enters a protected state, and causes the mall control to become a dotted line representation. In this case, as shown in FIG. 5J , even if the user wants to view the content accessed through the mall control by clicking on the mall control, the content accessed through the mall control cannot be displayed normally, for example, making the mall page blurred (such as As shown in FIG. 5K ) or via the mall control, the mall page (as shown in FIG. 5L ) cannot be entered. Thereby, information protection can be realized.

It should be understood that the dotted line represents only an example of the UI element entering the protected state. In fact, any appropriate method may be used to indicate that the UI element enters the protected state, such as replacing the UI element with another image, graying the UI element, etc. In addition, blurring is just an example of failure to display normally. In fact, the content of UI elements can be prevented from displaying normally by any appropriate means, such as overlaying with another image with appropriate transparency, disrupting the display order of the content of UI elements Wait.

Further, after making the target UI element enter the protected state, the electronic device may also detect an unprotection operation acting on the target UI element. If an unprotect operation is detected, the electronic device may cause the content accessed via the target UI element to be displayed normally. In some embodiments, when the electronic device detects the unprotection operation, it can also verify the identity of the user who initiates the unprotection operation. If the authentication is successful, the electronic device can cause the content accessed via the target UI element to display normally. For example, a user wants to view content accessed via a mall control that has entered a protected state, and either long-presses, double-clicks on the mall control (as shown in Figure 5N), or content associated with the mall control (as shown in Figure 5M), etc. Appropriate actions are taken to trigger the unprotected verification interface (for example, password, fingerprint, face, etc. verification interface) as shown in FIG. 50 . After the user is authenticated successfully, the mall control can be made to leave the protected state, and the mall page accessed via the mall control can be displayed normally, as shown in FIG. 5P .

The element protection operation is described above with reference to FIGS. 3A-5P , and the interface protection operation will be described below with reference to FIGS. 6A-10E . In the interface protection operation, the electronic device may detect the interface protection operation associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as interface protection controls, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, when the user clicks the interface protection control, all or part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection.

Further, after making the content of the target UI element enter the protected state, the electronic device may also detect an unprotect operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally. For example, when the unprotection operation is a click operation, and the UI includes at least one window, the target window where the click operation occurs in the at least one window may be determined as the target area. As another example, in a case where the unprotecting operation is an area drawing operation, a part of the UI surrounded by the area drawing operation may be determined as a target area. In some embodiments, the user can draw the target area in a predetermined shape (eg, any suitable shape such as a circle, a rectangle, etc.). For example, the user can draw a target area by dragging, scaling a predetermined shape, etc. on the screen. In this case, the target area may have a regular shape. Alternatively, the user can draw the target area arbitrarily. In this case, the target area may have an irregular shape. As yet another example, in the case that the unprotecting operation is a sliding wiping operation, the part of the UI where the sliding wiping operation occurs may be determined as the target area.

6A-6D show diagrams of a fourth example of UIs 600A-600D associated with interface protection operations according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 6A-6D , a scene of a single floating window is shown.

As shown in FIG. 6A , the UI includes an interface protection control 610 , a floating window 620 and other background parts. The electronic device can detect interface protection operations associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as the interface protection control 610, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, as shown in FIG. 6B , the user wants to protect at least a part of the content of the UI, and clicks the interface protection control 610 . When the user clicks the interface protection control 610, as shown in FIG. 6C, at least a part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection. It should be understood that although all the content of the UI is blurred as shown in FIG. 6C , only a part of the content of the UI may be blurred, for example, only the floating window 620 is blurred. In some embodiments, protected content is electronic device or user settable.

Further, after making at least a part of content of the UI into a protected state, the electronic device may also detect an unprotection operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally. For example, when the unprotection operation is a click operation, and the UI includes at least one window, the target window where the click operation occurs in the at least one window may be determined as the target area. For example, as shown in FIG. 6D , when the user clicks on the floating window 620, the electronic device may determine the floating window 620 as the target area for restoring normal display, so that the floating window 620 becomes clear again while continuing to maintain the UI. Other parts are blurred.

In addition, in some embodiments, when the electronic device detects the unprotection operation, it can also verify the identity of the user who initiated the unprotection operation (for example, password, fingerprint, face verification, etc.). If the authentication is successful, the electronic device can cause the target area to be displayed normally.

7A-7E show diagrams of a fifth example of UIs 700A-700E associated with interface protection operations, according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 7A-7E , a scene with multiple floating windows is shown.

As shown in FIG. 7A, the UI includes an interface protection control 710, a floating window 720, a floating window 730, and other background parts. The electronic device can detect interface protection operations associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as interface protection controls 710, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, as shown in FIG. 7B , the user wants to protect at least a part of the content of the UI, and clicks the interface protection control 710 . When the user clicks the interface protection control 710, as shown in FIG. 7C, at least a part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection. It should be understood that although it is shown in FIG. 7C that all content of the UI is blurred, only a part of the content of the UI may be blurred, for example, only the floating

windows

720 and 730 are blurred. In some embodiments, protected content is electronic device or user settable.

Further, after making at least a part of content of the UI into a protected state, the electronic device may also detect an unprotection operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally. For example, when the unprotection operation is a click operation, and the UI includes at least one window, the target window where the click operation occurs in the at least one window may be determined as the target area. For example, as shown in FIG. 7D , when the user clicks on the floating window 720, the electronic device may determine the floating window 720 as the target area for restoring normal display, so that the floating window 720 becomes clear again while continuing to maintain the UI. Other parts are blurred. As another example, when the user clicks on the floating window 730, the electronic device may determine the floating window 730 as the target area for restoring normal display, so that the floating window 730 becomes clear again while continuing to keep other parts of the UI blurred .

8A-8E show diagrams of sixth examples of UIs 800A-800E associated with interface protection operations, according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 8A-8E , a split-screen scene is shown.

As shown in FIG. 8A, the UI includes an interface protection control 810, a split-screen window 820, a split-screen window 830, and other background parts. The electronic device can detect interface protection operations associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as the interface protection control 810, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, as shown in FIG. 8B , the user wants to protect at least a part of the content of the UI, and clicks the interface protection control 810 . When the user clicks the interface protection control 810, as shown in FIG. 8C, at least a part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection. It should be understood that although it is shown in FIG. 8C that all content of the UI is blurred, only a part of the content of the UI may be blurred, for example, only the split-

screen windows

820 and 830 are blurred. In some embodiments, protected content is electronic device or user settable.

Further, after making at least a part of content of the UI into a protected state, the electronic device may also detect an unprotection operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally. For example, when the unprotecting operation is a click operation, and the UI includes at least one window, the target window where the click operation occurs in the at least one window may be determined as the target area. For example, as shown in FIG. 8D, when the user clicks on the split-screen window 820, the electronic device can determine the split-screen window 820 as the target area for restoring normal display, so that the split-screen window 820 becomes clear again, and at the same time continues to Keep the rest of the UI blurred. As another example, when the user clicks on the split-screen window 830, the electronic device may determine the split-screen window 830 as the target area for restoring normal display, so that the split-screen window 830 becomes clear again while continuing to maintain the UI. Other parts are blurred.

9A-9E show diagrams of a seventh example of UIs 900A-900E associated with interface protection operations, according to some embodiments of the present disclosure. Specifically, in the example of FIGS. 9A-9E , parallel horizon scenarios are shown.

As shown in FIG. 9A , the UI includes an interface protection control 910 , a parallel horizon window 920 , a parallel horizon window 930 , and other background parts. The electronic device can detect interface protection operations associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as interface protection controls 910, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, as shown in FIG. 9B , the user wants to protect at least a part of the content of the UI, and clicks the interface protection control 910 . When the user clicks the interface protection control 910 , as shown in FIG. 9C , at least a part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection. It should be understood that although all content of the UI is shown to be blurred in FIG. 9C , only a part of the content of the UI may be blurred, for example, only the

parallel horizon windows

920 and 930 are blurred. In some embodiments, protected content is electronic device or user settable.

Further, after making at least a part of content of the UI into a protected state, the electronic device may also detect an unprotection operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally. For example, when the unprotection operation is a click operation, and the UI includes at least one window, the target window where the click operation occurs in the at least one window may be determined as the target area. For example, as shown in FIG. 9D, when the user clicks on the parallel horizon window 920, the electronic device may determine the parallel horizon window 920 as the target area for restoring normal display, so that the parallel horizon window 920 becomes clear again, and at the same time continue to Keep the rest of the UI blurred. As another example, when the user clicks on the parallel horizon window 930, the electronic device may determine the parallel horizon window 930 as the target area for restoring normal display, so that the parallel horizon window 930 becomes clear again while continuing to maintain the UI. Other parts are blurred.

10A-10E show diagrams of eighth examples of UIs 1000A-1000E associated with interface protection operations, according to some embodiments of the present disclosure. Specifically, in the examples of FIGS. 10A-10E , partial content display scenarios are shown.

As shown in FIG. 1OA, the UI includes an interface guard control 1010 and various other UI elements. The electronic device can detect interface protection operations associated with the UI. For example, the electronic device may monitor whether the user has triggered any appropriate actions such as the interface protection control 1010, specific voice commands, and the like. If an interface protection operation is detected, the electronic device may prevent at least a part of the content of the UI from being displayed normally. For example, as shown in FIG. 10B , the user wants to protect at least a part of the content of the UI, and clicks the interface protection control 1010 . When the user clicks the interface protection control 1010, as shown in FIG. 10C , at least a part of the content of the UI is blurred. In this case, for example, when a user is on public transportation such as a subway, other people cannot see the content of the UI, thereby achieving information protection. It should be understood that although all content of the UI is shown to be blurred in FIG. 10C , only a part of the content of the UI may be blurred, for example, only controls associated with contextual intelligence and controls associated with parking may be blurred. In some embodiments, protected content is electronic device or user settable.

Further, after making at least a part of content of the UI into a protected state, the electronic device may also detect an unprotection operation acting on the UI. If an unprotection operation is detected, the electronic device may display at least a part of the content of the UI normally. In some embodiments, when the electronic device detects the unprotection operation, it can also determine the target area indicated by the location where the unprotection operation occurs in the UI, and display the target area normally.

In some embodiments, if the unprotecting operation is an area drawing operation, the electronic device may determine the part of the UI surrounded by the area drawing operation as the target area. For example, as shown in FIG. 10D , the user may use a finger to circle the target area 1020 on the screen, thereby making the target area 1020 clear again while continuing to keep other parts of the UI blurred.

Alternatively, in the case that the unlocking operation is a sliding wiping operation, the electronic device may determine a part of the UI where the sliding wiping operation occurs as the target area. For example, as shown in FIG. 10E , the user can wipe out the target area 1030 with a finger on the screen, thereby making the target area 1030 clear again while continuing to keep other parts of the UI blurred.

It should be understood that the target area 1020 is only an example, in fact, the user can circle any suitable area with a regular or irregular shape on the screen with a finger, and can wipe out any suitable area with a regular or irregular shape.

In the above, an example scenario of information protection is described in detail. Hereinafter, a flowchart of a graphical interface display method 1100 according to an embodiment of the present disclosure will be further described with reference to FIG. 11 . It should be understood that the method 1100 may be executed by the electronic device 100 described above with reference to FIG. 1 or the electronic device 200 described with reference to FIG. 2 . Method 1100 is described herein with reference to UIs 300A-1000E of FIGS. 3A-10E . However, it should be understood that the UIs 300A-1000E are merely examples and the method 1100 may be applicable to any suitable interface, including but not limited to the UIs 300A-1000E.

At block 1110, a user interface UI is displayed on the screen of the electronic device. At block 1120, the electronic device detects a protection operation associated with the UI. At block 1130, in response to detecting the protection operation, the electronic device disables content associated with at least a portion of the UI from being normally displayed on the screen. In this way, user information can be protected so that the user information cannot be viewed by unauthorized users, thereby enhancing the security of the electronic device and significantly improving user experience.

In some implementations, the element protection operation includes a control display operation and a selection operation, and the determining the target UI element corresponding to the element protection operation based on the element protection operation includes: based on the control display operation, in Displaying on the screen an element protection control corresponding to at least one UI element included in the UI; and based on the selection operation, determining the UI element corresponding to the element protection control on which the selection operation acts as the target UI elements. In this way, the target UI element can be intuitively selected through the element protection control, and the target UI element can be protected, thereby enhancing the flexibility of information protection, thereby improving user experience.

In some implementation manners, preventing the content associated with the target UI element from being displayed normally includes: blurring the content. Obscuring the content may include obscuring content displayed by the target UI element itself and/or obscuring content accessed via the target UI element. In this manner, unauthorized users cannot clearly view the content that the user desires to protect.

In some implementations, preventing the content associated with the target UI element from being displayed normally includes: preventing access to the content via the target UI element. In this manner, unauthorized users are rendered inaccessible to content that the user desires to protect.

In some implementations, the unprotecting operation is an area drawing operation, and determining the target area includes: determining a part of the UI surrounded by the area drawing operation as the target area. In this way, it is possible to unprotect the part in the circle, so that the user can easily view his desired content through an area drawing operation.

The following introduces the system architecture, partial clarity architecture and physical components for realizing the graphical interface display proposed by the embodiments of the present disclosure.

FIG. 12 shows a schematic diagram of a system architecture 1200 according to an embodiment of the present disclosure. At the hardware layer 1210, the display device and the touch device receive gesture events, trigger interrupts, and notify the display driver and the touch driver of the kernel layer 1220 to obtain gesture data. At the kernel layer 1220 , the display driver and the touch driver receive interrupts, acquire gesture data, and convert the gesture data into gesture events and report them to the hardware abstraction layer 1230 , so that the hardware abstraction layer 1230 reports the gesture events to the framework layer 1240 . Framework layer 1240 includes protection services 1245 . The protection service 1245 is used to realize the graphical interface display proposed by the embodiments of the present disclosure, so as to realize information protection.

In an element protection operation, the protection service 1245 monitors gesture events, triggers protection functions, and displays element protection controls. Protection service 1245 monitors selection operations and determines the type of protection. Protection types include control protection and layout protection, as described above with reference to FIGS. 3A-5P . The protection service 1245 protects the control or the layout, such as blurring the content of the control or the layout, or making the control gray so that the content cannot be accessed through the control. Further, the protection service 1245 monitors the unprotection operation, and invokes a verification interface (for example, a password, fingerprint, face, etc. verification interface). When the verification information is input by the user and the verification is successful, the protection service 1245 makes the content entering the protected state displayed normally.

In addition, in the interface protection operation, the protection service 1245 monitors the gesture event, triggers the protection function, and protects the UI, such as blurring all the contents of the UI. Further, the protection service 1245 monitors the unprotect operation and determines the type of the unprotect operation. Types of unprotect operations include tap, area draw, and swipe wipe, etc., as described above with reference to FIGS. 6A-10E . Additionally, protection service 1245 determines the multi-window type. The multi-window types include single floating window, multiple floating windows, split screen, parallel horizon, etc., as described above with reference to FIGS. 6A-10E . After determining the specific multi-window type, the protection service 1245 determines the multi-window position, matches the gesture focus with the multi-window position, and makes the matched window display normally. Alternatively, the protection service 1245 determines the gesture drawing area or the gesture sliding wiping area, and makes the determined area displayed normally.

FIG. 13 shows a schematic diagram of a local clarification architecture 1300 according to an embodiment of the present disclosure. As shown in FIG. 13, at block 1310, the electronic device detects that the interface protection control is triggered, and obtains a screenshot of the UI. In block 1320, the electronic device calls the canvas.drawBitmap function, and draws the protected UI according to the screenshot of the UI to overlay on the screen. At block 1330, the electronic device displays the protected UI. The electronic device monitors gesture events. In the event that an area drawing gesture is detected, the path.moveTo and path.lineTo functions are called at block 1340 . Whereas in the case where a swipe wipe gesture is detected, the paint.setStyle and paint.setStrokeWidth functions are called at block 1350 . In block 1360, the electronic device calls an onDraw function to draw a partially cleared UI. Thus, at block 1370, the electronic device displays the partially sharpened UI.

FIG. 14 shows a schematic diagram 1400 of physical elements according to an embodiment of the disclosure. As shown in FIG. 14 , the physical elements involved in the embodiments of the present disclosure include a processor 1440 (for example, CPU, or GPU, etc.), a display device 1430 (for example, an LCD display, an LED display, or a CRT display, etc.), and a touch Device 1420 . The user can display the UI on the screen of the electronic device, and operate various UI elements (for example, windows, controls, etc.) in the UI to control the display device to safely display information.

Claims

A graphical interface display method, comprising:

displaying a user interface UI on a screen of the electronic device;

detecting protection actions associated with the UI; and

Content associated with at least a portion of the UI is disabled from normal display on the screen in response to detecting the protection operation.
The method of claim 1, wherein the content associated with at least a portion of the UI includes at least one of:

the displayed content of at least a portion of the UI, and

Content accessed via at least a portion of the UI.
The method according to claim 1 or 2, wherein said protection operation comprises:

an element protection operation for protecting a UI element in the UI, the element protection operation prevents the content associated with the UI element from being displayed normally, or

An interface protection operation for protecting the UI, where the interface protection operation prevents all content of the UI from being displayed normally.
The method according to claim 3, wherein the protection operation is the element protection operation, and making the content unable to be displayed normally comprises:

determining a target UI element corresponding to the element protection operation based on the element protection operation; and

making the content associated with the target UI element unable to be displayed normally.
The method according to claim 4, wherein the element protection operation comprises a control display operation and a selection operation, and the determining the target UI element corresponding to the element protection operation based on the element protection operation comprises:

displaying on the screen an element protection control corresponding to at least one UI element included in the UI based on the control display operation; and

Based on the selection operation, the UI element corresponding to the element protection control on which the selection operation acts is determined as the target UI element.
The method according to claim 4 or 5, wherein making the content associated with the target UI element unable to be displayed normally comprises:

determining a mode in which to protect the target UI element, the mode indicating at least one of the following:

obscure said content, and

preventing access to the content via the target UI element; and

The content associated with the target UI element cannot be displayed normally based on the mode.
The method according to claim 4 or 5, wherein making the content associated with the target UI element unable to be displayed normally comprises:

Blur said content.
The method according to claim 4 or 5, wherein making the content associated with the target UI element unable to be displayed normally comprises:

Access to the content via the target UI element is prevented.
The method according to any one of claims 4-8, wherein the protection operation is the element protection operation, and the method further comprises:

detecting an unprotect operation on said target UI element; and

In response to detecting the unprotect operation, causing the content associated with the target UI element to be displayed normally.
The method of claim 9, wherein causing the content associated with the target UI element to be displayed normally comprises:

verify the identity of the user who initiated the unprotect operation; and

In response to the successful authentication, the content associated with the target UI element is displayed normally.
The method according to claim 3, wherein the protection operation is the interface protection operation, and making all the contents of the UI unable to be displayed normally comprises:

Blur all said content of said UI.
The method according to claim 3 or 11, wherein the protection operation is the interface protection operation, and the method further comprises:

detecting an unprotect operation acting on said UI;

in response to detecting the unfencing operation, determining a target area in the UI indicated by the location where the unfencing operation occurred; and

Makes the target area appear normally.
The method according to claim 12, wherein the unprotecting operation is a click operation, the UI includes at least one window, and determining the target area includes:

Determining a target window where the click operation occurs in the at least one window as the target area.
The method of claim 12, wherein the unprotecting operation is a region painting operation, and determining the target region comprises:

A part of the UI surrounded by the area drawing operation is determined as the target area.
The method according to claim 12, wherein the unprotecting operation is a sliding wipe operation, and determining the target area comprises:

Determining the part of the UI where the sliding and wiping operation occurs as the target area.
An electronic device, characterized by comprising: a processor, and a memory storing instructions, the instructions, when executed by the processor, cause the electronic device to execute the electronic device according to any one of claims 1 to 15. Methods.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions, and when the instructions are executed by an electronic device, the electronic device executes the electronic device according to any one of claims 1 to 15. Methods.
A computer program product, characterized in that the computer program product includes instructions, which when executed by an electronic device cause the electronic device to perform the method according to any one of claims 1 to 15.