CN114298709A

CN114298709A - Data processing method and device for touch screen, terminal and storage medium

Info

Publication number: CN114298709A
Application number: CN202010995938.7A
Authority: CN
Inventors: 慕伟虎
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2020-09-21
Filing date: 2020-09-21
Publication date: 2022-04-08

Abstract

The disclosure relates to a data processing method and device of a touch screen, a terminal and a storage medium. The method is applied to the terminal and comprises the following steps: starting a safety input system of a terminal system layer in response to the terminal switching to a preset input scene; detecting touch operation on a display interface of the safety input system; and reporting the input data of the touch operation to an application program of the input scene through the safety input system. According to the technical scheme of the embodiment of the disclosure, under a preset input scene, the detection and the report of the terminal touch operation are realized by using the safety input system of the terminal system layer. The terminal does not need to perform data processing on the detected touch operation, so that the condition that the input information of the user is intercepted or maliciously cracked is reduced, and the input safety of the user in a preset input scene is improved.

Description

Data processing method and device for touch screen, terminal and storage medium

Technical Field

The present disclosure relates to electronic information processing technologies, and in particular, to a data processing method and apparatus for a touch screen, a terminal, and a storage medium.

Background

With the popularization of intelligent mobile terminals and the rapid development of electronic commerce, more and more application programs with functions of online payment, online transaction and the like are available. In order to improve the security of scenes such as online payment, password input, fingerprint verification and the like can be performed by utilizing the input function of the terminal, so that the authentication of scenes such as payment and transaction is realized. However, the terminal processes the input data and transmits the input data through the network, and the input data is easily intercepted and cracked maliciously, so that the account security of the user is hidden, and the economic loss of the user is easily caused.

Disclosure of Invention

The disclosure provides a data processing method and device of a touch screen, a terminal and a storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a data processing method for a touch screen, where the method is applied to a terminal, and includes:

starting a safety input system of a terminal system layer in response to the terminal switching to a preset input scene;

detecting touch operation on a display interface of the safety input system;

and reporting the input data of the touch operation to an application program of the input scene through the safety input system.

In some embodiments, the starting of the secure input system of the terminal in response to the terminal switching to the predetermined input scenario includes:

and responding to the terminal switching to a preset input scene, and displaying a safety keyboard of the safety input system on a display interface of the terminal.

In some embodiments, the input data comprises: touch coordinates; the detecting a touch operation at a display interface of the secure input system includes:

and detecting the touch operation aiming at the safety keyboard to obtain the touch coordinate.

In some embodiments, the reporting, by the secure input system, the input data of the touch operation to the application program of the input scenario includes:

reporting display data of the safety input system and input data of the touch operation to an application program of the input scene through the safety input system; wherein the display data comprises: and each touch coordinate of the touch screen and the image information corresponding to the touch coordinate on the display interface.

In some embodiments, the method further comprises:

and reporting the detected input data of the touch operation through a data processing system of the terminal in response to the terminal being in a scene other than the preset input scene.

In some embodiments, the reporting, by a data processing system of the terminal, the input data of the detected touch operation includes:

converting the detected touch coordinates of the touch operation into the input data through a data processing system of the terminal;

and reporting the input data.

In some embodiments, the predetermined input scenario includes: a payment scenario for any application with transaction functionality.

According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus for a touch screen, where the apparatus is applied to a terminal, and the apparatus includes:

the starting module is used for responding to the switching of the terminal to a preset input scene and starting a safety input system of a terminal system layer;

the detection module is used for detecting touch operation on a display interface of the safety input system;

and the first reporting module is used for reporting the input data of the touch operation to the application program of the input scene through the safety input system.

In some embodiments, the start module includes:

and the display sub-module is used for responding to the switching of the terminal to a preset input scene and displaying the safety keyboard of the safety input system on the display interface of the terminal.

In some embodiments, the input data comprises: touch coordinates; the detection module comprises:

and the detection submodule is used for detecting the touch operation aiming at the safety keyboard to obtain the touch coordinate.

In some embodiments, the first reporting module includes:

the first reporting submodule is used for reporting display data of the safe input system and input data of the touch operation to an application program of the input scene through the safe input system; wherein the display data comprises: and each touch coordinate of the touch screen and the image information corresponding to the touch coordinate on the display interface.

In some embodiments, the apparatus further comprises:

and the second reporting module is used for responding to a scene, outside the preset input scene, of the terminal and reporting the detected input data of the touch operation through a data processing system of the terminal.

In some embodiments, the second reporting module includes:

the conversion sub-module is used for converting the detected touch coordinates of the touch operation into the input data through a data processing system of the terminal;

and the reporting submodule is used for reporting the input data.

According to a third aspect of the embodiments of the present disclosure, there is provided a terminal, the terminal at least comprising: a processor and a memory for storing executable instructions capable of running on the processor, wherein:

when the processor is used for running the executable instructions, the executable instructions execute the steps in the data processing method of any touch screen.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored therein computer-executable instructions, which when executed by a processor, implement the steps in the data processing method of the touch screen of any one of the above.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: in the embodiment of the disclosure, under a predetermined input scene, the detection and the report of the terminal touch operation are realized by using the security input system of the terminal system layer. The terminal does not need to perform data processing on the detected touch operation, so that the condition that the input information of the user is intercepted or maliciously cracked is reduced, and the input safety of the user in a preset input scene is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a first flowchart illustrating a method for processing data of a touch screen according to an exemplary embodiment;

FIG. 2 is a flowchart illustrating a data processing method of a touch screen according to an exemplary embodiment;

FIG. 3 is a flowchart illustrating a data processing method for a touch screen according to an exemplary embodiment;

FIG. 4 is a block diagram illustrating a data processing apparatus of a touch screen according to an exemplary embodiment;

fig. 5 is a block diagram illustrating an entity structure of a terminal according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

Fig. 1 is a flowchart illustrating a data processing method of a touch screen according to an exemplary embodiment, where the method is applied to a terminal, as shown in fig. 1, and includes the following steps:

step S101, responding to the terminal switching to a preset input scene, and starting a safety input system of a terminal system layer;

step S102, touch operation is detected on a display interface of the safety input system;

and step S103, reporting the input data of the touch operation to the application program of the input scene through the safety input system.

When the terminal is in input operation, if the character input is performed by the touch screen, an input method application program or a system self-contained input method is generally used; if the input is fingerprint input, voice input or image input, the sensing or acquisition component of the terminal is utilized to acquire the fingerprint, voice or image, and then corresponding input information is obtained through processing.

In any of the above input modes, the information provided by the user is acquired by using the sensing or acquisition function of the acquisition component of the terminal, and then the data is analyzed and processed by the data processing system of the terminal itself and encoded into data convenient for transmission and storage. For example, the terminal processes the touch coordinates acquired by the touch screen to obtain input data corresponding to the touch coordinates, and for example, the terminal acquires a fingerprint image by using the fingerprint acquisition assembly and then encodes the fingerprint image into data carrying fingerprint characteristic information through image processing. Then, the data processing system of the terminal may report the processed data to the corresponding application program for identification and use by the application program.

However, the above method is easy to reveal security information when the terminal processes and reports data, and is easy to be maliciously intercepted or stolen during the processing and reporting processes. Especially, in a scenario with functions of online transaction and the like, the input identification information may be stolen during the processing or reporting process of the terminal, thereby causing potential safety hazards.

Therefore, in the embodiment of the present disclosure, when the terminal is switched to a predetermined input scenario, the secure input system of the terminal system layer is started to detect the touch input, acquire data, report the data, and the like.

The security input system can directly report the input data of the detected input operation to the application program without data processing of the terminal. For example, the secure input system may draw graphical input data according to touch input, and directly report the input data without decryption, image processing, semantic analysis, or the like of the data.

It should be noted that the predetermined input scenario here can be set in advance according to the user requirement, for example: a login page, an information input page, a payment page, etc. of the specified application. The page input into the predetermined input scene can also be automatically identified according to the predetermined page characteristics: when a page containing characters such as 'input password/fingerprint' or payment two-dimensional codes, bar codes and the like needs to be input, the current scene is determined to be a preset input scene. Since the secure input system is terminal system-level and is not limited to a single application, it can be used when any application has the requirement. For example: payment scenes of various application programs such as any bank official application program, payment treasure, WeChat and E-commerce platform, and the like.

Therefore, under some scenes such as password input, payment and other scenes with higher safety requirements on input information, the safety input system can be used for detecting touch input and reporting input data, so that a data processing system of the terminal aiming at a touch screen is avoided, the risk of stealing the input data is reduced, and the data safety performance is improved.

In the embodiment of the present disclosure, when the terminal is switched to a predetermined input scene, the secure input system of the terminal system layer is started, and the secure input system may provide a TUI (Trust User Interface) as a display Interface. The display interface is used for receiving input operations of a user, such as: fingerprint input in screen, text input of authentication information such as account name and password, and the like.

In the embodiment of the present disclosure, the input control displayed by the display interface may be a graphical drawing interface, a fingerprint input interface, an encrypted input keyboard, or the like. For example, the encrypted input keyboard may display a keyboard having a different arrangement from that of a keyboard of a system-owned input method or the like, or having different display positions of characters.

By using the security keyboard, a user can be guided to input passwords, fingerprints, password graphs and the like, and after the touch screen detects a touch operation, only information such as a touch position and the like is acquired. The data processing system of the terminal does not participate in processing of data corresponding to the touch coordinates, and reports the data to the application program through the safety input system, so that the encryption state of input information is always maintained in the data processing and reporting process at the terminal side, the difficulty of interception and cracking is increased, and the input safety performance is improved.

Touch sensors are distributed in the touch screen, the action position of touch operation can be sensed, and the accurate coordinate of the touch operation can be determined according to the row and column positions of the touch sensors in the array. The terminal can execute corresponding operations aiming at different display interfaces according to the touch coordinates obtained by detecting the touch operations.

In the embodiment of the present disclosure, in the predetermined input scene, the terminal displays the touch coordinates obtained by the touch operation on the display interface provided by the security input system. Because the touch data are directly reported through the safety input system, the data processing system of the terminal does not need to process and respond the touch coordinates. The touch coordinates can be directly reported to the application program as touch data through the safety input system.

If viruses, malicious programs and the like exist to intercept the reported touch data, the opposite side can only obtain touch coordinates, but cannot interpret encrypted information such as passwords, fingerprints and graphs corresponding to the touch coordinates.

In the embodiment of the disclosure, when reporting the touch data to the application program, the secure input system may report the touch coordinate and the corresponding image information on the display interface to the application program at the same time. Because the touch coordinates are encrypted according to the rules provided by the security input system when the information such as characters, images or fingerprints corresponding to the touch coordinates is displayed on the display interface provided by the security input system, the application program may not know the actual information input by the user when the touch coordinates are directly reported. For example, the character arrangement position on the security keyboard provided by the security input system is different from the character arrangement position of the keyboard provided by programs such as other input methods of the terminal, and even the security keyboard with random arrangement positions can be dynamically generated each time the security input system is started. At this time, the secure input system can report the touch coordinates detected by the touch screen and the information of the secure keyboard to the application program at the same time, so that the application program can decode conveniently.

In some embodiments, the method further comprises:

In the embodiment of the present disclosure, the above-described security input system may be applied only to an input scenario set by a user or determined according to a predetermined rule, and in other scenarios, for example: the instant messaging, the short message editing, the search bar input and other various input scenes do not need to start a safety input system, but directly utilize a data processing system of the terminal to process the detected input data of the touch operation and report the input data to an application program and the like. The application program directly responds according to the acquired data.

and reporting the input data.

Here, the touch data detected by the touch screen is touch coordinates, and there is no actual meaning. Therefore, the data processing system of the terminal needs to convert the touch coordinates into corresponding input data according to the page information displayed by the application program, and report the data, so as to obtain a response to the touch operation.

In the embodiment of the disclosure, in consideration of the fact that a user needs to input a password or perform authentication operations such as fingerprint verification and the like in a payment scene with a transaction function, the authority of the transaction is acquired, and then deduction, transfer and the like of a bound account are realized. Under these circumstances, it is usually necessary to implement a fast transaction process, which is not suitable for a complicated authentication process, and therefore, the secure input system in the embodiment of the present disclosure may be used for input.

Therefore, the terminal can directly report the input information to the application program without complex processing, the operation is simple, quick transaction is easy to realize, and meanwhile, the information security in a payment scene is improved.

The disclosed embodiments also provide the following examples:

when the terminal performs payment, bank account information processing, login and registration of some application program accounts, and the like, information input by the user may include information such as personal privacy and passwords, and thus a more secure input environment is required.

In some embodiments, the application may open a CA (Certificate Authority) interface, thereby enabling entry of the trusted environment. Comprising the steps as shown in figure 2:

step S201, when the CA interface receives the input operation of the user, the TUI display request of the CA is triggered.

Step S202, according to the TUI display request, switching the display Environment of the CA from the REE (Rich Execution Environment) to the TEE (Trusted Execution Environment).

Here, the REE refers to an execution environment common to all mobile devices, and the TEE is a trusted execution environment with higher security.

Step S203, displaying a TA (Trusted Application) interface of the CA in the TEE.

Thus, high-safety input is realized. However, the TA interface at this time is only limited to the improvement of the security performance of the application side, but it is not distinguished whether the input data of the terminal-side touch screen is trusted or not, and the touch data is processed and reported through an input subsystem of the terminal itself. Therefore, the input data of the touch screen still has security hidden trouble.

In the embodiment of the disclosure, each input scene of the terminal is managed based on a system layer of the terminal, and whether the input scene is an input scene with higher security requirements, such as a payment scene, is distinguished. And if the current input scene is detected to be a scene needing to be executed in the trusted environment, the data processing is not carried out through a data processing system in an operating system of the terminal, and the TUI system is used for directly drawing and reporting the touch input. The CA interface of the application program directly responds to the TUI drawing time to complete the actions of payment and the like. As shown in fig. 3, the method comprises the following steps:

step S301, judging whether the scene is a payment scene;

step S302, if the payment scene is a payment scene, drawing touch input detected by a touch screen through TUI;

step S403, the CA interface responds to the event drawn by the TUI;

step S304, if the scene is a non-payment scene, processing touch input detected by the touch screen through an input subsystem of the terminal;

step S305, the application responds to the input event.

Therefore, under a payment scene, the TUI system non-signature program cannot interfere, and the risk of leakage of input data of the touch screen is reduced.

If the application page is a non-payment application page, the input operation is more complicated, and an input subsystem of the terminal needs to process data such as touch coordinates detected by a touch screen and report the obtained input data to the application. The application responds according to the input data.

By the method, the touch data in the payment scene are directly drawn by the TUI, the processing difficulty of the data is reduced, and the development cost is saved. Meanwhile, the risk that sensitive information input by a user is stolen is reduced, and therefore the safety of user input operation is improved.

Fig. 4 is a block diagram illustrating a data processing apparatus of a touch screen according to an exemplary embodiment, and as shown in fig. 4, the apparatus 400 is applied to a terminal and includes:

a starting module 401, configured to start a secure input system of a terminal system layer in response to a terminal being switched to a predetermined input scenario;

a detection module 402, configured to detect a touch operation on a display interface of the secure input system;

a first reporting module 403, configured to report the input data of the touch operation to the application program of the input scene through the secure input system.

In some embodiments, the start module includes:

In some embodiments, the first reporting module includes:

In some embodiments, the apparatus further comprises:

In some embodiments, the second reporting module includes:

and the reporting submodule is used for reporting the input data.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 5 is a block diagram illustrating a terminal 500 according to an example embodiment. For example, the terminal 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and so forth.

Referring to fig. 5, terminal 500 may include one or more of the following components: a processing component 501, a memory 502, a power supply component 503, a multimedia component 504, an audio component 505, an input/output (I/O) interface 506, a sensor component 507, and a communication component 508.

The processing component 501 generally controls the overall operation of the terminal 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 501 may include one or more processors 510 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 501 may also include one or more modules that facilitate interaction between the processing component 501 and other components. For example, the processing component 501 may include a multimedia module to facilitate interaction between the multimedia component 504 and the processing component 501.

The memory 510 is configured to store various types of data to support operations at the terminal 500. Examples of such data include instructions for any application or method operating on terminal 500, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 502 may be implemented by any type or combination of volatile or non-volatile storage devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 503 provides power to the various components of the terminal 500. The power supply component 503 may include: a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for terminal 500.

The multimedia components 504 include a screen that provides an output interface between the terminal 500 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 504 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the terminal 500 is in an operation mode, such as a photographing mode or a video mode. Each front camera and/or rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 505 is configured to output and/or input audio signals. For example, the audio component 505 includes a Microphone (MIC) configured to receive external audio signals when the terminal 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 510 or transmitted via the communication component 508. In some embodiments, audio component 505 further comprises a speaker for outputting audio signals.

The I/O interface 506 provides an interface between the processing component 501 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 507 includes one or more sensors for providing various aspects of status assessment for the terminal 500. For example, sensor component 507 can detect an open/closed state of terminal 500, relative positioning of components such as a display and keypad of terminal 500, sensor component 507 can also detect a change in position of terminal 500 or a component of terminal 500, the presence or absence of user contact with terminal 500, orientation or acceleration/deceleration of terminal 500, and a change in temperature of terminal 500. The sensor assembly 507 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 507 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 507 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 508 is configured to facilitate communications between the terminal 500 and other devices in a wired or wireless manner. The terminal 500 may access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 508 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 508 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, or other technologies.

In an exemplary embodiment, the terminal 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as the memory 502 comprising instructions, executable by the processor 510 of the terminal 500 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the method provided in any of the embodiments.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A data processing method of a touch screen is applied to a terminal and comprises the following steps:

detecting touch operation on a display interface of the safety input system;

2. The method of claim 1, wherein initiating a secure input system of the terminal in response to the terminal switching to a predetermined input scenario comprises:

3. The method of claim 2, wherein the input data comprises: touch coordinates; the detecting touch operation on the display interface of the security input system includes:

4. The method according to any one of claims 1 to 3, wherein reporting the input data of the touch operation to the application program of the input scenario through the secure input system comprises:

5. The method of claim 1, further comprising:

6. The method according to claim 5, wherein reporting the input data of the detected touch operation through a data processing system of the terminal comprises:

and reporting the input data.

7. The method of claim 1, wherein the predetermined input scenario comprises: a payment scenario for any application with transaction functionality.

8. A data processing device of a touch screen is applied to a terminal and comprises:

9. The apparatus of claim 8, wherein the activation module comprises:

10. The apparatus of claim 9, wherein the input data comprises: touch coordinates; the detection module comprises:

11. The apparatus according to any one of claims 8 to 10, wherein the first reporting module comprises:

12. The apparatus of claim 8, further comprising:

13. The apparatus of claim 12, wherein the second reporting module comprises:

and the reporting submodule is used for reporting the input data.

14. The apparatus of claim 8, wherein the predetermined input scenario comprises: a payment scenario for any application with transaction functionality.

15. A terminal, characterized in that the terminal comprises at least: a processor and a memory for storing executable instructions operable on the processor, wherein:

the processor is configured to execute the executable instructions, and the executable instructions perform the steps of the data processing method of the touch screen provided in any one of the above claims 1 to 7.

16. A non-transitory computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and when executed by a processor, the computer-executable instructions implement the steps in the data processing method of the touch screen provided in any one of claims 1 to 7.