CN116225274A - Identification method and device for touch operation, electronic equipment and storage medium - Google Patents

Abstract

The present application discloses a touch operation recognition method, device, electronic equipment, and storage medium. The method is applied to electronic equipment, and the electronic equipment includes: a touch screen, an application processor AP, and a baseband processor CP. The method Including: the AP monitors whether there is a touch operation on the touch screen; the CP monitors the acceleration data and stores the monitored acceleration data in the storage module; when the AP detects a touch operation, the CP Identify whether the touch operation is a knuckle tapping action, and send the identification result to the AP. With this embodiment, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data. Compared with the prior art, the operation of transmitting the acceleration data from the CP to the AP, and the AP performs identification according to the received acceleration data is omitted. The touch operation can be recognized in time, and the recognition rate of knuckle tapping on the screen can be improved.

Description

Recognition method, device, electronic device and storage medium for touch operation

technical field

The present application relates to the technical field of electronic equipment, and in particular to a touch operation recognition method, device, electronic equipment, and storage medium.

Background technique

With the diversification of functions of electronic devices such as mobile phones and tablets, users can save images or videos displayed on the screen through functions such as screen capture, area screen capture, or screen recording. In addition to starting related functions through menus or pressing physical buttons, you can also use your knuckles to click or double-click on the touch screen to start functions such as screen capture, area screen capture, or screen recording.

In actual operation, when using knuckles to tap the display screen to start related functions, there is a problem of low success rate or high misoperation.

Therefore, how to improve the success rate of recognizing knuckle operations is a technical problem to be solved urgently.

Contents of the invention

The present application provides a touch operation recognition method, device, electronic equipment and storage medium, which can improve the recognition rate of knuckle tapping on the screen.

In the first aspect, an embodiment of the present application provides a touch operation recognition method, which is applied to an electronic device, and the electronic device includes: a touch screen, an application processor (application processor, AP) and a baseband processor (cooperate processor, CP) ), the method includes:

The AP monitors whether there is a touch operation on the touch screen;

The CP monitors acceleration data, and stores the monitored acceleration data in a storage module;

When the AP detects a touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module, and sends the identification result to the AP.

When using this embodiment to identify the touch operation, when the AP detects the touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data. Compared with the technical solution in the prior art, where the CP transmits the acceleration data to the AP, and the AP identifies whether the touch operation is a knuckle tapping action according to the received acceleration data, the embodiment of the present application omits the storage module The acceleration data in the CP is transmitted from the CP to the AP operation, which can recognize the touch operation in a timely manner, and can improve the recognition rate of the knuckles tapping the screen.

In a possible implementation manner, the method for identifying a touch operation may further include:

Acquire the capacitance value of the touch screen when the AP detects a touch operation;

The AP identifies whether the touch operation is a knuckle tapping action according to the capacitance value;

When the recognition result obtained by the AP according to the capacitance value and the recognition result obtained by the CP according to the acceleration data are both knuckle tapping actions, it is determined that the touch operation is a knuckle tapping action.

In this embodiment, the AP recognizes the touch operation according to the capacitance value of the touch screen, and the touch operation is determined only when the recognition result obtained by the AP according to the capacitance value and the recognition result obtained by the CP according to the acceleration data are both knuckle tapping actions. The control operation is a knuckle tapping action, and the recognition accuracy is improved by adopting this embodiment.

In a possible implementation manner, the method for identifying a touch operation may further include:

When the recognition result obtained by the AP according to the capacitance value and the recognition result obtained by the CP according to the acceleration data are not both knuckle tapping actions, it is determined that the touch operation is a click operation.

In a possible implementation manner, the storage module is a storage module whose power consumption is lower than that of a double rate synchronous dynamic random access memory (DDR).

Using this embodiment, compared to using DDR to store acceleration data in the prior art, using a memory module with lower power consumption than DDR to store acceleration data is beneficial to reduce power consumption.

In the second aspect, an embodiment of the present application provides a touch operation method, which is applied to an electronic device, and the electronic device includes: a touch screen, an application processor AP, and a baseband processor CP, and the method includes:

The AP monitors whether there is a touch operation on the touch screen;

The CP monitors acceleration data, and stores the monitored acceleration data into a preset storage module;

When the AP detects a touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module, and sends the identification result to the AP;

In response to the touch operation, the AP performs a preset operation matching the recognized type of the touch operation.

In a possible implementation manner, when it is determined that the touch operation includes a knuckle tapping action, the preset operation includes: any one of screen capture, area screen capture, or screen recording.

In the third aspect, the embodiment of the present application provides a touch operation recognition device, which is applied to electronic equipment, and the electronic device includes: a touch screen, and the recognition device includes: an application processor AP and a baseband processor CP,

The AP is used to monitor whether there is a touch operation on the touch screen;

The CP is configured to monitor acceleration data, and store the monitored acceleration data in a preset storage module, and is configured to identify according to the acceleration data stored in the storage module when the AP detects a touch operation. Whether the touch operation is a knuckle tapping action, and send the recognition result to the AP.

In a fourth aspect, an embodiment of the present application provides a touch operation device, which is applied to an electronic device, and the electronic device includes: a touch screen, and the touch operation device includes: a response module and the third aspect Recognition device for touch operation;

The response module is configured to execute a preset operation matching the touch operation in response to the touch operation when the recognition device determines that the touch operation includes a knuckle tapping action.

In the fifth aspect, the embodiment of the present application provides an electronic device, including: a touch screen, an application processor AP, a baseband processor CP, and a memory storing instructions, and the instructions are executed by the AP and the CP When executed, the electronic device executes the method described in the first aspect or any possible implementation of the first aspect or the method described in the second aspect or any possible implementation of the second aspect. method.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores instructions, and when the instructions are executed by an electronic device, the electronic device performs the above-mentioned first aspect. Or the method described in any possible implementation manner of the first aspect, or the method described in the above second aspect or any possible implementation manner of the second aspect.

In a seventh aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes instructions, and the instructions, when executed by an electronic device, cause the electronic device to perform the above-mentioned first aspect or the first aspect. The method described in any possible implementation manner or the method described in the second aspect or any possible implementation manner of the second aspect.

It can be understood that, for the beneficial effects achieved by the above second aspect to the seventh aspect, reference may be made to the above first aspect or the corresponding beneficial effects of the method in any possible implementation manner of the first aspect.

Description of drawings

FIG. 1 is a schematic flowchart of a method for recognizing touch operations provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of information interaction between an AP and a CP in the prior art;

FIG. 3 is a schematic diagram of information interaction between an AP and a CP in an embodiment of the present application;

FIG. 4 is a schematic flowchart of a touch operation recognition method provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a touch operation method provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a touch-operated recognition device provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a touch operation device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 9 is a block diagram of a software system of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manner of the present application will be further described in detail below in conjunction with the accompanying drawings.

It should be understood that the "plurality" mentioned in this application means two or more. In the description of this application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in this article is just a description of the relationship between associated objects, It means that there can be three kinds of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solution of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

When using electronic devices with touch screens such as mobile phones and tablets, users can perform related operations on the electronic devices by tapping the touch screen with their knuckles. Identify the type of touch operation to determine whether the touch operation is a joint tapping operation or a common touch click operation. Then, in response to the determined type of the touch operation, a matching related operation is performed. At present, most smartphones use the system architecture of AP plus CP. AP is used to run the operating system and handle high-load multimedia applications, and CP is used to complete functions such as network interaction.

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of a method for recognizing a touch operation. As shown in FIG. 1, the method for recognizing a touch operation may include steps 101 to 106, wherein:

101. A touch operation on the touch screen is detected.

102. Obtain a first result according to the capacitance value of the touch screen, the first result is: determine whether the touch operation is a knuckle tapping action.

103. Obtain a second result according to the acceleration data, and the second result is: determine whether the touch operation is a knuckle tapping action.

In some possible implementations, the acceleration data can be monitored according to the knuckle virtual sensor. The virtual sensor is mainly aimed at the measurand that is difficult to measure directly or the cost of direct measurement is very high. Through modeling to measure the relevant quantities, and then process the measured signal, the information of the measurand can be obtained indirectly. In a sense, it is a mathematical model, which can realize the output of the measured data through the creation of an extended system model based on the relevant original data. The raw data of the virtual sensor can come from the virtual machining process or the virtual simulation process, and can also come from the real physical sensor.

104. Determine whether the first result and the second result both refer to a knuckle tapping action.

If yes, execute step 105; if not, execute step 106.

105. Determine that the touch operation is a knuckle tapping action.

106. Determine that the touch operation is an ordinary touch and click operation.

It should be noted that the difference between the embodiment of the present application and the prior art is reflected in the specific implementation of step 103 in FIG. 1 .

Referring to FIG. 2 and FIG. 3 , FIG. 2 is a schematic diagram of information interaction between an AP and a CP in the prior art, and FIG. 3 is a schematic diagram of information interaction between an AP and a CP in an embodiment of the present application. Specifically, as shown in FIG. 2, in the prior art, the CP monitors the acceleration data and stores the acceleration data in the storage module. When the AP determines that there is a touch operation, the CP sends the acceleration data stored in the storage module to the AP. On the AP side, the knuckle driver requests acceleration data, and determines whether the touch operation is a knuckle tapping action based on the knuckle algorithm and the acceleration data obtained from the CP. It can be seen from Figure 2 that the CP does not identify the monitored acceleration data, but transmits the acceleration data stored in the storage module to the AP, and the AP identifies it according to the received acceleration data to determine whether it is a knuckle tapping action. Because the acceleration data stored in the storage module is usually relatively large, for example, the transmitted acceleration data can be 128 numbers, and it takes a certain amount of time for the acceleration data to be transmitted from the CP to the AP. In addition, there may be errors such as data loss or change during data transmission. , which may affect the recognition efficiency and accuracy.

As an improvement, the technical solution adopted in the embodiment of the present application can refer to Figure 3. The CP monitors the acceleration data and stores the acceleration data in the storage module. The storage module can be used to store a preset number (such as 128 numbers, etc.) Acceleration data. When the storage module is full, the newly monitored acceleration data will overwrite the first stored data. When the AP determines that there is a touch operation, the CP determines the touch operation according to the preset number of acceleration data stored in the storage module. It determines whether the operation is a knuckle tapping action, and then sends the recognition result whether it is a knuckle tapping action to the AP. With this embodiment, the acceleration data stored in the storage module does not need to be transmitted from the CP to the AP, which saves transmission time, avoids errors in transmission, and improves the recognition rate of knuckle tapping on the screen.

In addition, in the prior art, when the electronic device turns on the knuckle recognition function, the DDR is woken up, and the acceleration data monitored by the CP is stored in the DDR. In this embodiment of the application, in order to save power consumption, without waking up DDR, the acceleration data monitored by the CP is stored in a storage module with lower power consumption than DDR (such as Island low power consumption). This method is beneficial to reduce power consumption.

It should be noted that the electronic device involved in the embodiment of the present application is an electronic device with a touch screen, specifically, a mobile phone, a tablet computer, etc., and the specific type of the electronic device is not limited in the embodiment of the present application. For ease of understanding, the method for identifying touch operations provided by the present application will be described in detail below by taking the electronic device as an example of a mobile phone.

Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a method for recognizing a touch operation provided by an embodiment of the present application. In this embodiment, the method for identifying a touch operation includes steps 401 to 404, wherein:

401. The AP monitors whether there is a touch operation on the touch screen.

Specifically, the AP can monitor whether there is a touch operation on the touch screen through a sensor on the electronic device.

402. The CP monitors the acceleration data, and stores the monitored acceleration data in the storage module.

Specifically, the CP can use the knuckle virtual sensor to obtain the monitored acceleration data.

403. When the AP detects a touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module.

404. The CP sends the identification result to the AP.

When using this embodiment to identify the touch operation, when the AP detects the touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module. Compared with the technical solution in the prior art, where the CP transmits the acceleration data in the storage module to the AP, and the AP identifies whether the touch operation is a knuckle tapping action according to the received acceleration data, the embodiment of the present application omits In order to transmit the acceleration data in the storage module from the CP to the AP, the touch operation can be recognized in time, and the recognition rate of the knuckle tapping the screen can be improved.

In some possible implementations, when the AP detects a touch operation, the AP identifies whether the touch operation is a knuckle tapping action according to the capacitance value of the touch screen; The network model combines the capacitance value of the touch screen to determine whether the touch operation is a knuckle tapping action. When the recognition result obtained by the AP based on the capacitance value of the touch screen and the recognition result obtained by the CP based on the acceleration data are both knuckle tapping actions, it is determined that the touch operation is a knuckle tapping action.

It can be understood that when the user clicks, slides or taps with knuckles on the touch screen, the capacitance value of the clicked, slide or tapped area will change, and the touch driver module can monitor the capacitance value of each area on the touch screen .

In this embodiment, the AP recognizes the touch operation according to the capacitance value of the touch screen, and only when the recognition result obtained by the AP according to the capacitance value data and the recognition result obtained by the CP according to the acceleration data are both knuckle tapping actions The touch operation is a knuckle tapping action, and the adoption of this embodiment improves the accuracy of recognition.

In some possible implementation manners, the storage module storing acceleration data is a storage module with lower power consumption than DDR. The CP stores acceleration data in a memory module with low power consumption.

Using this embodiment, compared with using DDR to store acceleration data in the prior art, using a low-power storage module to store acceleration data is beneficial to reduce power consumption.

As shown in Figure 5, the embodiment of the present application also provides a touch operation method, which is applied to an electronic device. The electronic device includes: a touch screen, an application processor AP, and a baseband processor CP. The touch operation method includes the following steps :

501. The AP monitors whether there is a touch operation on the touch screen.

502. The CP monitors the acceleration data, and stores the monitored acceleration data in the storage module.

In some possible implementation manners, the monitored acceleration data may be obtained by using the knuckle virtual sensor. The monitored acceleration data may be stored in a low-power storage module corresponding to the low-power island.

503. When the AP detects a touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module.

504. The CP sends the recognition result obtained according to the acceleration data to the AP.

In some possible implementation manners, it may also include: when the AP detects the touch operation, the AP identifies whether the touch operation is a knuckle tapping action according to the capacitance value of the touch screen; When the obtained recognition result and the recognition result obtained by the CP based on the acceleration data are both knuckle tapping actions, it is determined that the touch operation is a knuckle tapping action.

505. In response to the touch operation, the AP performs a preset operation matching the type of the touch operation.

Wherein, the preset operation may include: any one of operations such as screen capture, area screen capture, or screen recording.

When using this embodiment to perform a touch operation, when the AP detects the touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module. Compared with the technical solution in the prior art, where the CP transmits the acceleration data stored in the storage module to the AP, and the AP identifies whether the touch operation is a knuckle tapping action according to the received acceleration data, the embodiment of the present application saves The operation of transmitting the acceleration data from the CP to the AP is removed, and the touch operation can be recognized in a timely manner, which can improve the recognition efficiency of the knuckles tapping the screen, and is conducive to responding to the user's touch operation on the touch screen in a timely and accurate manner. .

As shown in Figure 6, the embodiment of the present application also provides a touch-operated identification device 600, which is applied to electronic equipment, the electronic equipment includes: a touch screen, and the identification device 600 includes: an application processor AP601 and a baseband processor CP602 , AP601 is used to monitor whether there is a touch operation on the touch screen; CP602 is used to monitor the acceleration data and store the monitored acceleration data in the storage module, and when AP601 detects a touch operation, according to the storage module The stored acceleration data identifies whether the touch operation is a knuckle tapping action, and is used to send the identification result to the AP601.

In some possible implementations, the AP601 is also used to identify whether the touch operation is a knuckle tapping action according to the capacitance value of the touch screen when the AP601 detects the touch operation; When the result and the recognition result obtained by the CP602 based on the acceleration data stored in the storage module are both knuckle tapping actions, the touch operation is determined to be a knuckle tapping action.

In some possible implementation manners, the memory module is a memory module with lower power consumption than DDR.

As shown in FIG. 7 , the embodiment of the present application also provides a touch operation device, which is applied to electronic equipment. The electronic equipment includes a touch screen. 702 includes: application processor AP7021 and baseband processor CP7022;

The response module 701 is configured to execute a preset operation matching the touch operation in response to the touch operation when the recognition device 702 determines that the touch operation includes a knuckle tapping action.

AP7021 is used to monitor whether there is a touch operation on the touch screen; CP7022 is used to monitor the acceleration data, and store the monitored acceleration data in the storage module, and is used when the AP7021 detects a touch operation, according to the storage module The stored acceleration data identifies whether the touch operation is a knuckle tapping action, and sends the identification result to the AP7021.

In some possible implementations, the AP7021 is also used to identify whether the touch operation is a knuckle tapping action according to the capacitance value of the touch screen when the AP7021 detects the touch operation; When the result and the recognition result obtained by CP7022 based on the acceleration data are both knuckle tapping actions, the touch operation is determined to be a knuckle tapping action.

In some possible implementation manners, the memory module is a memory module whose power consumption is lower than that of a double rate synchronous dynamic random access memory (DDR).

Next, the electronic device involved in the embodiment of the present application will be described.

FIG. 8 is a schematic structural diagram of an electronic device 800 provided by an embodiment of the present application, which may specifically be a mobile phone, a tablet computer, and other devices. 8, the electronic device 800 may include a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (universal serial bus, USB) interface 830, a charging management module 840, a power management module 841, a battery 842, and an antenna 1. Antenna 2, mobile communication module 850, wireless communication module 860, audio module 870, speaker 870A, receiver 870B, microphone 870C, earphone jack 870D, sensor module 880, button 890, motor 891, indicator 892, camera 893, screen 894, and a subscriber identification module (subscriber identification module, SIM) card interface 895, etc. Among them, the sensor module 880 may include a pressure sensor 880A, a gyroscope sensor 880B, an air pressure sensor 880C, a magnetic sensor 880D, an acceleration sensor 880E, a distance sensor 880F, a proximity light sensor 880G, a fingerprint sensor 880H, a temperature sensor 880J, a touch sensor 880K, an environment Light sensor 880L, bone conduction sensor 880M, etc.

The processor 810 may include one or more processing units, for example: the processor 810 may include an AP, a CP, a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (imagesignal processor, ISP) , controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

Wherein, the controller may be the nerve center and command center of the electronic device 800 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 810 for storing instructions and data. In some embodiments, the memory in processor 810 is a cache memory. The memory may hold instructions or data that the processor 810 has just used or recycled. If the processor 810 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 810 is reduced, thereby improving the efficiency of the system. The processor can also set low-power memory (for example, Island low power consumption, etc.) to reduce power consumption.

The electronic device 800 implements a display function through a GPU, a screen 894, an application processor, and the like. The GPU is a microprocessor for image processing, which is connected to the screen 894 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 810 may include one or more GPUs that execute program instructions to generate or alter display information.

The screen 894 is used to display images, videos and the like. Screen 894 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode, or an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-OLED, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 800 may include 1 or N screens 894, where N is an integer greater than 1.

The electronic device 800 can realize the shooting function through an ISP, a camera 893 , a video codec, a GPU, a screen 894 and an application processor.

The ISP is used to process the data fed back by the camera 893. For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 893 .

Camera 893 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 800 may include 1 or N cameras 893, where N is an integer greater than 1.

The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 800 . The external memory card communicates with the processor 810 through the external memory interface 820 to realize the data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 821 may be used to store computer-executable program code, which includes instructions. The processor 810 executes various functional applications and data processing of the electronic device 800 by executing instructions stored in the internal memory 821 . The internal memory 821 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phonebook, etc.) created by the electronic device 800 during use. In addition, the internal memory 821 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The acceleration sensor 880E can detect the acceleration of the electronic device 800 in various directions (generally three axes). When the electronic device 800 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 880E can also be used to identify the posture of the electronic device 800, and can be used in applications such as horizontal and vertical screen switching, pedometers, and the like. Of course, the acceleration sensor 880E can also be combined with the gyroscope sensor 880B to identify the posture of the electronic device 800 and be applied to switch between horizontal and vertical screens.

The gyro sensor 880B can be used to determine the motion posture of the electronic device 800 . In some embodiments, the angular velocity of the electronic device 800 about three axes (ie, x, y and z axes) may be determined by the gyro sensor 880B. The gyro sensor 880B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 880B detects the shaking angle of the electronic device 800, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 800 through reverse movement to achieve anti-shake. The gyroscope sensor 880B can also be used for horizontal and vertical screen switching, navigation, and somatosensory game scenes.

It can be understood that, the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 800 . In other embodiments of the present application, the electronic device 800 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The electronic device provided in the embodiment of the present application may be user equipment (user equipment, UE), such as a mobile terminal (such as a mobile phone), a tablet computer, and other devices.

In addition, an operating system runs on top of the above components. For example, it may be the Android open source operating system developed by Google.

The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In order to more clearly illustrate the touch operation recognition method provided by the embodiment of the present application, the embodiment of the present application takes an Android (Android) system with a layered architecture as an example to illustrate the software system of the electronic device.

As shown in FIG. 9 , the electronic device may include a hardware layer and a software layer, wherein an Android system with a layered architecture may include an application layer, an application framework layer, a system library layer and a kernel layer. In some optional embodiments, the system of the electronic device may also include layers not mentioned above in the technical architecture, such as Android Runtime (Android Runtime). The application layer can include a series of application packages, such as navigation application, music application, video application, and knuckle tapping screen application, etc. The application package can include video, chat and other applications, as well as a system user interface (system user interface, SystemUI). The knuckle tap screen application can be used for screenshots, screen recordings, long screenshots, and area screenshots.

Applications such as video and chat are used to provide users with corresponding services. For example, a user uses a video application to watch a video, uses a chat application to chat with other users, and uses a music application to listen to music.

The SystemUI is used to manage the human-computer interaction interface (user interface, UI) of the electronic device. In the embodiment of the present application, the SystemUI is used to monitor the touch operation on the touch screen.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer may include a window management service module (window management service, WMS), a display rotation module (also known as DisplayRotation), an application management service module (activity management service, AMS), and an input management module (also known as Input).

WMS is used to manage window programs. The window manager can obtain the screen size, determine whether there is a status bar, cut out images on the screen and capture the screen, etc. In the embodiment of the present application, the WMS can create and manage the windows corresponding to the application.

The display rotation module is used to control the rotation of the screen, so that the screen presents a vertical or horizontal layout through rotation. For example, when it is determined that the screen needs to be rotated, Surfaceflinger is notified to switch between the horizontal and vertical screens of the application interface.

AMS is used to launch specific applications according to user operations. For example, after the compositing operation of the image is completed, the image is triggered to be displayed on the main key on the screen. After the image is displayed, it is triggered to perform the cutout operation on the image that needs to be cut out, and the application stack corresponding to the video application is created, so that Video applications work fine.

The system library layer can include multiple functional modules, such as: sensor module (also known as sensor) and SurfaceFlinger.

The sensor module is used to obtain the data collected by the sensor, such as collecting the ambient light under the screen. Collect gravity direction information of electronic equipment. Alternatively, the sensor module can also adjust the brightness of the screen according to the ambient light, and determine the horizontal and vertical screen state information of the electronic device according to the gravity direction information of the electronic device, and the horizontal and vertical screen state information is used to indicate whether the electronic device is in a landscape or portrait state.

Surfaceflinger is a system service for functions such as layer creation, control, and management.

In addition, the system library layer may also include: a surface manager (surface manager), a media library (MediaLibraries), a 3D graphics processing library (such as: OpenGL ES), a 2D graphics engine (such as: SGL), and so on. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc. 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. In the embodiment of the present application, the kernel layer includes at least a touch driver module and a display driver module.

The display driver module is used for displaying the synthesized image on the screen according to the image data provided by the module of the application framework layer and the application program of the application layer. For example, the video application transmits one frame of image data of the video to the display driver module, and the display driver module displays one frame of image in the video on the touch screen according to the image data. The SystemUI transmits the image data to the display driver module, and the display driver module displays the synthesized image on the screen.

The touch driver module is used to monitor the capacitance data of each area of the touch screen. When the user clicks or slides on the touch screen, the capacitance value of the clicked or slided area will change, and the touch driver module can monitor the change of the capacitance value of each area on the touch screen, and send a message of capacitance value change to the input management module The capacitance value data change message carries information such as the magnitude of change of the capacitance value of each area of the touch screen and the time when the change occurs.

The input management module can determine the touch operation according to the reported capacitance value change message, and then send the recognized touch operation to other modules. The touch operation here may include a knuckle tap operation, a click operation, a drag operation, and a specific gesture operation (such as an upward slide gesture operation, a horizontal slide gesture operation, etc.).

The hardware layer includes a screen and an ambient light sensor, etc., and the ambient light sensor is used to detect ambient light information under the screen, etc. The application processor monitors the touch operation on the touch screen. The baseband processor monitors the acceleration data and stores the monitored acceleration data in the storage module. When the AP detects a touch operation, the CP identifies the touch operation according to the acceleration data stored in the storage module. Whether the control operation is a knuckle tapping action; the CP sends the recognition result to the AP.

The technical architecture above lists the modules and devices that may be involved in the present application in the electronic equipment. In practical applications, electronic equipment may include all or part of the modules and devices of the above-mentioned technical architecture, as well as other modules and devices not mentioned in the above-mentioned technical architecture. Of course, it may also include only the modules and devices of the above-mentioned technical architecture. This embodiment There is no limit to this.

The embodiment of the present application also provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

An embodiment of the present application provides a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

This application implements all or part of the processes in the methods of the above embodiments, which can be completed by instructing related hardware through computer programs. The computer program can be stored in a computer-readable storage medium, and the computer program can be executed by a processor. , the steps in each of the foregoing method embodiments may be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program codes to a photographing device/electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (random access memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media. Such as U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunication signals under legislation and patent practice.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed methods and electronic devices may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other Presence or addition of features, wholes, steps, operations, elements, components and/or collections thereof.

It should also be understood that the term "and/or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

Reference to "one embodiment" or "some embodiments" or the like in the specification of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless specifically stated otherwise.

The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still implement the foregoing embodiments Modifications to the technical solutions described in the examples, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application, and should be included in the Within the protection scope of this application.

Claims (10)

1. A recognition method for a touch operation, characterized in that it is applied to an electronic device, the electronic device comprising: a touch screen, an application processor AP and a baseband processor CP, the method comprising: The AP monitors whether there is a touch operation on the touch screen; The CP monitors acceleration data, and stores the monitored acceleration data in a storage module; When the AP detects a touch operation, the CP identifies whether the touch operation is a knuckle tapping action according to the acceleration data stored in the storage module, and sends the identification result to the AP. 2. The method according to claim 1, further comprising: Acquire the capacitance value of the touch screen when the AP detects a touch operation; The AP identifies whether the touch operation is a knuckle tapping action according to the capacitance value; When the recognition result obtained by the AP according to the capacitance value and the recognition result obtained by the CP according to the acceleration data are both knuckle tapping actions, it is determined that the touch operation is a knuckle tapping action. 3. The method according to claim 2, further comprising: When the recognition result obtained by the AP according to the capacitance value and the recognition result obtained by the CP according to the acceleration data are not both knuckle tapping actions, it is determined that the touch operation is a click operation. 4. The method according to any one of claims 1 to 3, characterized in that, The storage module is a storage module whose power consumption is lower than that of a double-rate synchronous dynamic random access memory (DDR). 5. A touch operation method, characterized in that it is applied to an electronic device, the electronic device comprising: a touch screen, an application processor AP and a baseband processor CP, the method comprising: The AP and the CP use the method according to any one of claims 1 to 4 to identify the type of touch operation; In response to the touch operation, the AP performs a preset operation matching the type of the touch operation. 6. The method of claim 5, wherein, When it is determined that the touch operation includes a knuckle tapping action, the preset operation includes: any one of screen capture, area screen capture, or screen recording. 7. An identification device for touch operation, characterized in that it is applied to electronic equipment, and the electronic equipment includes: a touch screen, and the identification device includes: an application processor AP and a baseband processor CP, The AP is used to monitor whether there is a touch operation on the touch screen; The CP is configured to monitor acceleration data, store the monitored acceleration data in a storage module, and identify the touch operation according to the acceleration data stored in the storage module when the AP detects a touch operation. Check whether the control operation is a knuckle tapping action, and send the recognition result to the AP. 8. A touch operation device, characterized in that it is applied to electronic equipment, and the electronic equipment includes: a touch screen, and the touch operation device includes: a response module and the touch operation device according to claim 7 identification device; The response module is configured to execute a preset operation matching the touch operation in response to the touch operation when the recognition device determines that the touch operation includes a knuckle tapping action. 9. An electronic device, characterized in that it comprises: a touch screen, an application processor AP, a baseband processor CP, and a memory storing instructions, and when the instructions are executed by the AP and the CP, the The electronic device executes the method according to any one of claims 1 to 4 or the method according to any one of claims 5 to 6. 10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions, and when the instructions are executed by an electronic device, the electronic device executes any one of claims 1-4. The method described in item or the method described in any one of 5 to 6.

Images