WO2020019355A1

WO2020019355A1 - Touch control method for wearable device, and wearable device and system

Info

Publication number: WO2020019355A1
Application number: PCT/CN2018/097675
Authority: WO
Inventors: 龚树强; 龚建勇; 仇存收
Original assignee: 华为技术有限公司
Priority date: 2018-07-27
Filing date: 2018-07-27
Publication date: 2020-01-30
Also published as: CN112334860A; CN112334860B

Abstract

Disclosed are a touch control method for a wearable device, and a wearable device and system, wherein same relate to the technical field of communications, can improve the sensitivity and accuracy of a wearable device when a user gesture is recognized, and can reduce the probability of a wearable device or terminal being incorrectly triggered. The method comprises: a wearable device using a fingerprint sensor to detect a touch operation input by a user; the wearable device determining whether the touch operation includes the input of a fingerprint; if the touch operation includes the input of a fingerprint, the wearable device recognizing a control gesture corresponding to the touch operation; and the wearable device sending the control gesture to a terminal; alternatively, the wearable device sending an operation instruction corresponding to the control gesture to the terminal, so that the terminal performs an operation instruction corresponding to the control gesture, wherein a communication connection is established between the wearable device and the terminal.

Description

Touch method, wearable device and system of wearable device

Technical field

The present application relates to the field of communications technologies, and in particular, to a touch method, a wearable device, and a system for a wearable device.

Background technique

At present, mobile phones, tablets and other terminals support access to accessories such as headphones. Taking a mobile phone and a Bluetooth headset as an example, after a Bluetooth connection is established between the mobile phone and the Bluetooth headset, the user can use the Bluetooth headset to play songs in the mobile phone, talk to contacts, and so on.

Generally, a Bluetooth headset is provided with one or more function keys (such as volume +, volume-and other keys), and the user can control the mobile phone to implement functions related to audio playback by operating these function keys. Some Bluetooth headsets are also equipped with a touchpad, and users can implement the functions of the corresponding function keys by performing preset gestures (for example, clicking, sliding, etc.) on the touchpad. For example, if it is detected that the user has performed a click operation on the touchpad of the Bluetooth headset, the Bluetooth headset may generate a playback instruction corresponding to the click operation, and send the playback instruction to the mobile phone, so that the mobile phone performs the playback function in response to the playback instruction .

However, due to the small size and size of the Bluetooth headset, the area of the touchpad provided on the Bluetooth headset is correspondingly small. In this way, a sensing unit (such as a sensing capacitor) in the touchpad for recognizing user gestures The number of Bluetooth headsets is also relatively small, making it impossible for Bluetooth headsets to improve the sensitivity and accuracy when using these limited sensing units to recognize user gestures.

Summary of the Invention

The present application provides a touch method, a wearable device, and a system for a wearable device, which can improve the sensitivity and accuracy of the wearable device when recognizing a user's gesture, and reduce the chance of the wearable device or terminal being triggered by mistake.

In order to achieve the above purpose, this application uses the following technical solutions:

In a first aspect, the present application provides a touch method for a wearable device. The wearable device is provided with a fingerprint sensor. Then, the wearable device uses the fingerprint sensor to detect a touch operation input by the user; further, the wearable device can determine the touch operation. Whether the touch operation includes fingerprint input; if the touch operation includes fingerprint input, it means that the touch operation is not a user's accidental touch behavior, so the wearable device can recognize the control gesture corresponding to the touch operation; and send the control gesture to A terminal; or, sending an operation instruction corresponding to the control gesture to the terminal, so that the terminal executes the operation instruction corresponding to the control gesture, and a communication connection is established between the wearable device and the terminal.

That is to say, in the embodiment of the present application, the fingerprint sensor has a small size and a high degree of integration, and the fingerprint sensor is set in a wearable device to identify a user's touch operation, instead of using the size in a traditional wearable device. The larger touchpad recognizes gestures performed by the user, thereby improving the integration of the wearable device. At the same time, because the number of sensing units in the fingerprint sensor is greater, and the fingerprint sensor can identify whether the user's touch operation is triggered by a finger instead of being accidentally touched, the sensitivity and accuracy of the wearable device when recognizing the user's gesture will be Increase, while reducing the chance of wearable devices and terminals being triggered by mistake.

In a possible design method, after the wearable device uses the fingerprint sensor to detect a touch operation input by a user, the method further includes: in response to the touch operation, the wearable device collects N consecutive images formed on the fingerprint sensor, At least one of the N consecutive images includes a fingerprint pattern, and N is an integer greater than 1. At this time, the wearable device recognizes the control gesture corresponding to the touch operation, including: the wearable device continues to the N consecutive images according to the fingerprint pattern The change in the image identifies the control gesture corresponding to the touch operation.

In a possible design method, the wearable device collects N consecutive images formed on the fingerprint sensor, including: when the fingerprint sensor detects that the user's finger is in contact with the fingerprint sensor, it starts to collect the formed images on the fingerprint sensor at a preset frequency. Image; when the fingerprint sensor detects that the user's finger leaves the fingerprint sensor, it stops collecting images formed on the fingerprint sensor to obtain the N consecutive images.

In a possible design method, the wearable device collects N consecutive images formed on the fingerprint sensor, including: when the fingerprint sensor detects that the user's finger is in contact with the fingerprint sensor, it starts to collect the formed images on the fingerprint sensor at a preset frequency. Image; when the fingerprint sensor detects that the user ’s finger leaves the fingerprint sensor, it continues to collect the image formed on the fingerprint sensor within a preset time; if it does not detect that the user's finger has touched the fingerprint sensor within the preset time, it stops collecting the fingerprint sensor On the formed image, the N consecutive images are obtained.

In a possible design method, the wearable device recognizes the control gesture corresponding to the touch operation according to the change of the fingerprint pattern in the N consecutive images, and specifically includes: the wearable device according to a preset fingerprint characteristic, An image including a fingerprint pattern in N consecutive images is identified; further, the wearable device recognizes a control gesture corresponding to the touch operation according to a size change and / or a position change of the fingerprint pattern in the N consecutive images. That is, the principle that a fingerprint sensor can collect a fingerprint pattern is used in this application to recognize a control gesture input by a user through continuous changes of the fingerprint pattern.

In a possible design method, the wearable device recognizes the control gesture corresponding to the touch operation according to the size change and / or position change of the fingerprint pattern in the N consecutive images, including: when the N consecutive images There are consecutive X images containing the fingerprint pattern, and the position of the fingerprint pattern in the X images is the same, then the control gesture corresponding to the touch operation is a click operation, X≤N; or, when the N consecutive images are in There are consecutive Y images containing the fingerprint pattern, and the position of the fingerprint pattern in the Y images is the same, then the control gesture corresponding to the touch operation is a long press operation, X <Y ≦ N; or, when the N consecutive images There are consecutive Z images containing the fingerprint pattern, and the displacement amount of the fingerprint pattern in the Z images is greater than the distance threshold, then the control gesture corresponding to the touch operation is a sliding operation, Z ≦ N, or, when the N consecutive images Between L1 consecutive images containing the fingerprint pattern and L2 consecutive images containing the fingerprint pattern, there are L3 images not containing the fingerprint pattern, the control gesture corresponding to the touch operation is a double-tap operation Operation, L3 is less than a preset threshold, 1 <L1 + L2 + L3 ≦ N.

In a possible design method, before the wearable device sends a control gesture to the terminal; or before the wearable device sends an operation instruction corresponding to the control gesture to the terminal, the method further includes: the wearable device determines that the touch operation is not Incorrect operation. That is, in this application, only when the wearable device recognizes that the touch operation received by the fingerprint sensor includes fingerprint input, it will continue to collect the image formed on the fingerprint sensor and identify the control gesture corresponding to the touch operation. And sending the identified control gesture or operation instruction to the terminal.

In a possible design method, the wearable device determines that the above-mentioned touch operation is not an accidental touch operation, including: if P1 images in the N consecutive images contain a fingerprint pattern, and P1 is greater than a preset value, the wearable device determines the The touch operation is not an accidental touch operation.

In a possible design method, if P2 images in the N consecutive images contain a fingerprint pattern, and P2 is less than the preset value, the wearable device determines that the touch operation is a false touch operation; then, the wearable device will The fingerprint sensor is switched from the working state to the sleep state, thereby avoiding the user's accidental touch operation on the wearable device to wake up the terminal, and reducing the power consumption of the wearable device and the mobile phone.

In a possible design method, before the wearable device receives a touch operation input by the user to the fingerprint sensor, the method further includes: in response to a wakeup operation input by the user, the wearable device switches the fingerprint sensor from a sleep state to a working state. That is, the fingerprint sensor may be in a sleep state before receiving a touch operation input by a user to reduce power consumption of the wearable device.

In a second aspect, the present application provides a wearable device, including: a fingerprint sensor, one or more processors, a memory, and one or more programs; wherein the processor is coupled to the memory, and the one or more programs are stored In the memory, when the wearable device is running, the processor executes one or more programs stored in the memory, so that the wearable device executes any one of the touch methods of the wearable device.

The fingerprint sensor may be disposed on a side that is not in contact with the user when the wearable device is worn; the wearable device may be a Bluetooth headset, smart glasses, or a smart watch.

According to a third aspect, the present application provides a computer storage medium including computer instructions, and when the computer instructions are run on the wearable device, the wearable device executes the touch method of the wearable device according to any one of the foregoing.

In a fourth aspect, the present application provides a computer program product that, when the computer program product runs on the wearable device, causes the wearable device to perform the touch method of the wearable device according to any one of the above.

In a fifth aspect, the present application provides a touch control system including a wearable device and a terminal, wherein the wearable device is provided with a fingerprint sensor, and a communication connection is established between the wearable device and the terminal; The wearable device is configured to: use the fingerprint sensor to detect a touch operation input by a user; determine whether the touch operation includes a fingerprint input; if the touch operation includes a fingerprint input, identify that the touch operation corresponds to Sending the control gesture to the terminal, or sending an operation instruction corresponding to the control gesture to the terminal; the terminal is configured to: receive the control gesture sent by the wearable device Or receive an operation instruction corresponding to the control gesture sent by the wearable device; and execute an operation instruction corresponding to the control gesture.

According to a sixth aspect, the present application provides a touch system including a wearable device and a terminal, wherein the wearable device is provided with a fingerprint sensor, and a communication connection is established between the wearable device and the terminal; The wearable device is configured to: use the fingerprint sensor to detect a touch operation input by a user; in response to the touch operation, collect N consecutive images formed on the fingerprint sensor, at least one of the N consecutive images The images include a fingerprint pattern, and N is an integer greater than 1. The N consecutive images are sent to the terminal; the terminal is configured to: receive N consecutive images sent by the wearable device; The change in the N consecutive images identifies a control gesture corresponding to the touch operation; and executes an operation instruction corresponding to the control gesture.

Understandably, the terminal described in the second aspect, the computer storage medium described in the third aspect, the computer program product described in the fourth aspect, and the systems described in the fifth and sixth aspects are all used for The corresponding method provided above is executed, and therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and details are not described herein again.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a first schematic view of a touch scenario of a wearable device according to an embodiment of the present application; FIG.

2 is a schematic structural diagram of a fingerprint sensor according to an embodiment of the present application;

FIG. 3 is a first schematic structural diagram of a wearable device according to an embodiment of the present application; FIG.

4 is a schematic structural diagram of a smart glasses according to an embodiment of the present application;

5 is a schematic structural diagram of a terminal according to an embodiment of the present application;

6 is a schematic interaction diagram of a touch method of a wearable device according to an embodiment of the present application;

FIG. 7 is a second schematic diagram of a touch scenario of a wearable device according to an embodiment of the present application; FIG.

FIG. 8 is a third schematic view of a touch scenario of a wearable device according to an embodiment of the present application; FIG.

FIG. 9 is a fourth schematic view of a touch scenario of a wearable device according to an embodiment of the present application; FIG.

FIG. 10 is a second schematic structural diagram of a wearable device according to an embodiment of the present application.

detailed description

The embodiments of the embodiments of the present application will be described in detail below with reference to the drawings.

As shown in FIG. 1, a touch method of a wearable device provided in an embodiment of the present application can be applied to a touch system composed of a wearable device 11 and a terminal 12. A wireless communication connection or a wired communication connection may be established between the wearable device 11 and the terminal 12.

The wearable device 11 may be a wireless headset, a wired headset, smart glasses, a smart helmet, or a smart watch. The terminal 12 may be a device such as a mobile phone, a tablet computer, a notebook computer, an Ultra-mobile Personal Computer (UMPC), a Personal Digital Assistant (PDA), and the like in this embodiment of the present application.

Taking the wearable device 11 as an example of a Bluetooth headset, as shown in FIG. 1, a fingerprint sensor 201 is provided on the Bluetooth headset in the embodiment of the present application. The fingerprint sensor 201 may be disposed on a side that is not directly in contact with the user when the user wears it. For example, the fingerprint sensor 201 may be disposed on a housing of a Bluetooth headset, or the fingerprint sensor 201 may be separately configured as a control module and The case of the Bluetooth headset is connected.

When a user's finger touches the collection surface exposed in the above-mentioned fingerprint sensor 201, the fingerprint sensor 201 can collect a fingerprint pattern formed by the user's finger on the collection surface. Exemplarily, the fingerprint sensor 201 shown in FIG. 2 includes a plurality of sensing units 201b arranged in an array, and a collection surface 201a covering the sensing units 201b. Fingerprints on a user's finger generally include concave troughs (valleys) and convex crests (ridges). After the user's finger contacts the collection surface 201a of the fingerprint sensor 201, since the human body is a conductive medium, the sensing unit 201b in the fingerprint sensor 201 can generate electrical signals corresponding to the valley and the ridge, respectively. Taking the sensing unit 201b as an example of the sensing capacitor, the capacitance difference generated by the sensing capacitor corresponding to the valley in the fingerprint is the first capacitance difference, and the capacitance difference generated by the sensing capacitor corresponding to the peak in the fingerprint is the second capacitance difference. The capacitance difference at different positions on 201 can draw the user's fingerprint pattern.

In addition, if the fingerprint sensor 201 is an optical fingerprint sensor, the sensing unit 201b may be a photoelectric sensor (such as a photodiode or a phototransistor). Of course, the above-mentioned fingerprint sensor 201 may be a capacitive fingerprint sensor, an optical fingerprint sensor, a radio frequency fingerprint sensor, or an ultrasonic fingerprint sensor, and the embodiment of the present application does not place any restrictions on this.

It can be understood that when the number of the sensing units 201b is larger, the accuracy and sensitivity when collecting and identifying the fingerprint pattern of the user are higher. Due to the high integration of the sensing unit 201b in the fingerprint sensor 201, for a fingerprint sensor 201 and an ordinary touchpad of the same size, the number of the sensing units 201b in the fingerprint sensor 201 is much larger than that of the ordinary touchpad. Number of. For example, a 2cm * 2cm fingerprint sensor 201 may include more than 50 * 50 sensing units, while the number of sensing units in a 2cm * 2cm ordinary touchpad may be only about a dozen.

In the embodiment of the present application, the above-mentioned fingerprint sensor 201 may be used instead of the ordinary touchpad originally provided in the wearable device to reduce the size of the wearable device. At the same time, due to the large number of sensing units in the fingerprint sensor 201, the user's fingerprint pattern collected is clear and accurate. Therefore, the wearable device can recognize the user's finger in the fingerprint through the multiple fingerprint patterns continuously collected by the fingerprint sensor 201. Specific gestures performed by the sensor 201, such as a slide gesture, a double-tap gesture, and the like.

In this way, the wearable device may determine an operation instruction corresponding to the recognized specific gesture according to a preset correspondence relationship between different gestures and different operation instructions, such as a play instruction, a volume adjustment instruction, or a pause instruction. Furthermore, the wearable device may send a corresponding operation instruction to the terminal, so that the terminal executes the operation instruction, so as to implement the wearable device to control the related functions in the terminal through the user's touch operation on the wearable device.

Of course, after the wearable device recognizes the specific gesture performed by the user using the fingerprint pattern collected by the fingerprint sensor 201, the wearable device may also send the recognized gesture to the terminal, and the terminal executes a corresponding operation instruction according to the gesture. Without any restrictions.

Further, as shown in FIG. 3, in addition to the above-mentioned fingerprint sensor 201, the wearable device 11 may further include a microphone 201 (for example, a bone conduction microphone), an acceleration sensor 203, a proximity light sensor 204, a communication module 205, a speaker 206, The computing module 207, the storage module 208, and the power supply 209 and other components. It can be understood that the above-mentioned wearable device 11 may have more or fewer components than those shown in FIG. 3, may combine two or more components, or may have different component configurations. The various components shown in FIG. 3 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing or application specific integrated circuits.

It should be noted that in the above embodiments, the Bluetooth headset is used as an example for the wearable device 11. It can be understood that the fingerprint sensor 201 can also be set on other wearable devices such as smart glasses, smart helmets or smart bracelets. Is used to recognize a gesture performed by the user on the fingerprint sensor 201.

For example, as shown in FIG. 4, the above-mentioned fingerprint sensor 201 may be integrated in the smart glasses 301. For example, the fingerprint sensor 201 may be disposed on a frame or temple of the smart glasses 301. When a user's finger touches the fingerprint sensor 201 on the smart glasses 301, the fingerprint sensor 201 can collect a fingerprint pattern on the fingerprint sensor 201 at a certain frequency. In this way, the smart glasses 301 can recognize specific gestures performed by the user on the fingerprint sensor 201 according to changes in the position and size of the user's finger in the collected multiple fingerprint patterns.

As shown in FIG. 5, the terminal 12 in the voice control system may be a mobile phone 100. The mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a radio frequency module 150, a communication module 160, and an audio module. 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display 194, and SIM card interface 195. The sensor module can include pressure sensor 180A, gyroscope sensor 180B, barometric pressure sensor 180C, magnetic sensor 180D, acceleration sensor 180E, distance sensor 180F, proximity light sensor 180G, fingerprint sensor 180H, temperature sensor 180J, touch sensor 180K, and ambient light sensor. 180L, bone conduction sensor, etc.

The structure illustrated in the embodiment of the present invention does not limit the mobile phone 100. It may include more or fewer parts than shown, or some parts may be combined, or some parts may be split, or different parts may be arranged. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU) Wait. Among them, different processing units can be independent devices or integrated in the same processor.

The controller may be a decision maker that instructs the various components of the mobile phone 100 to coordinate work according to instructions. It is the nerve center and command center of the mobile phone 100. The controller generates operation control signals according to the instruction operation code and timing signals, and completes the control of fetching and executing the instructions.

The processor 110 may further include a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. You can save instructions or data that the processor has just used or recycled. If the processor needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided, the processor's waiting time is reduced, and the efficiency of the system is improved.

In some embodiments, the processor 110 may include an interface. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit (inter-integrated circuit, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transceiver (universal asynchronous receiver / transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input / output (GPIO) interface, subscriber identity module (SIM) interface, And / or universal serial bus (universal serial bus, USB) interfaces.

The I2C interface is a two-way synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor may include multiple sets of I2C buses. The processor can be coupled to touch sensors, chargers, flashes, cameras, etc. through different I2C bus interfaces. For example, the processor may couple the touch sensor through the I2C interface, so that the processor and the touch sensor communicate through the I2C bus interface to implement the touch function of the mobile phone 100.

The I2S interface can be used for audio communication. In some embodiments, the processor may include multiple sets of I2S buses. The processor may be coupled to the audio module through an I2S bus to implement communication between the processor and the audio module. In some embodiments, the audio module can transmit audio signals to the communication module through the I2S interface, so as to implement the function of receiving calls through a Bluetooth headset.

The PCM interface can also be used for audio communications, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module and the communication module may be coupled through a PCM bus interface. In some embodiments, the audio module can also transmit audio signals to the communication module through the PCM interface, so as to implement the function of receiving calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication, and the sampling rates of the two interfaces are different.

The UART interface is a universal serial data bus for asynchronous communication. This bus is a two-way communication bus. It converts the data to be transferred between serial and parallel communications. In some embodiments, a UART interface is typically used to connect the processor and the communication module 160. For example, the processor communicates with the Bluetooth module through a UART interface to implement the Bluetooth function. In some embodiments, the audio module can transmit audio signals to the communication module through the UART interface, so as to implement the function of playing music through a Bluetooth headset.

The MIPI interface can be used to connect processors with peripheral devices such as displays, cameras, etc. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor and the camera communicate through a CSI interface to implement a shooting function of the mobile phone 100. The processor and the display communicate through a DSI interface to implement a display function of the mobile phone 100.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor with a camera, a display screen, a communication module, an audio module, a sensor, and the like. GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface can be used to connect a charger to charge the mobile phone 100, and can also be used to transfer data between the mobile phone 100 and peripheral devices. It can also be used to connect headphones and play audio through headphones. It can also be used to connect other electronic devices, such as AR devices.

The interface connection relationship between the modules shown in the embodiments of the present invention is only a schematic description, and does not constitute a limitation on the structure of the mobile phone 100. The mobile phone 100 may use different interface connection modes or a combination of multiple interface connection modes in the embodiments of the present invention.

The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module may receive a charging input of a wired charger through a USB interface. In some embodiments of wireless charging, the charging management module may receive a wireless charging input through a wireless charging coil of the mobile phone 100. While the charging management module is charging the battery, it can also supply power to the terminal device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charge management module 140 and the processor 110. The power management module receives inputs from the battery and / or charge management module, and supplies power to a processor, an internal memory, an external memory, a display screen, a camera, and a communication module. The power management module can also be used to monitor battery capacity, battery cycle times, battery health (leakage, impedance) and other parameters. In some embodiments, the power management module 141 may also be disposed in the processor 110. In some embodiments, the power management module 141 and the charge management module may also be provided in the same device.

The wireless communication function of the mobile phone 100 can be implemented by the antenna module 1, the antenna module 2 the radio frequency module 150, the communication module 160, the modem, and the baseband processor.

The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, a cellular network antenna can be multiplexed into a wireless LAN diversity antenna. In some embodiments, the antenna may be used in conjunction with a tuning switch.

The radio frequency module 150 may provide a communication processing module applied to the mobile phone 100 and including a wireless communication solution such as 2G / 3G / 4G / 5G. It may include at least one filter, switch, power amplifier, Low Noise Amplifier (LNA), and the like. The radio frequency module receives electromagnetic waves from the antenna 1, and processes the received electromagnetic waves by filtering, amplifying, etc., and transmitting them to the modem for demodulation. The radio frequency module can also amplify the signal modulated by the modem and turn it into electromagnetic wave radiation through the antenna 1. In some embodiments, at least part of the functional modules of the radio frequency module 150 may be disposed in the processor 150. In some embodiments, at least part of the functional modules of the radio frequency module 150 may be provided in the same device as at least part of the modules of the processor 110.

The modem may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs sound signals through audio equipment (not limited to speakers, receivers, etc.), or displays images or videos through a display screen. In some embodiments, the modem may be a separate device. In some embodiments, the modem may be independent of the processor and disposed in the same device as the radio frequency module or other functional modules.

The communication module 160 can provide wireless local area networks (WLAN), Bluetooth (Bluetooth, BT), global navigation satellite system (GNSS), frequency modulation (FM) applied to the mobile phone 100. , A communication processing module for a wireless communication solution such as near field communication (NFC), infrared technology (infrared, IR) and the like. The communication module 160 may be one or more devices that integrate at least one communication processing module. The communication module receives the electromagnetic wave through the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor. The communication module 160 may also receive a signal to be transmitted from the processor, frequency-modulate it, amplify it, and turn it into electromagnetic wave radiation through the antenna 2.

In some embodiments, the antenna 1 of the mobile phone 100 is coupled to a radio frequency module, and the antenna 2 is coupled to a communication module. The mobile phone 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology may include a global mobile communication system (GSM), a general packet radio service (GPRS), a code division multiple access (CDMA), and broadband. Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation navigation system (BDS), and a quasi-zenith satellite system (quasi -zenith satellite system (QZSS)) and / or satellite-based augmentation systems (SBAS).

The mobile phone 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display includes a display panel. The display panel can adopt LCD (liquid crystal display), OLED (organic light-emitting diode), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode) emitting diodes, AMOLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (QLEDs), etc. In some embodiments, the mobile phone 100 may include one or N display screens, where N is a positive integer greater than 1.

Still shown in FIG. 1, the mobile phone 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen, and an application processor.

ISP is used to process data from camera feedback. For example, when taking a picture, the shutter is opened, and the light is transmitted to the light receiving element of the camera through the lens. The light signal is converted into an electrical signal, and the light receiving element of the camera passes the electrical signal to the ISP for processing and converts the image to the naked eye. ISP can also optimize the image's noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, an ISP may be provided in the camera 193.

The camera 193 is used to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs digital image signals to the DSP for processing. DSP converts digital image signals into image signals in standard RGB, YUV and other formats. In some embodiments, the mobile phone 100 may include one or N cameras, where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals. In addition to digital image signals, it can also process other digital signals. For example, when the mobile phone 100 is selected at a frequency point, the digital signal processor is used to perform a Fourier transform on the frequency point energy.

Video codecs are used to compress or decompress digital video. The mobile phone 100 may support one or more codecs. In this way, the mobile phone 100 can play or record videos in multiple encoding formats, such as: MPEG1, MPEG2, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of a biological neural network, such as the transfer mode between neurons in the human brain, the NPU can quickly process input information and continuously learn by itself. Through the NPU, applications such as smart cognition of the mobile phone 100 can be implemented, such as: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor through an external memory interface to implement a data storage function. For example, save music, videos and other files on an external memory card.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running instructions stored in the internal memory 121. The memory 121 may include a storage program area and a storage data area. The storage program area may store an operating system, at least one application required by a function (such as a sound playback function, an image playback function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone 100. In addition, the memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, other volatile solid-state storage devices, a universal flash memory (universal flash storage, UFS), etc. .

The mobile phone 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, and an application processor. Such as music playback, recording, etc.

The audio module is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module can also be used to encode and decode audio signals. In some embodiments, the audio module may be disposed in the processor 110, or some functional modules of the audio module may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals. The mobile phone 100 can listen to music through a speaker or listen to a hands-free call.

The receiver 170B, also referred to as the "handset", is used to convert audio electrical signals into sound signals. When the mobile phone 100 answers a call or a voice message, it can answer the voice by holding the receiver close to the human ear.

The microphone 170C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound through the mouth close to the microphone, and input the sound signal into the microphone. The mobile phone 100 may be provided with at least one microphone. In some embodiments, the mobile phone 100 may be provided with two microphones, in addition to collecting sound signals, it may also implement a noise reduction function. In some embodiments, the mobile phone 100 may further be provided with three, four, or more microphones to collect sound signals, reduce noise, and also identify sound sources, and implement a directional recording function.

The headset interface 170D is used to connect a wired headset. The earphone interface can be a USB interface or a 3.5mm open mobile terminal platform (OMTP) standard interface, and the American Cellular Telecommunications Industry Association (United States of America, CTIA) standard interface.

The pressure sensor 180A is used to sense a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor may be disposed on the display screen. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The capacitive pressure sensor may be at least two parallel plates having a conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The mobile phone 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen, the mobile phone 100 detects the intensity of the touch operation according to a pressure sensor. The mobile phone 100 may also calculate the touched position according to the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity lower than the first pressure threshold is applied to the short message application icon, an instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction for creating a short message is executed.

The gyro sensor 180B may be used to determine the movement posture of the mobile phone 100. In some embodiments, the angular velocity of the mobile phone 100 about three axes (ie, x, y, and z axes) may be determined by a gyro sensor. A gyroscope sensor can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor detects the shake angle of the mobile phone 100, and calculates the distance that the lens module needs to compensate according to the angle, so that the lens can cancel the shake of the mobile phone 100 by the reverse movement to achieve anti-shake. The gyroscope sensor can also be used for navigation and somatosensory game scenes.

The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the mobile phone 100 calculates altitude by using the air pressure value measured by the air pressure sensor to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The mobile phone 100 can detect the opening and closing of the flip leather case by using a magnetic sensor. In some embodiments, when the mobile phone 100 is a flip machine, the mobile phone 100 can detect the opening and closing of the flip according to a magnetic sensor. Further, according to the opened and closed state of the holster or the opened and closed state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the mobile phone 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the mobile phone 100 is stationary. It can also be used to identify the posture of the terminal, and is used in applications such as switching between horizontal and vertical screens, and pedometers.

Distance sensor 180F for measuring distance. The mobile phone 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the mobile phone 100 may use a distance sensor to measure distances to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. Infrared light is emitted outward through a light emitting diode. Use photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the mobile phone 100. When insufficient reflected light is detected, it can be determined that there is no object near the mobile phone 100. The mobile phone 100 can use a proximity light sensor to detect that the user is holding the mobile phone 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor can also be used in holster mode, and the pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The mobile phone 100 can adaptively adjust the brightness of the display screen according to the perceived ambient light brightness. The ambient light sensor can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor can also cooperate with the proximity light sensor to detect whether the mobile phone 100 is in a pocket to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The mobile phone 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application lock, fingerprint photographing, fingerprint answering calls, etc.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the mobile phone 100 uses the temperature detected by the temperature sensor to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor exceeds a threshold, the mobile phone 100 performs a performance reduction of a processor located near the temperature sensor in order to reduce power consumption and implement thermal protection.

The touch sensor 180K is also called "touch panel". Can be set on the display. Used to detect touch operations on or near it. The detected touch operation can be passed to the application processor to determine the type of touch event and provide corresponding visual output through the display screen.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor can acquire a vibration signal of a human body acoustical vibration bone mass. Bone conduction sensors can also touch the human pulse and receive blood pressure beating signals. In some embodiments, a bone conduction sensor may also be provided in the headset. The audio module 170 may analyze a voice signal based on a vibration signal of a oscillating bone mass obtained by the bone conduction sensor to implement a voice function. The application processor may analyze the heart rate information based on the blood pressure beating signal obtained by the bone conduction sensor to implement a heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys can be mechanical keys. It can also be a touch button. The mobile phone 100 receives key input, and generates key signal inputs related to user settings and function control of the mobile phone 100.

The motor 191 may generate a vibration alert. The motor can be used for incoming vibration alert and touch vibration feedback. For example, the touch operation applied to different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations on different areas of the display can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, receiving information, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, and can also be used to indicate messages, missed calls, notifications, and so on.

The SIM card interface 195 is used to connect to a subscriber identity module (SIM). The SIM card can be contacted and separated from the mobile phone 100 by inserting or removing the SIM card interface. The mobile phone 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple SIM cards can be inserted into the same SIM card interface at the same time. The types of the multiple cards may be the same or different. The SIM card interface is also compatible with different types of SIM cards. The SIM card interface is also compatible with external memory cards. The mobile phone 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the mobile phone 100 uses an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the mobile phone 100 and cannot be separated from the mobile phone 100.

With reference to FIG. 1 to FIG. 5 described above, in the embodiment of the present application, when the wearable device 11 detects a touch operation on the fingerprint sensor 201, N (N) formed on the fingerprint sensor 201 can be collected by the fingerprint sensor 201 at a certain frequency. > 1) continuous images. Furthermore, the wearable device 11 can recognize gestures performed by the user on the fingerprint sensor 201 by comparing changes in parameters such as the size and position of the fingerprint pattern in the N consecutive images. In this way, the wearable device 11 can send a corresponding operation instruction to the terminal 12 according to the recognized gesture, so as to implement the wearable device 11 to control related functions in the terminal 12.

That is to say, in the embodiment of the present application, the fingerprint sensor has a small size and a high degree of integration, and the fingerprint sensor is set in a wearable device to recognize a gesture performed by a user, instead of using a size in a traditional wearable device. The larger touchpad recognizes gestures performed by the user, thereby improving the integration of the wearable device. At the same time, because the number of sensing units in the fingerprint sensor is greater, the sensitivity and accuracy of the wearable device when recognizing a user's gesture will be improved, thereby reducing the chance of the wearable device and terminal being accidentally triggered.

In order to facilitate understanding, a touch method of a wearable device provided by an embodiment of the present application will be specifically introduced below with reference to the accompanying drawings. In the following embodiments, a mobile phone is used as a terminal, and a Bluetooth headset is used as a wearable device.

FIG. 6 is a schematic flowchart of a touch method of a wearable device according to an embodiment of the present application. As shown in FIG. 6, the touch method may include:

S601. The mobile phone establishes a Bluetooth connection with the Bluetooth headset.

When the user wants to use a Bluetooth headset, the Bluetooth function of the Bluetooth headset can be turned on. At this time, the Bluetooth headset can send a paired broadcast to the outside. If the mobile phone has the Bluetooth function turned on, the mobile phone can receive the pairing broadcast and prompt the user that the relevant Bluetooth device has been scanned. When the user selects a Bluetooth headset on the phone, the phone can pair with the Bluetooth headset and establish a Bluetooth connection. Subsequently, the mobile phone and the Bluetooth headset can communicate through the Bluetooth connection. Of course, if the mobile phone and the Bluetooth headset have been successfully paired before establishing this Bluetooth connection, the mobile phone can automatically establish a Bluetooth connection with the scanned Bluetooth headset.

In addition, if the headset used by the user has Wi-Fi function, the user can also operate the mobile phone to establish a Wi-Fi connection with the headset. Or, if the earphone used by the user is a wired earphone, the user can also insert the plug of the earphone cable into the corresponding earphone interface of the mobile phone to establish a wired connection, which is not limited in this embodiment of the present application.

When the mobile phone establishes a Bluetooth connection with the Bluetooth headset, the mobile phone can also use the Bluetooth headset connected at this time as a legitimate Bluetooth device. For example, the mobile phone may save the identification of the legal Bluetooth device (such as the MAC address of a Bluetooth headset, etc.) locally on the mobile phone. In this way, when a subsequent mobile phone receives an operation instruction or data from a Bluetooth device, the mobile phone can determine whether the Bluetooth device communicating at this time is a legitimate Bluetooth device according to the saved identifier of the legal Bluetooth device. When the mobile phone determines that an illegal Bluetooth device is currently sending an operation instruction or data to the mobile phone, the mobile phone may discard the operation instruction or data to improve the security during the use of the mobile phone. Of course, a phone can manage one or more legitimate Bluetooth devices. As shown in FIG. 7, the user can enter the management interface 701 of legal devices from the setting function, and the user can add or delete legal Bluetooth devices in the management interface 701.

S602 (optional): In response to a preset operation input by the user to the Bluetooth headset, the Bluetooth headset wakes up the fingerprint sensor and enters a working state.

In order to reduce the power consumption of the Bluetooth headset, after the Bluetooth headset is started, the fingerprint sensor on the Bluetooth headset is set to a sleep state with low power consumption by default. In the sleep state, the Bluetooth headset can scan the electrical signals generated by each sensing unit in the fingerprint sensor at a lower working frequency, or the Bluetooth headset can also temporarily turn off the fingerprint sensor (for example, power off the fingerprint sensor).

In addition, the Bluetooth headset can preset one or more preset operations for awakening the fingerprint sensor. For example, a Bluetooth headset may be preset with a wake-up word (eg, "Hello, Little E"). When the Bluetooth headset detects that the wake-up word is included in the voice information input by the user through the microphone, it indicates that the user has performed a preset operation for waking the fingerprint sensor. At this time, the Bluetooth headset can switch the fingerprint sensor from a sleep state to a working state. Alternatively, the Bluetooth headset may be preset with a tapping operation (eg, double tapping) to wake up the fingerprint sensor. When the Bluetooth headset detects that the user has performed the tapping operation through the acceleration sensor, the Bluetooth headset can switch the fingerprint sensor from a sleep state to a working state. Alternatively, or alternatively, the Bluetooth headset may be preset with a touch operation (for example, a tap operation) to wake up the fingerprint sensor. If the fingerprint sensor in the sleep state detects that the user has performed the touch operation, it can switch from the sleep state to the working state. After the fingerprint sensor enters the working state, each sensing unit in the fingerprint sensor can be scanned at a higher working frequency (for example, 10 Hz) to collect an image formed on the fingerprint sensor.

Of course, after the Bluetooth headset is started, the fingerprint sensor of the Bluetooth headset may be set to a working state by default. At this time, the Bluetooth headset may skip step S602 and perform the following steps S603-S606.

In other embodiments of the present application, after the Bluetooth headset is started, if no user operation is detected on the Bluetooth headset within a preset time, the Bluetooth headset may also automatically enter a sleep state. For example, a Bluetooth headset can enter a BLE (Bluetooth Low Energy) mode, thereby further reducing the power consumption of the Bluetooth headset. When the Bluetooth headset enters the sleep state, some sensors (such as the acceleration sensor or the microphone described above) can be reserved to work at a lower frequency. After receiving the above-mentioned preset operation input by the user to the Bluetooth headset, the Bluetooth headset can be switched from the sleep state The operation mode is the following steps S603-S606.

S603. The Bluetooth headset collects N consecutive images formed on the fingerprint sensor. At least one of the N consecutive images includes a fingerprint pattern, and N is an integer greater than 1.

After the above fingerprint sensor enters the working state, the Bluetooth headset can use the fingerprint sensor to continuously acquire N consecutive images formed on the collection surface of the fingerprint sensor at a certain working frequency. Since the user's finger is a conductive object, when the user touches and leaves the fingerprint sensor, the corresponding capacitance signal in the fingerprint sensor changes. Therefore, after the fingerprint sensor enters the working state, the motion of the user's finger touching and leaving the fingerprint sensor can be sensed. Then, the Bluetooth headset can continuously acquire the image formed on the fingerprint sensor from the fingerprint sensor sensing that the user's finger touches the fingerprint sensor, until the fingerprint sensor senses that the user's finger leaves the fingerprint sensor, thereby obtaining the above-mentioned N consecutive images.

Or, because the user may not always touch the fingerprint sensor when inputting gestures on the fingerprint sensor (for example, when the user performs a double-click operation, the user's finger will leave the fingerprint sensor between two click operations for a short time), so when the fingerprint sensor senses After the user's finger leaves the fingerprint sensor, the user can continue to work for a certain period of time (for example, 2 seconds), that is, the image formed on the fingerprint sensor can be continuously collected within these 2 seconds. If the user's finger is not touched by the fingerprint sensor within 2 seconds, it means that the gesture input by the user has ended, the Bluetooth headset can control the fingerprint sensor to enter the sleep state again to reduce the power consumption of the Bluetooth headset. At this time, the above-mentioned N consecutive images refer to the images continuously acquired 2 seconds after the fingerprint sensor detects that the user's finger touches the fingerprint sensor, until the fingerprint sensor detects that the user's finger leaves the fingerprint sensor. Other images including fingerprint patterns.

Alternatively, the Bluetooth headset can also start to continuously capture images formed on the fingerprint sensor after the fingerprint sensor enters the working state, until the fingerprint sensor senses that the user's finger leaves the fingerprint sensor or the fingerprint sensor senses that the user's finger leaves the fingerprint sensor. Up to time, the above N continuous images are obtained. Since the user may not touch the fingerprint sensor immediately after the fingerprint sensor enters the working state, in this way, there may be other images not including the fingerprint pattern in the first few images of the above N consecutive images.

Further, since the fingerprint patterns of the fingers of ordinary users are composed of wave valleys and wave peaks with a certain regular pattern, the Bluetooth headset can learn fingerprint characteristics of general fingerprints through some samples of fingerprint patterns in advance. The fingerprint feature can be stored in the Bluetooth headset in the form of a model or a vector. The fingerprint feature can be used to indicate the fingerprint feature of a specific user (such as legal user A), and can also be used to indicate the fingerprint feature common to the fingerprints of most ordinary users. Of course, the above-mentioned fingerprint characteristics may also be obtained by the Bluetooth headset from other devices (such as a mobile phone or a cloud server), and this embodiment of the present application does not place any restrictions on this.

In this way, the Bluetooth headset can identify whether the image has the above-mentioned fingerprint characteristics in one or more images collected by the fingerprint sensor, thereby determining whether the current touch operation on the fingerprint sensor is a false touch operation. If the Bluetooth headset determines that the touch operation on the fingerprint sensor is not an accidental touch operation, the Bluetooth headset may continue to perform the following steps S604-S606; otherwise, the Bluetooth headset may discard the captured image and enter the sleep state again to Reduce the power consumption of Bluetooth headsets.

Exemplarily, a Bluetooth headset may start to collect images from a fingerprint sensor, and determine in real time whether each image captured by the fingerprint sensor has the above-mentioned fingerprint characteristics (that is, whether the captured image includes a fingerprint pattern). If the above fingerprint patterns are not included in consecutive M (M≤N) images, the user may not touch the fingerprint sensor at this time, or there may be other objects (such as hair, clothes, or face) other than the user's finger at this time. Etc.) The fingerprint sensor was touched by mistake. Therefore, the Bluetooth headset does not need to continue to collect images formed on the fingerprint sensor, and it is not necessary to perform the following steps S604-S606.

Alternatively, the Bluetooth headset may periodically detect whether the image collected by the fingerprint sensor has the above-mentioned fingerprint characteristics. For example, after the fingerprint sensor starts collecting images, the Bluetooth headset may randomly select one image from each of the collected five images to detect whether it has the above fingerprint characteristics. As another example, a Bluetooth headset may also randomly select one image from each image collected in 500ms to detect whether it has the above-mentioned fingerprint feature. If an image with the fingerprint characteristics is detected, it can be determined that the current touch operation on the fingerprint sensor is not an accidental touch operation. The Bluetooth headset can continue to continuously capture images formed on the fingerprint sensor, and perform the following steps S604-S606 .

Alternatively, the Bluetooth headset may also count the number of images including the fingerprint pattern in the N consecutive images after acquiring the N consecutive images collected by the fingerprint sensor. If the number of images containing fingerprint patterns in the N consecutive images is less than the first threshold (for example, less than 3), it may indicate that the user may have just touched the fingerprint sensor by mistake, otherwise, the Bluetooth headset may determine that the fingerprint sensor occurred this time The touch operation on this is not an accidental touch operation. Or, if the number of the N consecutive images that do not contain a fingerprint pattern is greater than the second threshold (for example, less than 10), it may also indicate that the user may have just touched the fingerprint sensor by mistake, otherwise, the Bluetooth headset can determine this time The touch operation that occurs on the fingerprint sensor is not an accidental touch operation.

That is, in the embodiment of the present application, only when the Bluetooth headset recognizes that the touch operation received by the fingerprint sensor includes a fingerprint input, it will continue to collect the image formed on the fingerprint sensor and identify the control corresponding to the touch operation. A gesture, and sending the identified control gesture or operation instruction to the terminal. Otherwise, if the touch operation received on the fingerprint sensor does not include fingerprint input, that is, the touch operation input by the user is a wrong touch operation, the mobile phone need not perform the following steps S604-S606, thereby avoiding the user's The accidental touch operation wakes up the mobile phone and also reduces the power consumption of the Bluetooth headset and mobile phone.

It should be noted that if the Bluetooth headset recognizes that the touch operation input by the user is a false touch operation during the process of collecting the N consecutive images, the Bluetooth headset may stop collecting the fingerprint sensor. If the Bluetooth headset recognizes the touch operation entered by the user as a touch operation by mistake, the Bluetooth headset can stop recognizing the touch operation entered by the user; if the Bluetooth headset is identifying the touch operation Only after the control gesture corresponding to the touch operation is recognized that the touch operation input by the user is a touch operation by mistake, the Bluetooth headset may stop sending the identified control gesture or operation instruction to the mobile phone.

S604. The Bluetooth headset recognizes the control gesture input by the user according to the fingerprint pattern in the N consecutive images.

In step S604, after the Bluetooth headset obtains the N consecutive images collected by the fingerprint sensor, it can identify the change of the fingerprint pattern in the N consecutive images. For example, the size of the fingerprint pattern can be gradually reduced or even completely disappeared in the N consecutive images, and the position of the fingerprint pattern can be continuously moved in the N consecutive images. Then, according to the change of the fingerprint pattern, the Bluetooth headset may determine what gesture is the specific control gesture input by the user to the fingerprint sensor in step S603.

Exemplarily, the Bluetooth headset can recognize the user to input a control gesture on the fingerprint sensor according to the change in the position and size of the fingerprint pattern in the N consecutive images. As shown in (a) of FIG. 8, if there are consecutive X (X is less than the first preset value) images in the above N consecutive images including the fingerprint pattern 801 and the size of the fingerprint pattern 801 does not change significantly, then It can be determined that the user has entered a single click operation on the fingerprint sensor. As shown in (b) of FIG. 8, if there are consecutive Y (Y is greater than the second preset value, the second preset value is greater than or equal to the first preset value) among the above N consecutive images, the fingerprints are included Pattern 801, and the size of the fingerprint pattern 801 does not change significantly, it can be determined that the user has entered a long press operation on the fingerprint sensor. As shown in (c) of FIG. 8, if there are consecutive Z (Z≤N) images among the above N consecutive images, the fingerprint pattern 801 is gradually moved from the A point to the B point, and the A point and the If the distance between points B is greater than the distance threshold, it can be determined that the user has entered a swipe operation on the fingerprint sensor.

Further, if the Bluetooth headset determines that the control gesture input by the user on the fingerprint sensor is a sliding operation, the Bluetooth headset may also determine the sliding direction and motion trajectory of the sliding operation according to the position of the fingerprint pattern 801 in the N consecutive images. And other parameters. For example, the sliding direction of the sliding operation may be up or down, and the motion trajectory of the sliding operation may be a closed graphic such as a circle.

Because the sensing unit in the fingerprint sensor is small in size and highly integrated, the fingerprint pattern in each image collected by the Bluetooth headset using the fingerprint sensor is more clear and accurate. Based on these fingerprint patterns, Bluetooth headsets have higher accuracy and sensitivity when recognizing control gestures entered by users. At the same time, the volume of Bluetooth headsets will not increase due to the size of the fingerprint sensor.

The process of identifying the control gesture by the Bluetooth headset may be performed by a computing module (such as a CPU) in the Bluetooth headset. Alternatively, the calculation module may also be integrated into the fingerprint sensor of the Bluetooth headset, and the above steps S603-S604 may be performed by the fingerprint sensor.

In addition, after the Bluetooth headset recognizes the control gesture, the following steps S605a or S605b may be performed.

S605a. The Bluetooth headset sends the control gesture to the mobile phone.

In step S605a, the Bluetooth headset may send the control gesture (for example, a long press operation) identified in step S604 to the mobile phone, and the mobile phone determines the corresponding operation instruction according to the control gesture, and performs the control gesture according to the following step S606 Corresponding operation instructions.

For example, after the Bluetooth headset recognizes that the control gesture input by the user on the fingerprint sensor is a long-press operation, it can send an identification (for example, 01) of the long-press operation to the mobile phone. The correspondence relationship between different control gestures and different operation instructions on the Bluetooth headset in each application is stored in the mobile phone in advance. Then, the mobile phone can determine the operation instruction corresponding to the long-press operation in the running application according to the correspondence relationship. Command to pause playback. Furthermore, the mobile phone can execute the instruction for pausing playback, so that the audio being played is paused. In this way, the user can control a running application in the mobile phone to implement related functions by performing a control gesture on the fingerprint sensor of the Bluetooth headset.

S605b. The Bluetooth headset sends an operation instruction corresponding to the control gesture to the mobile phone.

In step S605b, the correspondence relationship between different control gestures and different operation instructions may be stored in the Bluetooth headset in advance. For example, the operation instruction corresponding to the upward sliding operation is an instruction to increase the volume, and the operation instruction corresponding to the downward sliding operation is an instruction to decrease the volume. Then, after the Bluetooth headset recognizes the control gesture of the user on the fingerprint sensor, it can determine an operation instruction corresponding to the control gesture according to the corresponding relationship. Further, the Bluetooth headset may send the operation instruction corresponding to the control gesture to the mobile phone, so that the mobile phone may execute the operation instruction corresponding to the control gesture according to the following step S606.

Optionally, after the Bluetooth headset establishes a Bluetooth connection with the mobile phone, the mobile phone may set the Bluetooth module in the mobile phone to a sleep state. For example, if the Bluetooth connection between the Bluetooth headset and the mobile phone does not transmit data for a certain period of time (for example, 1 minute), the mobile phone can switch the Bluetooth module in the mobile phone to the sleep state to reduce the power consumption of the mobile phone. For another example, when the mobile phone enters the lock screen state, the Bluetooth module can be automatically switched to the sleep state. Therefore, before the Bluetooth headset sends the identified control gesture or an operation instruction corresponding to the control gesture to the mobile phone, the Bluetooth headset may also send a wake-up command to the mobile phone. In this way, the mobile phone can switch the Bluetooth module to a working state in response to the wake-up instruction, so that the mobile phone can recover the Bluetooth connection with the Bluetooth headset in advance, so that the mobile phone can quickly respond after receiving the operation instruction sent by the Bluetooth headset through the Bluetooth connection.

In other embodiments of the present application, the Bluetooth headset may also send the N consecutive images to the mobile phone after collecting the N consecutive images formed on the fingerprint sensor. Furthermore, the mobile phone recognizes the control gesture input by the user based on the fingerprint patterns in the N consecutive images, and determines and executes an operation instruction corresponding to the recognized control gesture. In this way, the Bluetooth headset does not need to perform gesture recognition and other tasks, which can reduce the implementation complexity and power consumption of the Bluetooth headset.

S606. The mobile phone executes an operation instruction corresponding to the control gesture.

In step S606, if the mobile phone receives the operation instruction sent by the Bluetooth headset, the mobile phone can directly execute the operation instruction. If the mobile phone receives the identified control gesture sent by the Bluetooth headset, the mobile phone may first determine an operation instruction corresponding to the control gesture, and then execute the operation instruction. Since the operation instruction set for the same control gesture may be different in different applications, after the mobile phone receives the control gesture sent by the Bluetooth headset, it can determine the operation instruction corresponding to the control gesture in the current application according to the specific application being run.

In some embodiments of the present application, when the Bluetooth headset sends the control gesture or operation instruction to the mobile phone, it may also send its own device identification (such as a MAC address) to the mobile phone. Because the mobile phone stores the identification of the legal Bluetooth device that has passed the authentication, the mobile phone can determine whether the currently connected Bluetooth headset is a legal Bluetooth device based on the received device identification. If the Bluetooth headset is a legitimate Bluetooth device, the mobile phone may further execute the operation instruction corresponding to the control gesture recognized by the Bluetooth headset, otherwise, the mobile phone may discard the above-mentioned control gesture or operation instruction sent by the Bluetooth headset, thereby avoiding the maliciousness of illegal Bluetooth devices. Security issues caused by manipulating the phone.

In addition, as shown in FIG. 9, the user may also enter a setting interface 901 for managing legal devices in the mobile phone. In the setting interface 901, a user may manually add a new control gesture or delete an old control gesture to a corresponding legal device. In addition, the user can manually set an operation instruction corresponding to each control gesture, so that the user can obtain a customized touch experience on a legitimate device.

So far, through steps S601-S606, the Bluetooth headset can use the fingerprint sensor to recognize a control gesture input by the user on the fingerprint sensor, and then control the mobile phone to execute an operation instruction corresponding to the control gesture. Due to the smaller size and higher integration of the sensing unit in the fingerprint sensor, the Bluetooth headset has higher sensitivity and accuracy when recognizing user gestures. At the same time, because the fingerprint sensor can recognize misoperation gestures triggered by non-user fingers, so When using the above touch control method to control the mobile phone, the probability of the Bluetooth headset and the mobile phone being triggered by mistake can be reduced, and the power consumption of the Bluetooth headset and the mobile phone can be reduced.

In some embodiments of the present application, an embodiment of the present application discloses a wearable device. As shown in FIG. 10, the wearable device may include: a fingerprint sensor 1001; one or more processors 1002; a memory 1003; communication The interface 1004; one or more application programs (not shown); and one or more computer programs 1005. The above-mentioned devices may be connected through one or more communication buses 1006. The one or more computer programs 1005 are stored in the memory 1003 and are configured to be executed by the one or more processors 1002. The one or more computer programs 1005 include instructions. 6 and the corresponding steps in the respective embodiments.

In addition, in combination with the wearable device shown in FIG. 2, the processor 1002 may be the computing module 207 in FIG. 2, the memory 1003 may be the storage module 208 in FIG. 2, and the communication interface 1004 may be the communication module 205 in FIG. 2. . Of course, the wearable device shown in FIG. 10 may further include components such as a microphone 201, an acceleration sensor 203, a proximity light sensor 204, a speaker 206, and a power supply 209 shown in FIG. 2, which are not limited in the embodiment of the present application.

Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of the description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be allocated as required. Completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For specific working processes of the system, device, and unit described above, reference may be made to corresponding processes in the foregoing method embodiments, and details are not described herein again.

Each functional unit in each of the embodiments of the present application may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially or partly contribute to the existing technology or all or part of the technical solutions may be embodied in the form of a software product. The computer software product is stored in a storage device. The medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor to perform all or part of the steps of the method described in the embodiments of the present application. The foregoing storage media include: flash media, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs, which can store program codes.

The above description is only a specific implementation of the embodiments of the present application, but the scope of protection of the embodiments of the present application is not limited to this. Any changes or replacements within the technical scope disclosed in the embodiments of the present application should be covered in the present. Within the protection scope of the application examples. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.

Claims

A touch method of a wearable device, wherein the wearable device is provided with a fingerprint sensor, and the method includes:

The wearable device uses the fingerprint sensor to detect a touch operation input by a user;

Determining whether the wearable device includes a fingerprint input in the touch operation;

If the touch operation includes a fingerprint input, the wearable device recognizes a control gesture corresponding to the touch operation;

The wearable device sends the control gesture to a terminal, or the wearable device sends an operation instruction corresponding to the control gesture to the terminal, so that the terminal executes the corresponding operation with the control gesture. Operation instruction, a communication connection is established between the wearable device and the terminal.
The method for touching a wearable device according to claim 1, wherein after the wearable device uses the fingerprint sensor to detect a touch operation input by a user, further comprising:

In response to the touch operation, the wearable device collects N consecutive images formed on the fingerprint sensor, at least one of the N consecutive images contains a fingerprint pattern, and N is an integer greater than 1;

Wherein, the wearable device identifying the control gesture corresponding to the touch operation includes:

The wearable device recognizes a control gesture corresponding to the touch operation according to a change of the fingerprint pattern in the N consecutive images.
The touch method of a wearable device according to claim 2, wherein the wearable device collects N consecutive images formed on the fingerprint sensor, comprising:

When the fingerprint sensor detects that a user's finger contacts the fingerprint sensor, it starts to acquire an image formed on the fingerprint sensor at a preset frequency;

When the fingerprint sensor detects that a user's finger leaves the fingerprint sensor, it stops collecting images formed on the fingerprint sensor to obtain the N consecutive images.
The touch method of a wearable device according to claim 2, wherein the wearable device collects N consecutive images formed on the fingerprint sensor, comprising:

When the fingerprint sensor detects that a user's finger contacts the fingerprint sensor, it starts to acquire an image formed on the fingerprint sensor at a preset frequency;

When the fingerprint sensor detects that a user's finger leaves the fingerprint sensor, it continues to collect images formed on the fingerprint sensor within a preset time;

If it is not detected that the user's finger touches the fingerprint sensor within the preset time, stop collecting images formed on the fingerprint sensor to obtain the N consecutive images.
The method for touching a wearable device according to any one of claims 2-4, wherein the wearable device recognizes all of the wearable devices based on a change in the fingerprint pattern in the N consecutive images. The control gestures corresponding to touch operations include:

The wearable device recognizes an image including a fingerprint pattern among the N consecutive images according to a preset fingerprint characteristic;

The wearable device recognizes a control gesture corresponding to the touch operation according to a size change and / or a position change of the fingerprint pattern in the N consecutive images.
The touch method for a wearable device according to claim 5, wherein the wearable device recognizes the fingerprint device according to a change in size and / or position of the fingerprint pattern in the N consecutive images. Control gestures for touch operations, including:

When there are consecutive X images in the N consecutive images including the fingerprint pattern, and the positions of the fingerprint patterns in the X images are the same, the control gesture corresponding to the touch operation is a click operation, X≤ N; or,

When there are consecutive Y images in the N consecutive images including the fingerprint pattern, and the positions of the fingerprint patterns in the Y images are the same, the control gesture corresponding to the touch operation is a long press operation, X < Y≤N; or,

When consecutive Z images in the N consecutive images include the fingerprint pattern, and the amount of displacement of the fingerprint pattern in the Z images is greater than a distance threshold, the control gesture corresponding to the touch operation is a sliding operation, Z≤N, or,

When there are L3 images that do not include the fingerprint pattern between L1 consecutive images that include the fingerprint pattern and L2 consecutive images that include the fingerprint pattern, the touch operation corresponds to The control gesture is a double-tap operation, L3 is less than a preset threshold, and 1 <L1 + L2 + L3 ≦ N.
The method for touching a wearable device according to any one of claims 1-6, characterized in that the control gesture is sent to the terminal at the wearable device; or Before the operation instruction corresponding to the control gesture is sent to the terminal, the method further includes:

The wearable device determines that the touch operation is not an accidental touch operation.
The touch method of a wearable device according to claim 7, wherein the wearable device determines that the touch operation is not a touch operation by mistake, comprising:

If there are P1 images in the N consecutive images including a fingerprint pattern, and P1 is greater than a preset value, the wearable device determines that the touch operation is not an accidental touch operation.
The touch method of the wearable device according to claim 8, wherein the method further comprises:

If P2 images of the N consecutive images include a fingerprint pattern, and P2 is less than the preset value, the wearable device determines that the touch operation is a touch operation by mistake;

The wearable device switches the fingerprint sensor from a working state to a sleep state.
The method for touching a wearable device according to any one of claims 1-9, before the wearable device receives a touch operation input by the user to the fingerprint sensor, further comprising:

In response to a wake-up operation input by a user, the wearable device switches the fingerprint sensor from a sleep state to a working state.
A wearable device, comprising a processor, a memory, a fingerprint sensor, and a communication interface connected to the processor, wherein:

The fingerprint sensor is configured to: receive a touch operation input by a user;

The processor is configured to: determine whether a fingerprint input is included in the touch operation; if the touch operation includes a fingerprint input, identify a control gesture corresponding to the touch operation;

The communication interface is configured to send the control gesture to a terminal, or send an operation instruction corresponding to the control gesture to the terminal, so that the terminal performs an operation corresponding to the control gesture. An instruction that a communication connection is established between the wearable device and the terminal.
The wearable device according to claim 11, wherein:

The fingerprint sensor is further configured to collect N consecutive images formed by the touch operation on the fingerprint sensor, at least one of the N consecutive images including a fingerprint pattern, and N is an integer greater than 1;

The processor used to identify the control gesture corresponding to the touch operation is specifically:

The processor is specifically configured to identify a control gesture corresponding to the touch operation according to a change of the fingerprint pattern in the N consecutive images.
The wearable device according to claim 12, wherein the fingerprint sensor for collecting N consecutive images formed by the touch operation on the fingerprint sensor is specifically:

The fingerprint sensor is specifically configured to: when it is detected that a user's finger contacts the fingerprint sensor, start to acquire an image formed on the fingerprint sensor at a preset frequency; when it is detected that the user's finger leaves the fingerprint sensor, stop The images formed on the fingerprint sensor are collected to obtain the N consecutive images.
The wearable device according to claim 12, wherein the fingerprint sensor for collecting N consecutive images formed by the touch operation on the fingerprint sensor is specifically:

The fingerprint sensor is specifically configured to: when it is detected that a user's finger contacts the fingerprint sensor, start to acquire an image formed on the fingerprint sensor at a preset frequency; when it is detected that the user's finger leaves the fingerprint sensor, in Continue to collect images formed on the fingerprint sensor within a preset time; if no user's finger is detected to contact the fingerprint sensor within the preset time, stop collecting images formed on the fingerprint sensor to obtain the Describe N consecutive images.
The wearable device according to any one of claims 12 to 14, wherein the processor for identifying a control gesture corresponding to the touch operation is specifically:

The processor is specifically configured to: identify an image including a fingerprint pattern in the N consecutive images according to a preset fingerprint characteristic; and change a size and / or a position of the fingerprint pattern in the N consecutive images according to the fingerprint pattern Change to recognize a control gesture corresponding to the touch operation.
The wearable device according to claim 15, wherein:

The processor is specifically configured to: when there are consecutive X images in the N consecutive images including the fingerprint pattern, and the positions of the fingerprint patterns in the X images are the same, determine the corresponding ones of the touch operations. The control gesture is a single-click operation, X ≦ N; or, when there are consecutive Y images in the N consecutive images including the fingerprint pattern, and the positions of the fingerprint patterns in the Y images are the same, determine the The control gesture corresponding to the touch operation is a long-press operation, X <Y ≦ N; or, when there are consecutive Z images in the N consecutive images containing the fingerprint pattern, and the fingerprint pattern in the Z images has The displacement is greater than the distance threshold, it is determined that the control gesture corresponding to the touch operation is a sliding operation, Z ≦ N, or when L1 consecutive images including the fingerprint pattern and L2 consecutive fingerprints are included in the N consecutive images There are L3 images between the consecutive images of the pattern that do not contain the fingerprint pattern. It is determined that the control gesture corresponding to the touch operation is a double-click operation, L3 is less than a preset threshold, and 1 <L1 + L2 + L3 ≦ N.
The wearable device according to any one of claims 12 to 16, wherein:

The processor is further configured to determine that the touch operation is not an accidental touch operation.
The wearable device according to claim 17, wherein the processor for determining that the touch operation is not a touch operation is specifically:

The processor is specifically configured to: if P1 of the N consecutive images include a fingerprint pattern, and P1 is greater than a preset value, determine that the touch operation is not an accidental touch operation.
The wearable device according to claim 18, wherein:

The processor is further configured to: if there are P2 images in the N consecutive images including a fingerprint pattern, and P2 is less than the preset value, determine that the touch operation is a false touch operation; and remove the fingerprint sensor from The working state is switched to the sleep state.
The wearable device according to any one of claims 12 to 19, wherein:

The processor is further configured to: if a wake-up operation input by a user is detected, switch the fingerprint sensor from a sleep state to a working state.
The wearable device according to any one of claims 11-20, wherein the fingerprint sensor is disposed on a side that is not in contact with a user when the wearable device is worn; the wearable device is a Bluetooth headset , Smart glasses or smart watch.
A computer-readable storage medium having instructions stored in the computer-readable storage medium, characterized in that when the instructions are run on a wearable device, the wearable device is caused to execute the instructions in claims 1-10. The touch method of the wearable device according to any one of the preceding claims.
A computer program product containing instructions, wherein when the computer program product is run on a wearable device, the wearable device is caused to execute a wearable device according to any one of claims 1-10. Touch method of wearable device.
A touch system, comprising a wearable device and a terminal, the wearable device is provided with a fingerprint sensor, and a communication connection is established between the wearable device and the terminal; wherein,

The wearable device is configured to: use the fingerprint sensor to detect a touch operation input by a user; determine whether the touch operation includes a fingerprint input; if the touch operation includes a fingerprint input, identify the touch operation A corresponding control gesture; sending the control gesture to the terminal, or sending an operation instruction corresponding to the control gesture to the terminal;

The terminal is configured to: receive a control gesture sent by the wearable device, or receive an operation instruction corresponding to the control gesture sent by the wearable device; and execute an operation instruction corresponding to the control gesture.