WO2022022609A1 - Procédé de prévention de toucher accidentel et dispositif électronique - Google Patents

Procédé de prévention de toucher accidentel et dispositif électronique Download PDF

Info

Publication number
WO2022022609A1
WO2022022609A1 PCT/CN2021/109122 CN2021109122W WO2022022609A1 WO 2022022609 A1 WO2022022609 A1 WO 2022022609A1 CN 2021109122 W CN2021109122 W CN 2021109122W WO 2022022609 A1 WO2022022609 A1 WO 2022022609A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
ultrasonic
ultrasonic echo
scene
data
Prior art date
Application number
PCT/CN2021/109122
Other languages
English (en)
Chinese (zh)
Inventor
李经纬
刘浩东
李琳
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022022609A1 publication Critical patent/WO2022022609A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
    • H04M1/026—Details of the structure or mounting of specific components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72454—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/725—Cordless telephones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W52/00—Power management, e.g. TPC [Transmission Power Control], power saving or power classes
    • H04W52/02—Power saving arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • the present application relates to the field of terminals, and in particular, to a method and electronic device for preventing accidental touch.
  • the mistaken touch of the mobile phone screen will bring a bad experience to the user.
  • the call is interrupted by accidentally hitting the hang-up button; due to capacitive factors such as skin,
  • the mobile phone in the pocket or backpack is unlocked by mistake or the application is clicked by mistake, etc., which may bring public opinion or risk of withdrawal to the user. Therefore, accurate detection of the state of the mobile phone and prevention of accidental touches can improve the user experience.
  • the present application provides a method and electronic device for preventing accidental touch, which are used to solve the problem of accidental touch in use of the electronic device.
  • an embodiment of the present application provides a method for preventing accidental touch, which is applied to an electronic device, where the electronic device includes an ultrasonic transmitter and an ultrasonic receiver, wherein the method includes:
  • the ultrasonic transmitter transmits ultrasonic signals N times, each ultrasonic signal includes multiple ultrasonic signals, N is greater than or equal to 2, and N is a positive integer.
  • the ultrasonic receiver receives N times of ultrasonic echo signals, wherein one ultrasonic echo signal is generated by the reflection of one ultrasonic signal, and each ultrasonic echo signal includes a plurality of ultrasonic echo signals.
  • the electronic device may obtain first data according to each received ultrasonic echo signal, where the first data includes signal strengths and propagation times of multiple ultrasonic echo signals.
  • the electronic device may obtain the first scene type in which the electronic device is located according to the first data of the N ultrasonic echo signals.
  • the first scene type may be an occlusion scene or a non-occlusion scene. If the first scene type is a occlusion scene, the electronic device enables the anti-mistouch function.
  • the electronic device when it is detected that the electronic device is currently blocked in a pocket or a backpack, the electronic device can automatically activate the anti-mistouch function, so as to prevent the occurrence of false touches and reduce the power consumption of the electronic device, It provides users with a friendly operating environment and improves the user experience.
  • the ultrasonic transmitter and the ultrasonic receiver are arranged on the top of the electronic device, wherein the top is further provided with any one or more of the following electronic devices: an earpiece, a front-facing camera, a microphone, Proximity light sensor, ambient light sensor, etc.
  • an ultrasonic transmitter is integrated in the earpiece, or the earpiece is an ultrasonic transmitter, which can transmit ultrasonic signals.
  • an ultrasonic receiver is integrated in the microphone, or the microphone is an ultrasonic receiver, which can receive ultrasonic signals.
  • the method may further specifically include: the electronic device inputs the first data of the N times ultrasonic echo signals into the first classification model to obtain the first scene type in which the electronic device is located.
  • the first classification model is obtained by using the first training data to train the first training model, and the first training data may include S sample data, S is greater than or equal to 2, and S is a positive integer.
  • the S pieces of sample data include sample data obtained under multiple known scene types, and one sample data includes second data of N times ultrasonic echo signals generated by transmitting N times ultrasonic signals under one known scene type.
  • the second data includes information such as signal strength and propagation time of the plurality of ultrasonic echo signals.
  • Several known scene types include: unoccluded scene and occluded scene.
  • the method may further specifically include: the electronic device generates a first image from the first data of the N-th ultrasonic echo signal, and the color value of the first image represents the signal of the ultrasonic echo signal Intensity, the horizontal axis of the first image represents the receiving batch of ultrasonic echo signals, and the vertical axis of the first image can represent the transmission time from transmitting ultrasonic signals to receiving ultrasonic echo signals. Then, the electronic device inputs the first image into the first classification model, and the first scene type in which the electronic device is located can be obtained.
  • a sample data includes a second image corresponding to a known scene type
  • the second image is generated from the second data of the N times ultrasonic echo signals
  • the color value of the second image indicates that under a known scene type
  • the signal strength of the received ultrasonic echo signal, the abscissa coordinate of the second image represents the acceptance batch of ultrasonic echo signals received in a known scene type, and the vertical axis coordinate of the second image represents a known scene
  • the transmission time from the transmission of ultrasonic signals to the reception of ultrasonic echo signals.
  • the first training model may be an extreme gradient boosting XGBoost model, or a neural network NN model, or a gradient boosting decision tree GBDT model, or a random forest RF model.
  • the occluded scene may include any one or more of the following: the electronic device is located in a pocket, the electronic device is located in a bag, the electronic device is blocked by a book, the electronic device is blocked by hair, the electronic device Covered by palms, electronic devices covered by clothing, etc.
  • the anti-mistouch function includes any one or more of the following: the screen of the electronic device is turned off, the electronic device does not respond to fingerprints to unlock the screen, the electronic device does not respond to face recognition to unlock the screen, the electronic device does not respond to face recognition to unlock the screen, Respond to sliding up to unlock the screen, the electronic device does not respond to gestures to unlock the screen, the electronic device does not respond to raising the hand to brighten the screen, the electronic device does not respond to raising the hand to answer incoming calls, the electronic device does not respond to fingerprints to answer incoming calls, etc.
  • the electronic device when it is detected that the electronic device is in an entertainment scene, the electronic device turns off the anti-mistouch function, and the entertainment scene may include any one or more of the following: the electronic device plays video, plays music, runs games etc.
  • the electronic device does not enable the anti-mistouch function if the proximity light sensor does not detect that the object is blocked.
  • the electronic device if the ambient light sensor detects that the ambient light brightness is higher than the first brightness value, eg, 10 lux (lx), the electronic device does not enable the accidental touch prevention function.
  • the first brightness value eg, 10 lux (lx)
  • the ultrasonic transmitter may transmit ultrasonic signals N times at intervals of a transmission period T, and the continuous transmission time t of one ultrasonic signal is shorter than the transmission period T of the ultrasonic signal.
  • an embodiment of the present application provides an electronic device, the electronic device includes: an ultrasonic transmitter, an ultrasonic receiver, a display screen, a memory, and a processor coupled to the memory, where data and executable data are stored in the memory instruction.
  • the processor can transmit ultrasonic signals N times through the ultrasonic transmitter, each ultrasonic signal includes multiple ultrasonic signals, N is greater than or equal to 2, and N is a positive integer.
  • the processor can receive N times ultrasonic echo signals through the ultrasonic receiver, wherein, one ultrasonic echo signal is generated by the reflection of one ultrasonic signal, and each ultrasonic echo signal includes a plurality of ultrasonic echo signals.
  • the processor may also obtain first data according to each received ultrasonic echo signal, where the first data includes signal strengths and propagation times of multiple ultrasonic echo signals.
  • the processor may obtain the first scene type in which the electronic device is located according to the first data of the N ultrasonic echo signals.
  • the first scene type may be an occlusion scene or a non-occlusion scene. If the first scene type is an occlusion scene, the processor controls the display screen to enable the anti-mistouch function.
  • the electronic device when it is detected that the electronic device is currently blocked in a pocket or a backpack, the electronic device can automatically activate the anti-mistouch function, so as to prevent the occurrence of false touches and reduce the power consumption of the electronic device , provides users with a friendly operating environment and improves the user experience.
  • the ultrasonic transmitter and the ultrasonic receiver are arranged on the top of the electronic device, wherein the top is further provided with any one or more of the following electronic devices: an earpiece, a front-facing camera, a microphone, Proximity light sensor, ambient light sensor, etc.
  • an ultrasonic transmitter is integrated into the earpiece, or the earpiece is an ultrasonic transmitter, which can transmit ultrasonic signals.
  • an ultrasonic receiver is integrated in the microphone, or the microphone is an ultrasonic receiver, which can receive ultrasonic signals.
  • the processor inputs the first data of the N times ultrasonic echo signals into the first classification model to obtain the first scene type in which the electronic device is located.
  • the first classification model is obtained by using the first training data to train the first training model.
  • the first training data may include S sample data, where S is greater than or equal to 2, and S is a positive integer.
  • the S pieces of sample data include sample data obtained under multiple known scene types, and one sample data includes second data of N times ultrasonic echo signals generated by transmitting N times ultrasonic signals under one known scene type.
  • the second data includes information such as signal strength and propagation time of the plurality of ultrasonic echo signals.
  • Several known scene types include: unoccluded scene and occluded scene.
  • the processor may further specifically include: the processor generates a first image from the first data of the N times ultrasonic echo signals, and the color value of the first image represents the signal strength of the ultrasonic echo signals,
  • the abscissa coordinate of the first image represents the receiving batch of ultrasonic echo signals, and the ordinate axis coordinate of the first image may represent the transmission time from transmitting the ultrasonic signal to receiving the ultrasonic echo signal.
  • the processor inputs the first image into the first classification model, and can obtain the first scene type in which the electronic device is located.
  • a sample data includes a second image corresponding to a known scene type
  • the second image is generated from the second data of the N times ultrasonic echo signals
  • the color value of the second image indicates that under a known scene type
  • the signal strength of the received ultrasonic echo signal, the abscissa coordinate of the second image represents the acceptance batch of ultrasonic echo signals received in a known scene type, and the vertical axis coordinate of the second image represents a known scene
  • the transmission time from the transmission of ultrasonic signals to the reception of ultrasonic echo signals.
  • the first training model may be an extreme gradient boosting XGBoost model, or a neural network NN model, or a gradient boosting decision tree GBDT model, or a random forest RF model.
  • the occluded scene may include any one or more of the following: the electronic device is located in a pocket, the electronic device is located in a bag, the electronic device is occluded by a book, the electronic device is occluded by hair, the electronic device Covered by palms, electronic devices covered by clothing, etc.
  • the anti-mistouch function includes any one or more of the following: the electronic device turns off the screen, the electronic device does not respond to fingerprints to unlock the screen, the electronic device does not respond to face recognition to unlock the screen, the electronic device does not respond to face recognition to unlock the screen, Respond to sliding up to unlock the screen, the electronic device does not respond to gestures to unlock the screen, the electronic device does not respond to raising the hand to brighten the screen, the electronic device does not respond to raising the hand to answer incoming calls, the electronic device does not respond to fingerprints to answer incoming calls, etc.
  • the processor when it is detected that the processor is in an entertainment scene, the processor turns off the anti-mistouch function, and the entertainment scene may include any one or more of the following: playing video, playing music, running games, and the like.
  • the processor does not enable the anti-mistouch function if the proximity light sensor does not detect that the object is blocked.
  • the processor does not enable the false-touch prevention function if the ambient light sensor detects that the ambient light brightness is higher than the first brightness value, eg, 10 lux (lx), the processor does not enable the false-touch prevention function.
  • the first brightness value eg, 10 lux (lx)
  • the ultrasonic transmitter may transmit ultrasonic signals N times at intervals of a transmission period T, and the continuous transmission time t of one ultrasonic signal is less than the transmission period T of the ultrasonic signal.
  • an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program runs on an electronic device, the electronic device performs the execution of the first aspect The operation corresponding to the provided method.
  • an embodiment of the present application provides a computer program product, which, when the computer program product runs on a computer, causes the computer to execute the method described in the first aspect.
  • the electronic device can accurately detect the current state.
  • the electronic device can automatically activate the anti-mistouch function, such as turning off the electronic device.
  • Screen do not respond to screen unlocking (which can include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), do not respond to raising your hand to brighten the screen, do not respond to raising your hand to answer incoming calls, do not respond to fingerprints to answer incoming calls, turn off always on Display (always on display, AOD), etc., can prevent the occurrence of false touches and reduce the power consumption of electronic devices by transmitting ultrasonic signals at intervals, providing users with a friendly operating environment and improving user experience.
  • the ultrasonic signal can be transmitted at intervals by setting the duty ratio, which reduces the power consumption compared with the existing method of continuously measuring the distance of the ultrasonic signal to detect obstacles, and can achieve low power consumption and constant In the open state, determine whether the current electronic device is blocked.
  • the solution of the present application can effectively identify static objects around the electronic device, so that the electronic device can more effectively and accurately determine whether the current electronic device is in a blocked state in scenarios such as pockets and backpacks.
  • the use of ultrasonic sensors instead of optical proximity sensors to achieve the anti-mistouch function can also reduce the number of electronic devices in electronic devices and save the front opening of the screen, narrow the frame space of electronic devices, increase the screen ratio of electronic devices, and improve electronic devices. Dust and water resistance, etc.
  • FIG. 1 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
  • FIG. 2 is a software structural block diagram of an electronic device provided by an embodiment of the present application.
  • 3a is a schematic diagram of the appearance of an electronic device provided by an embodiment of the present application.
  • Figure 3b is a schematic diagram of an ultrasonic emission sound field provided by an embodiment of the present application.
  • 3c is a schematic diagram of an ultrasonic echo path provided by an embodiment of the present application.
  • FIG. 4 is an example diagram of an impulse response of an ultrasonic echo signal in some scenarios provided by an embodiment of the present application.
  • FIG. 5a is a schematic diagram of a user scenario provided by an embodiment of the present application.
  • FIG. 5b is a schematic diagram of a user scenario provided by an embodiment of the present application.
  • FIG. 5c is a schematic diagram of a user scenario provided by an embodiment of the present application.
  • 6a is a schematic diagram of a user interface provided by an embodiment of the present application.
  • 6b is a schematic diagram of a user interface provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of a method for preventing false touches provided by an embodiment of the present application.
  • FIG. 8 is a signal strength diagram of an ultrasonic echo signal in some scenarios provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a convolutional neural network algorithm provided by an embodiment of the present application.
  • FIG. 10 is a block diagram of functional modules of an electronic device for preventing accidental touches provided by an embodiment of the present application.
  • the present application provides a method and electronic device for preventing accidental touch, which are used to solve the problem of accidental touch in use of the electronic device.
  • This application is based on machine learning, by intermittently transmitting ultrasonic waves and collecting the echoes reflected by the ultrasonic waves encountering obstacles, and according to the signal characteristics of the echoes, it is detected whether the current electronic device is in a blocked state.
  • the electronic device can automatically activate the anti-mistouch mode. In the anti-mistouch mode, the electronic device does not respond to touch operations, unlock the screen, raise the hand to brighten the screen, AOD and other commands.
  • the electronic device can accurately detect whether it is currently in a blocked state, and when it is determined that the electronic device is currently blocked in a pocket or a backpack, the electronic device can automatically activate the anti-mistouch function, such as the electronic device Turn off the screen, do not respond to screen unlocking (which can include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), do not respond to raising your hand to brighten the screen, do not respond to raising your hand to answer incoming calls, turn off AOD, etc., to prevent false positives.
  • screen unlocking which can include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.
  • do not respond to raising your hand to brighten the screen do not respond to raising your hand to answer incoming calls, turn off AOD, etc.
  • the occurrence of a touch situation and the power consumption of the electronic device are reduced, a user-friendly operating environment is provided, and the user experience is improved.
  • ultrasonic sensors instead of optical proximity sensors to achieve the anti-mistouch function can also reduce the number of electronic devices in electronic devices and save the front opening of the screen, narrow the frame space of electronic devices, increase the screen ratio of electronic devices, and improve electronic devices.
  • the application of ultrasonic sensors is more extensive, especially in the case of strong light, water mist, etc., the optical proximity sensor is easy to fail.
  • One of the solutions in the prior art is to use ultrasonic signals to continuously measure distances to determine whether an object is dynamically approaching or moving away.
  • the electronic device continuously transmits ultrasonic signals, and by acquiring the time and signal strength changes of the ultrasonic signals from transmitting to receiving echoes, it is judged whether an object is dynamically approaching or moving away near the electronic device.
  • the ultrasonic signal needs to be continuously transmitted, the audio channel needs to be turned on all the time, and the power consumption is large; and due to the different materials and positions of different obstacles, the time for the ultrasonic signal to encounter the ultrasonic echo signal reflected by different obstacles , signal strength, etc.
  • the ultrasonic signal can be transmitted at intervals by setting the duty cycle, which reduces the power consumption compared with the aforementioned ultrasonic signal ranging scheme, and can determine whether the current electronic device is blocked in the low-power normally-on state.
  • the solution of the present application can effectively identify static objects around the electronic device, so that the electronic device can more effectively and accurately determine whether the current electronic device is in a blocked state in scenarios such as pockets and backpacks.
  • Ultrasound is a sound wave with a frequency higher than 20,000 hertz (Hz). Since the frequency of the sound wave that can be discerned by the human ear is about 20 to 20,000 Hz, generally when the vibration frequency of the sound wave is greater than 20,000 Hz, the human ear cannot hear it.
  • the lower limit of the frequency of the ultrasonic wave is approximately equal to The upper limit of human hearing, so it is called ultrasound.
  • the exemplary electronic device 100 provided in the embodiment of the present application is introduced. It should be understood that the electronic device 100 may have more or fewer components than those shown in the figures, may combine two or more components, or may have different component configurations.
  • the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • FIG. 1 is a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194 and Subscriber identification module (subscriber identification module, SIM) card interface 195 and so on.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a gravity sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, ultrasonic sensor 180M, etc.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processor
  • graphics processor graphics processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and processor 110 latency is reduced, thereby increasing the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may contain multiple sets of I2C buses.
  • the processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193 and the like through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .
  • the I2S interface can be used for audio communication.
  • the processor 110 may contain multiple sets of I2S buses.
  • the processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is typically used to connect the processor 110 with the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc.
  • the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like.
  • the GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the SIM interface can be used to communicate with the SIM card interface 195 to realize the function of transferring data to the SIM card or reading data in the SIM card.
  • the USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones.
  • the interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger may be a wireless charger or a wired charger.
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low frequency baseband signal is processed by the baseband processor and passed to the application processor.
  • the application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 .
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR).
  • WLAN wireless local area networks
  • BT Bluetooth
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication
  • IR infrared technology
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110, perform frequency modulation on it, amplify it, and convert it into an electromagnetic wave for radiation through the antenna 2.
  • the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • Display screen 194 is used to display images, videos, and the like.
  • Display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
  • the display screen 194 has a touch function, which may be called a touch screen, that is, the electronic device 100 can respond according to the corresponding position of the user touching the display screen 194 .
  • the electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • the ISP is used to process the data fed back by the camera 193 .
  • the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin tone.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193 .
  • Camera 193 is used to capture still images or video.
  • the object is projected through the lens to generate an optical image onto the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • a digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, and the like.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs.
  • the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.
  • MPEG Moving Picture Experts Group
  • MPEG2 moving picture experts group
  • MPEG3 MPEG4
  • MPEG4 Moving Picture Experts Group
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 .
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 .
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application required for at least one function (such as a face recognition function, a fingerprint recognition function, a mobile payment function, etc.) and the like.
  • the storage data area may store data created during the use of the electronic device 100 (such as face information template data, fingerprint information template, etc.) and the like.
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.
  • the audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .
  • Speaker 170A also referred to as a "speaker" is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
  • the receiver 170B also referred to as "earpiece" is used to convert audio electrical signals into sound signals.
  • the voice can be answered by placing the receiver 170B close to the human ear.
  • the microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals.
  • the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.
  • the earphone jack 170D is used to connect wired earphones.
  • the earphone interface 170D can be the USB interface 130, or can be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • the pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals.
  • the pressure sensor 180A may be provided on the display screen 194 .
  • the capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes.
  • the electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.
  • the gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 .
  • the angular velocity of electronic device 100 about three axes ie, x, y, and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenarios.
  • Gravity sensor 180C is used to measure gravity.
  • the electronic device 100 can measure the direction of gravity and the data value of gravity through the gravity sensor 180C to assist the conversion of the display screen.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D.
  • the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).
  • the magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the electronic device 100 emits infrared light to the outside through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power.
  • Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.
  • the proximity light sensor and the ultrasonic sensor can be coupled to determine whether the electronic device enables the anti-mistouch function. For example, when either the proximity light sensor or the ultrasonic sensor detects that the current electronic device is in a blocked state, the electronic device is turned on. Anti-mistouch function; or when both the proximity light sensor and the ultrasonic sensor detect that the current electronic device is in a blocked state, the electronic device will turn on the anti-mistouch function, etc.
  • the ambient light sensor 180L is used to sense ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touch.
  • the ambient light sensor and the ultrasonic sensor can be coupled to determine whether the electronic device enables the anti-mistouch function.
  • the ultrasonic sensor detects that the electronic device is in a blocked state
  • the ambient light sensor detects that the brightness of the ambient light is higher than At a certain brightness value
  • the anti-mistouch function is still not turned on; or when the ambient light sensor detects that the ambient light brightness is lower than the first brightness value, such as 10 lux (lx), and the ultrasonic sensor detects that the current electronic device is in a blocked state , the electronic device will turn on the anti-mistouch function, etc.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.
  • the temperature sensor 180J is used to detect the temperature.
  • the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection.
  • the electronic device 100 when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature.
  • the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to touch operations may be provided through display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.
  • the ultrasonic sensor 180M is used to detect the current state of the electronic device 100 by transmitting and receiving ultrasonic waves, and then the processor 110 determines whether to enable the anti-mistouch mode.
  • the keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key.
  • the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
  • Motor 191 can generate vibrating cues.
  • the motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback.
  • touch operations acting on different applications can correspond to different vibration feedback effects.
  • the motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.
  • the SIM card interface 195 is used to connect a SIM card.
  • the SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 .
  • the electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • the SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different.
  • the SIM card interface 195 can also be compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication.
  • FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces.
  • the Android system can be divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and system libraries, and a kernel layer.
  • the application layer can include a series of application packages.
  • the application package may include applications (also referred to as applications) such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.
  • applications also referred to as applications
  • the application layer may also include an application in the anti-mistouch mode, and when the application in the anti-mistouch mode runs, the ultrasonic sensor needs to be called to send/receive ultrasonic signals.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a Local Profile Assistant (LPA), and an ultrasonic sensor call Control Manager, etc.
  • a window manager a content provider
  • a view system a phone manager
  • a resource manager a notification manager
  • LPA Local Profile Assistant
  • ultrasonic sensor call Control Manager etc.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.
  • Content providers are used to store and retrieve data and make these data accessible to applications.
  • the data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications.
  • a display interface can consist of one or more views.
  • the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide the communication function of the electronic device 100 .
  • the management of call status including connecting, hanging up, etc.).
  • the resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.
  • the notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications from applications running in the background, and can also display notifications on the screen in the form of a dialog interface. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.
  • the Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.
  • the core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.
  • a system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • surface manager surface manager
  • media library Media Libraries
  • 3D graphics processing library eg: OpenGL ES
  • 2D graphics engine eg: SGL
  • the Surface Manager is used to manage the display subsystem and provides a fusion of two-dimensional (2-Dimensional, 2D) and three-dimensional (3-Dimensional, 3D) layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display drivers, camera drivers, audio drivers, sensor drivers, and virtual card drivers.
  • the sensor driving includes the driving of the ultrasonic sensor, and the driving of the ultrasonic sensor is used to drive the ultrasonic sensor 180M.
  • the ultrasonic sensor 180M is used for sending and receiving ultrasonic signals.
  • the workflow of the software and hardware of the electronic device 100 is exemplified in conjunction with capturing a photographing scene.
  • a corresponding hardware interrupt is sent to the kernel layer.
  • the kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer.
  • the application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon, for example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer.
  • the camera 193 captures still images or video.
  • Ultrasonic sensors are sensors developed using the characteristics of ultrasonic waves.
  • Ultrasound is a kind of mechanical wave whose vibration frequency is higher than that of sound wave. It has the characteristics of high frequency, short wavelength, small diffraction phenomenon, good directionality, and can become a ray and propagate in a direction.
  • Ultrasound is highly directional. Ultrasonic waves can propagate in gases, liquids and solids, and the propagation speed is different. Ultrasonic waves will also have phenomena such as refraction, reflection, and diffraction, and they will be attenuated during the propagation process.
  • Ultrasonic waves propagate in the air, and their frequency is low, generally tens of kilohertz (kHz), while in solids and liquids, the frequencies are higher, and the propagation attenuation in air is also faster, while in liquids and solids, Relative attenuation is smaller and spreads farther.
  • kHz kilohertz
  • Ultrasonic sensors may include ultrasonic transmitters and ultrasonic receivers, and in the embodiments of the present application, reference to ultrasonic transmitters and ultrasonic receivers is intended to cover all functional alternatives that may be collectively referred to as ultrasonic sensors.
  • the ultrasonic transmitter is used to transmit ultrasonic signals. When the ultrasonic signal encounters obstacles, it will reflect the ultrasonic echo to the ultrasonic receiver, so that the ultrasonic sensor can detect the object to be measured.
  • the ultrasonic transmitter and ultrasonic receiver of the ultrasonic sensor can be concentrated on the same device or can be separated.
  • the ultrasonic sensor can even be any available combination of ultrasonic transmitter and ultrasonic receiver with the same function.
  • the number of ultrasonic sensors may be one or more.
  • the ultrasonic transmitter is intended to comprise one or more ultrasonic transmitters and the ultrasonic receiver is intended to comprise one or more ultrasonic receivers.
  • the number of ultrasonic transmitters and the number of ultrasonic receivers may or may not be equal. This embodiment of the present application does not impose any restrictions on the quantity and position of the ultrasonic sensors on the electronic device 100 .
  • One or more earpieces, speakers, microphones of the electronic device 100 for audio functions may also be used for the measurement of ultrasound. It can be understood that the earpiece and the loudspeaker can be used as an ultrasonic transmitter, and the microphone can be used as an ultrasonic receiver, which can save the component cost and internal space of the electronic device 100 .
  • an earpiece 301 may be installed above the front of the mobile phone 300, and the earpiece 301 may be used as an ultrasonic transmitter for transmitting ultrasonic signals.
  • a noise reduction microphone 302 may be installed on the top of the mobile phone 300, and the microphone 302 may be used as an ultrasonic receiver for receiving ultrasonic echo signals.
  • FIG 3b shows a schematic diagram of the emission range of the ultrasonic transmitter.
  • the transmission trajectory of the ultrasonic signal is a conical beam with the ultrasonic transmitter as the origin, and the center line of the conical beam is the ultrasonic wave.
  • the emission direction of the signal, the ultrasonic signal is scattered to the surrounding space along the direction of the first angle with the emission direction, forming a conical beam.
  • the specific value of the first angle is related to the specific design of the ultrasonic transmitter. There are no restrictions on the application.
  • the transmitted ultrasonic signal can be a single-frequency continuous wave (continuous wave, CW), a linear frequency modulation continuous wave (linear frequency modulation, LFM), a ZC sequence (Zadoff-Chu sequence), etc.
  • CW continuous wave
  • LFM linear frequency modulation
  • ZC sequence Zadoff-Chu sequence
  • power consumption can be reduced by intermittently transmitting ultrasonic waves, that is, by setting a duty ratio.
  • the ultrasonic wave signal can be continuously transmitted for 150 milliseconds in a period of 1 second.
  • the channel for transmitting ultrasonic waves is powered off, that is, ultrasonic signals are not transmitted, so that the duty cycle is 0.15, which can ensure that the power consumption of transmitting ultrasonic waves is at a relatively low level.
  • the ultrasonic waves emitted by the earpiece will pass through different paths to form ultrasonic echoes and propagate to the microphone.
  • the main paths of the ultrasonic echoes involved in this embodiment of the present application are shown in Figure 3c:
  • Solid structure sound path 303 ultrasonic waves emitted from the earpiece 301 pass through the internal solid structure of the mobile phone 300 and propagate to the microphone 302 .
  • the propagation speed of sound in solids (>2000m/s) is faster than that in air (about 340m/s), so the propagation time of solid structure sound path 303 is the shortest, and the propagation time of this path 303 is very stability.
  • Direct air sound path 304 the earpiece 301 emits ultrasonic waves, which are propagated into the air, and are directly propagated to the microphone 302 without being reflected.
  • the propagation time of the direct air sound path 304 is slower than that of the solid structure sound path 303, but the path 304 is less affected by external objects, so the propagation time of the air direct sound path 304 is also relatively stable.
  • Air reflected sound path 305 the earpiece 301 emits ultrasonic waves and propagates into the air.
  • the ultrasonic signal When encountering obstacles above the microphone and the earpiece, the ultrasonic signal generates ultrasonic echoes through reflection, refraction, diffraction, etc., and the ultrasonic echoes propagate to the microphone 302. . Due to uncertainties such as the existence of obstacles, materials, occlusion positions, and occlusion distances, the air-reflected sound path 305 cannot accurately measure its propagation time.
  • the processor of the mobile phone 300 determines whether there is an obstacle above the mobile phone 300 according to the propagation time, sound wave intensity and other information of the different ultrasonic echo signals, and then determines whether it is necessary to activate the protection.
  • Accidental touch mode the processor of the mobile phone 300 determines whether there is an obstacle above the mobile phone 300 according to the propagation time, sound wave intensity and other information of the different ultrasonic echo signals, and then determines whether it is necessary to activate the protection.
  • Accidental touch mode is the processor of the mobile phone 300 determines whether there is an obstacle above the mobile phone 300 according to the propagation time, sound wave intensity and other information of the different ultrasonic echo signals, and then determines whether it is necessary to activate the protection.
  • Accidental touch mode the processor of the mobile phone 300 determines whether there is an obstacle above the mobile phone 300 according to the propagation time, sound wave intensity and other information of the different ultrasonic echo signals, and then determines whether it is necessary to activate the protection.
  • Accidental touch mode the processor of the mobile phone 300 determines whether there is an
  • the ultrasonic echo Sig R received by the microphone 302 is the linear superposition of the ultrasonic signals on each path after the time t of the ultrasonic Sig T emitted by the earpiece 301 . Therefore, the baseband signal of the ultrasonic echo Sig R received by the microphone 302 can be expressed as:
  • the baseband signal received by the ultrasonic receiver is a replica of a delay ⁇ i of the ultrasonic sequence transmitted by the ultrasonic transmitter Sig(t- ⁇ i ) .
  • the amplitude A i will not change drastically, and the phase It is possible to change with the movement.
  • the effect of the path on the signal is equivalent to passing through a linear system with impulse response h:
  • ⁇ (t) is the Dirac shock function. Therefore, if the impulse response h(t) of the signal can be obtained, the amplitudes of the delays of different paths can be known, so as to separate the paths of different delays.
  • the correlation calculation is used, which is equivalent to the conjugation of the reverse sequence of convolution, which satisfies the commutative law.
  • FIG. 4 shows a schematic diagram of the impulse response of a single-frame ultrasonic echo signal collected by a mobile phone microphone in some scenarios, wherein the abscissa of each figure represents the path traveled by the ultrasonic waves The ordinate of each figure represents the intensity of the ultrasonic echo signal impulse response. The larger the value, the higher the intensity.
  • the waveform of the correlation peak in the figure can reflect the occlusion of objects near the mobile phone.
  • the main peak represents the impact response of the ultrasonic wave directly from the earpiece to the microphone path through the internal solid structure of the mobile phone and the air
  • the peak after the main peak represents the impact of the ultrasonic wave reaching the microphone after it is reflected from an obstacle after it is emitted from the earpiece.
  • the main peak and the next peak after the main peak may have different waveform fusion, amplitude changes, etc., and then show different data characteristics. .
  • Figure a in Figure 4 is an example diagram of the ultrasonic echo impulse response when the palm is blocked at 0 cm above the mobile phone;
  • Figure b in Figure 4 is an example diagram of the ultrasonic echo impulse response when the palm is blocked at 6 cm above the mobile phone;
  • Figure c in Figure 4 is an example diagram of the ultrasonic echo impulse response when the palm is blocked at 8 cm above the mobile phone;
  • Figure d in Figure 4 is an example diagram of the ultrasonic echo impulse response when the mobile phone is in a backpack;
  • Figure 4 e The picture is an example diagram of the ultrasonic echo impulse response when the mobile phone is in the pocket of jeans;
  • the picture f in FIG. 4 is an example diagram of the ultrasonic echo impulse response when the mobile phone is not blocked above. It can be seen from Figure 4 that under different scenarios, the width, position and height of the main peak and the width, position and height of the next peak after the main peak show different shapes.
  • This application is mainly used in the detection of whether the electronic device is currently in a blocked state.
  • the electronic device can automatically activate the anti-mistouch mode.
  • the anti-mistouch mode the The electronic device does not respond to commands such as touch operations, unlocking the screen, raising the hand to brighten the screen, AOD, etc., which can prevent the occurrence of false touches and reduce the power consumption of the electronic device, improving the user experience.
  • the screen unlocking may include sliding screen unlocking, touch unlocking, password unlocking, gesture unlocking, fingerprint unlocking, face unlocking, voice unlocking, voiceprint unlocking, and the like.
  • the electronic device restricts the response to the command, not limited to the above-mentioned touch operation, screen unlocking, raising your hand to brighten the screen, AOD, etc., but also other commands, such as raising your hand to answer an incoming call, automatic Adjusting brightness, etc., developers can set according to specific conditions, and this application does not make any restrictions.
  • Figure 5a shows a common application scenario, the pocket scenario: the user puts the mobile phone 501 in the pocket 502, the mobile phone 501 can detect that it is in a blocked state, and then in the anti-mistouch mode, the mobile phone 501 does not respond to touch operations , unlock the screen, raise your hand to brighten the screen, AOD and other commands to prevent accidental touches.
  • This embodiment does not impose any restrictions on the material of the pocket, and the material of the pocket may be cotton, chiffon, polyester, mixed fabric, and the like.
  • Figure 5b shows another common application scenario, the luggage scenario: the user puts the mobile phone 503 in the backpack 504, the mobile phone 503 can detect that it is in a blocked state, and then in the anti-mistouch mode, the mobile phone 503 does not respond to touch Commands such as operation, unlocking the screen, raising your hand to brighten the screen, and AOD can prevent accidental touches.
  • the luggage scene here is a broad concept, which can include backpacks, single-shoulder backpacks, handbags, wallets, handbags, boxes, etc. This embodiment does not impose any restrictions on the material of the backpack, and the material of the backpack can be cotton, cowhide, etc. , leather, canvas, plastic, mixed materials, and more.
  • FIG. 5c shows an application scenario in which the palm 506 covers the upper part of the mobile phone 505, and the palm 506 has different distances from the mobile phone 505, such as 2 cm.
  • the palm 506 when the palm 506 is close to the top of the mobile phone 505, that is, at a distance of 0 cm, the mobile phone is considered to be blocked, and the anti-mistouch function is enabled at this time.
  • the application scenarios shown in FIGS. 5a, 5b, and 5c do not limit the embodiments of the present application.
  • the anti-mistouch mode is not only applied to pocket scenarios, backpack scenarios, luggage scenarios, and palm blocking scenarios, but also applied to other electronic devices.
  • the occluded situation such as book occlusion, face occlusion (during a call), etc., can be set by the developer according to the specific situation, which is not limited in this application.
  • a setting bar 602 of “Anti-Accidental Touch Mode” may be displayed on the setting user interface 601 of the electronic device, and the user may manually select an option 603 to enable/disable the “Anti-Accidental Touch Mode”.
  • the "anti-accidental touch mode” is turned on, when the electronic device detects that it is currently in a blocked state, the electronic device turns on the anti-accidental touch function, which effectively prevents the occurrence of a false-touch situation.
  • the anti-mistouch function can be that the electronic device turns off the screen, the electronic device does not respond to the screen unlocking (which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), the electronic device does not respond to raising the hand to turn on the screen, and the electronic device does not respond. In response to raising your hand to answer an incoming call, the electronic device turns off AOD and other functions.
  • the anti-accidental touch function of the electronic device may be interrupted.
  • the electronic device can be set not to activate the anti-mistouch function, so even if the electronic device is detected to be blocked, for example, when the user plays games on the horizontal screen of the mobile phone, the hand is blocked. Going to the top of the electronic device does not activate the automatic screen off either. Or the user can set to turn on or off the anti-mistouch function by himself, or set one or more applications to turn on or off the anti-mistouch function by himself. If the user chooses to turn off the "anti-mistouch mode" function, the electronic device will turn off the aforementioned anti-mistouch function. In the general settings, the setting of "Anti-Accidental Touch Mode" can take effect for all applications.
  • a shortcut button 605 for the anti-touch mode may also be displayed, so that the user can quickly turn on/off the anti-touch mode.
  • This embodiment of the present application does not impose any restrictions on the name and icon of the "anti-touch mode" in the user interface, and what is shown in FIG. 6a and FIG. 6b is just an example.
  • GUI graphical user interface
  • an incoming call scenario when the electronic device has not yet activated the anti-mistouch state, other judgment logic may be added to the audio driver layer to turn off ultrasonic detection in certain usage scenarios, thereby improving user experience and further reducing power consumption .
  • the entertainment scenario may include the user using an electronic device to watch videos, listen to music, play games, etc.; the incoming call scenario includes the electronic device receiving an incoming call or instant messaging software (such as WeChat, QQ, Skype, Face Time, etc.) voice chat, video chat, etc.
  • Ultrasonic sensors can also be combined with other sensors for coupling judgment, such as gravity sensors, gyroscope sensors, and ambient light sensors.
  • the processor of the electronic device can perform a coupling analysis on whether the detections reported by the ambient light sensor and the ultrasonic sensor are currently in a blocked state. As long as there is one that indicates that the current electronic device is in a blocked state, the electronic device can determine whether the detection is currently in a blocked state. It is currently blocked, and then the electronic device turns on the anti-mistouch function, and the touch screen is locked.
  • the method is applied to an electronic device with an ultrasonic transmitter and an ultrasonic receiver.
  • the ultrasonic transmitter and the ultrasonic receiver may be arranged on the top of the electronic device, or may be located at other positions, which are not limited in this embodiment.
  • the ultrasonic transmitter and the ultrasonic receiver are intended to cover any electronic device capable of transmitting and receiving ultrasonic waves, and are not limited to the ultrasonic transmitter and ultrasonic receiver in the narrow sense.
  • the ultrasonic transmitter and ultrasonic receiver can be centralized on the same device, or they can be separated.
  • the number of ultrasonic transmitters or ultrasonic receivers may be one or more.
  • the ultrasonic transmitter is intended to comprise one or more ultrasonic transmitters and the ultrasonic receiver is intended to comprise one or more ultrasonic receivers.
  • the number of ultrasonic transmitters and the number of ultrasonic receivers may or may not be equal, which is not limited in this embodiment of the present application.
  • FIG. 7 is a schematic flowchart of a method for preventing accidental touch provided by an embodiment of the present application. As shown in Figure 7, the method may include:
  • the electronic device transmits ultrasonic signals N times, wherein N is greater than or equal to 2, and N is a positive integer.
  • the electronic device may intermittently transmit ultrasonic signals to the surroundings through the ultrasonic transmitter, that is, the electronic device may transmit multiple ultrasonic signals at intervals; wherein, one ultrasonic signal may include multiple ultrasonic signals.
  • This application does not impose any restrictions on the type of ultrasonic transmitter used, the frequency, direction, intensity, etc. of the transmitted ultrasonic waves, which can be adjusted according to the actual situation, and will not be repeated here.
  • the electronic device receives N times ultrasonic echo signals.
  • a primary ultrasonic echo signal may include a plurality of ultrasonic echo signals.
  • the electronic device obtains first data according to each received ultrasonic echo signal.
  • the first data of each received ultrasonic echo signal may include signal strengths, propagation times, etc. of multiple ultrasonic echo signals.
  • the first data of the signal may be an impulse response graph of the ultrasonic echo signal.
  • the electronic device can acquire the impulse response graph of the N times ultrasonic echo signals, and combine the first data of the N times ultrasonic echo signals into a first image.
  • N is 10
  • each of the a and b in FIG. 8 is a first image generated by impulse response maps of 10 frames of ultrasonic echo signals.
  • the first image is composed of a plurality of first pixels.
  • the abscissa represents the number of frames, that is, the number of times to collect the received ultrasonic echo signals; the ordinate represents the ranging, that is, the relative distance of the ultrasonic signal from transmission to reception; the color of each first pixel point
  • the value represents the signal strength of an ultrasonic echo signal, and different color values represent different signal strengths of the ultrasonic echo signal.
  • the lighter the color the greater the signal strength of the ultrasonic echo signal.
  • picture a in FIG. 8 is a first image collected when the mobile phone is in a leather bag
  • picture b in FIG. 8 is a first image collected when the top of the mobile phone is unobstructed.
  • the electronic device inputs the first data of the N times ultrasonic echo signals into the first classification model to obtain the first scene type.
  • the first classification model is a credible model obtained by training the first training model based on the machine learning algorithm and using the first training data. How the first classification model is trained will be explained later, and will not be repeated here.
  • the first scene type may include two types of electronic equipment being blocked and electronic equipment not being blocked, and may also be more subdivided scene types, such as leather bag blocking scene, canvas bag blocking scene, cotton shirt pocket blocking scene, jeans pocket scene.
  • scene types such as leather bag blocking scene, canvas bag blocking scene, cotton shirt pocket blocking scene, jeans pocket scene.
  • a convolutional neural network (convolutional neural networks, CNN) algorithm may be used to perform feature extraction on the first data of the collected N times ultrasonic echo signals, that is, the first image, to obtain the first feature data, and then The first feature data is input into the first classification model.
  • CNN convolutional neural networks
  • the convolutional neural network algorithm mainly has two operators, one is the convolutional layer and the other is the pooling layer.
  • Convolutional layers can be used to extract features, and pooling layers can be used to reduce the number of parameters.
  • the convolution kernel is used to extract features.
  • the convolution kernel can be a matrix.
  • the convolution layer can perform convolution operations within the sliding window by sliding a sliding window to extract image features at different positions. .
  • the output of the convolutional layer is input to the pooling layer.
  • Commonly used pooling can be maximum pooling and average pooling. Maximum pooling is to extract the most obvious features. Average pooling is to consider each pixel and extract the average feature.
  • the pooling layer also slides a sliding window, and takes the maximum value or average value within the sliding window.
  • the first image generated by a certain ultrasonic echo signal is used as the input image and input into the convolutional neural network algorithm.
  • the convolutional neural network algorithm can perform two-layer convolution on the first image.
  • the feature extraction of the layered layer finally obtains the first feature data.
  • first input the input image into the convolutional layer with pooling layer perform the first rough detection on the input image, extract the approximate positions of the feature points of the input image, and obtain the first level Level 1 feature map;
  • the feature map is then input to the convolutional layer with a pooling layer, and the predicted feature points of the Level 1 feature map are taken as the center to re-extract more accurate feature point positions to obtain the second-level Level 2 feature map; then the Level 2 feature map passes through
  • the fully connected layer can combine the features corresponding to each image to output the first feature data; the finally generated first feature data is input into the first classification model.
  • step S105 determine whether the electronic device is in a blocked state according to the first scene type, if so, go to step S106, if not, do nothing, continue to send and receive ultrasonic signals, that is, go to step S101.
  • the determination result may be directly output as that the electronic device is blocked or the electronic device is not blocked.
  • the first scene type is a plurality of subdivided scene types, such as a leather bag occlusion scene, a canvas bag occlusion scene, a cotton shirt pocket occlusion scene, a jeans pocket occlusion scene, a palm occlusion scene, a book occlusion scene, a hair occlusion scene, etc., then It is necessary to determine whether the electronic device is in a blocked state according to the detected specific scene type.
  • the result of the first scene type detected by the ultrasonic sensor may also be combined with the scene type detected by other sensors, such as a gravity sensor, a gyroscope sensor, and an ambient light sensor, for coupling judgment.
  • the processor of the electronic device can couple and analyze the reported results of the ambient light sensor and the ultrasonic sensor. For example, although the ultrasonic sensor detects that the electronic device is blocked, the ambient light sensor detects that the brightness of the ambient light is higher than a certain brightness value.
  • the anti-mistouch function is still not turned on; or when the ambient light sensor detects that the ambient light brightness is lower than a certain brightness value, and the ultrasonic sensor detects that the current electronic device is blocked, the electronic device will turn on the anti-mistouch function, etc. .
  • the processor of the electronic device can also perform coupling analysis on the reported results of the proximity light sensor and the ultrasonic sensor. For example, when either the proximity light sensor or the ultrasonic sensor detects that the current electronic device is blocked, the electronic device turns on the anti-mistouch function. ; or when both the proximity light sensor and the ultrasonic sensor detect that the current electronic device is in a blocked state, the electronic device will turn on the anti-mistouch function, etc. It can be understood that in other embodiments of the present application, other sensors can also be combined or other coupling logics can be used. It is sufficient to achieve the purpose set forth in this application.
  • the electronic device enables the anti-mistouch function.
  • the anti-mistouch function may be that the electronic device turns off the screen, the electronic device does not respond to screen unlocking (which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), and the electronic device does not respond to raising a hand
  • screen unlocking which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.
  • the electronic device does not respond to raising a hand
  • the electronic device does not respond to raising the hand to answer the incoming call
  • the electronic device does not respond to the fingerprint to answer the incoming call
  • the electronic device turns off AOD and other functions.
  • the anti-accidental touch function of the electronic device may be interrupted.
  • the electronic device can be set not to activate the anti-mistouch function, so even if the electronic device is detected to be blocked, for example, when the user plays games on the horizontal screen of the mobile phone, the hand is blocked. Going to the top of the electronic device does not activate the automatic screen off either. Or the user can set to turn on or off the anti-mistouch function by himself, or set one or more applications to turn on or off the anti-mistouch function by himself.
  • the following describes how the first classification model is trained.
  • the first classification model is a credible model obtained by training the first training model using the first training data based on a machine learning classification algorithm.
  • the first training data includes multiple sample data
  • the multiple sample data is sample data obtained in multiple scenarios
  • one sample data includes first sample data of N times ultrasonic echo signals in a known scenario and The first sample scene type for this scene.
  • the first sample scene type is a known scene type.
  • a plurality of first sample data constitute a first sample data vector, which corresponds to a first sample scene type label formed by a plurality of first sample scene types.
  • the sample data can be divided into two parts, one part of the sample data is used to train the model, and the other part of the sample data can be used to test the accuracy of the model.
  • a first sample data is a second image generated by the second data of N times of ultrasonic echo signals generated by transmitting N times of ultrasonic signals, and the second data of each received ultrasonic echo signal may include a plurality of The signal strength, propagation time, etc. of the ultrasonic echo signal.
  • the second image is composed of a plurality of second pixels.
  • the abscissa represents the number of frames, that is, the number of times of collecting the received ultrasonic echo signals, and the ordinate represents ranging, that is, the ultrasonic signal is from The relative distance from transmission to reception, the color value of each second pixel represents the signal strength of an ultrasonic echo signal, and different color values represent different signal intensities of the ultrasonic echo signal. In the example of Figure 8, the color The shallower the signal strength of the ultrasonic echo signal is.
  • the first sample scene type may include two types of electronic equipment being blocked and electronic equipment not being blocked, and may also be more subdivided scene types, such as a leather bag blocking scene, a canvas bag blocking scene, a cotton shirt pocket blocking scene,
  • the embodiments of the present application do not limit the situations in which the jeans pocket occlusion scene, the palm occlusion scene, the book occlusion scene, the hair occlusion scene, and the like are blocked by various occluders.
  • the machine learning classification models used in the training process may include but are not limited to: extreme gradient boosting (XGBoost) model, neural network (NN) model, gradient boosting decision tree (GBDT) ) model, random forest (RF) model, etc.
  • XGBoost extreme gradient boosting
  • NN neural network
  • GBDT gradient boosting decision tree
  • RF random forest
  • an XGBoost model can be used to train to obtain a credible first classification model.
  • the XGBoost model is an integrated machine learning model that uses the gradient boosting framework and is based on a decision tree. It can be composed of multiple decision trees.
  • the decision tree here is a classification and regression tree (CART), CART
  • CART classification and regression tree
  • the decision is a binary tree, and the values of the internal node features are "Yes” and "No".
  • the branch with the value of "Yes” for each node can be used as the left branch of the node, and the value of "No” can be used. ” as the right branch of the node; the basic idea of the XGBoost model is to gradually build multiple decision trees according to the characteristics of the samples.
  • a training model also called a weak classifier
  • the first sample data vector is input into the weak classifier to obtain a sample recognition result, if the sample recognition result is the same as the first If the scene type label does not match, it indicates that the current weak classifier needs to be iterated.
  • the specific iterative process can be understood as adjusting the value of the weak classifier according to the residual between the sample identification result and the scene type label of the first sample.
  • the first classification model is credible, and its confidence level can be 95% or 98% in the test, which can actually be adjusted according to specific needs.
  • FIG. 10 shows a block diagram of functional modules of an electronic device for preventing accidental touches provided by an embodiment of the present application.
  • the functional modules of the electronic device can be implemented by hardware, software or a combination of hardware and software to implement the solution of the present application.
  • the functional modules described in FIG. 10 may be combined or separated into several sub-blocks to implement the scheme of the present application. Accordingly, what is described above in this application may support any possible combination or separation or further definition of the functional modules described below.
  • the electronic device includes an ultrasonic transmitter and an ultrasonic receiver, and the ultrasonic transmitter and the ultrasonic receiver may be disposed on the top of the electronic device, or may be located at other positions, which are not limited in this embodiment.
  • the ultrasonic transmitter and the ultrasonic receiver are intended to cover any electronic device capable of transmitting and receiving ultrasonic waves, and are not limited to the ultrasonic transmitter and ultrasonic receiver in the narrow sense.
  • the ultrasonic transmitter and ultrasonic receiver can be centralized on the same device, or they can be separated.
  • the number of ultrasonic transmitters or ultrasonic receivers may be one or more.
  • the ultrasonic transmitter is intended to comprise one or more ultrasonic transmitters and the ultrasonic receiver is intended to comprise one or more ultrasonic receivers.
  • the number of ultrasonic transmitters and the number of ultrasonic receivers may or may not be equal, which is not limited in this embodiment of the present application.
  • the electronic device may specifically include: an ultrasonic signal transmitting module, an ultrasonic echo signal receiving module, a signal feature extraction module, a scene classification module, a false-touch prevention function enabling module, and an offline model training module.
  • the ultrasonic signal transmitting module is used for transmitting ultrasonic signals for N times, wherein N is greater than or equal to 2, and N is a positive integer.
  • the ultrasonic signal transmitting module can intermittently transmit ultrasonic signals to the surroundings through the ultrasonic transmitter, that is, the electronic device can transmit ultrasonic signals multiple times at intervals; wherein, one ultrasonic signal can include multiple ultrasonic signals.
  • This application does not impose any restrictions on the type of ultrasonic transmitter used, the frequency, direction, intensity, etc. of the transmitted ultrasonic waves, which can be adjusted according to the actual situation, and will not be repeated here.
  • the ultrasonic echo signal receiving module is used for receiving N times ultrasonic echo signals.
  • the transmitted ultrasonic signal will generate ultrasonic echo when encountering obstacles.
  • the ultrasonic echo signal receiving module can receive the ultrasonic echo signal through the ultrasonic receiver. , liquid) through propagation, attenuation, refraction, reflection, diffraction, etc.
  • a primary ultrasonic echo signal may include a plurality of ultrasonic echo signals.
  • the signal feature extraction module is configured to perform feature extraction on the obtained first data of each received ultrasonic echo signal to obtain the first feature data, which is used as the input of the first classification model.
  • the first data of the ultrasonic echo signals received each time may include signal strengths, propagation times, etc. of a plurality of ultrasonic echo signals, and the first data of the ultrasonic echo signals received each time may be A graph of the impulse response of an ultrasonic echo signal.
  • the electronic device may acquire the impulse response graph of the N times ultrasonic echo signals, and combine the first data of the N times ultrasonic echo signals into a first image. For details, refer to step S103 in the foregoing method embodiment.
  • the signal feature extraction module may use a convolutional neural network (convolutional neural networks, CNN) algorithm to perform feature extraction on the first data of the collected N times ultrasonic echo signals, that is, the first image, to obtain the first a feature data, and then input the first feature data into the first classification model.
  • CNN convolutional neural networks
  • the scene classification module is configured to input the first data of the N ultrasonic echo signals into the first classification model to obtain the first scene type.
  • the first classification model is a credible model obtained by training the first training model using the first training data based on the machine learning algorithm.
  • the first scene type may include two types of electronic equipment being blocked and electronic equipment not being blocked, and may also be more subdivided scene types, such as leather bag blocking scene, canvas bag blocking scene, cotton shirt pocket blocking scene, jeans pocket scene.
  • scene types such as leather bag blocking scene, canvas bag blocking scene, cotton shirt pocket blocking scene, jeans pocket scene.
  • the scene classification module is further configured to determine whether the electronic device is in a blocked state according to the first scene type. If the first scene type is classified into two types: the electronic device is blocked and the electronic device is not blocked, the scene classification module can directly output the determination result that the electronic device is blocked or the electronic device is not blocked. If the first scene type is a plurality of subdivided scene types, such as a leather bag occlusion scene, a canvas bag occlusion scene, a cotton shirt pocket occlusion scene, a jeans pocket occlusion scene, a palm occlusion scene, a book occlusion scene, a hair occlusion scene, etc., then The scene classification module needs to determine whether the electronic device is in a blocked state according to the detected specific scene type.
  • the scene classification module may further combine the results of the first scene type detected by the ultrasonic sensor with the scene types detected by other sensors to perform coupling judgment, such as a gravity sensor, a gyroscope sensor, an ambient light sensor, and a proximity light sensor. Wait.
  • the scene classification module can perform coupling analysis on the reported results of the ambient light sensor and the ultrasonic sensor.
  • the anti-mistouch function is still not turned on; or when the ambient light sensor detects that the ambient light brightness is lower than a certain brightness value, and the ultrasonic sensor detects that the current electronic device is blocked, the electronic device will turn on the anti-mistouch function, etc.
  • the scene classification module may also perform coupling analysis on the reported results of the proximity light sensor and the ultrasonic sensor. For example, when either the proximity light sensor or the ultrasonic sensor detects that the current electronic device is in a blocked state, the electronic device is turned on. Anti-mistouch function; or when both the proximity light sensor and the ultrasonic sensor detect that the current electronic device is in a blocked state, the electronic device will turn on the anti-mistouch function, etc. It can be understood that in other embodiments of the present application, other sensors can also be combined or other coupling logics can be used. It is sufficient to achieve the purpose set forth in this application.
  • the anti-mistouch function enabling module is used to automatically turn on the anti-mistouch function when it is detected that the electronic device is in a blocked state.
  • the anti-mistouch function may be that the electronic device turns off the screen, the electronic device does not respond to screen unlocking (which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), and the electronic device does not respond to raising a hand
  • screen unlocking which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.
  • the electronic device does not respond to raising a hand
  • the electronic device does not respond to raising the hand to answer the incoming call
  • the electronic device does not respond to the fingerprint to answer the incoming call
  • the electronic device turns off AOD and other functions.
  • the anti-accidental touch function of the electronic device may be interrupted.
  • the electronic device can be set to not activate the anti-mistouch function, so even if the electronic device is detected to be blocked, for example, when the user plays games on the horizontal screen of the mobile phone, the hand is blocked. Going to the top of the electronic device does not activate the automatic screen off either. Or the user can set to turn on or off the anti-mistouch function by himself, or set one or more applications to turn on or off the anti-mistouch function by himself.
  • the electronic device can accurately detect whether it is currently in a blocked state, and when it is determined that the electronic device is currently blocked in a pocket or a backpack, the electronic device can automatically activate the anti-mistouch function, such as The screen of the electronic device is turned off, the screen does not respond to unlocking (which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.), does not respond to raising the hand to brighten the screen, does not respond to raising the hand to answer incoming calls, turns off AOD, etc.
  • unlocking which may include sliding up the screen to unlock, fingerprint unlocking, gesture unlocking, face recognition unlocking, etc.
  • ultrasonic sensors instead of optical proximity sensors to achieve the anti-mistouch function can also reduce the number of electronic devices in electronic devices and save the front opening of the screen, narrow the frame space of electronic devices, increase the screen ratio of electronic devices, and improve electronic devices. Dust and water resistance, etc.
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Environmental & Geological Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Telephone Function (AREA)

Abstract

La présente invention concerne un procédé de prévention d'un toucher accidentel et un dispositif électronique, pour utilisation dans la résolution du problème en ce que le dispositif électronique est involontairement touché pendant l'utilisation. Grâce à l'utilisation des modes de réalisation de la présente invention, le dispositif électronique peut détecter avec précision s'il est actuellement dans un état protégé, lorsque le dispositif électronique est déterminé comme étant dans un scénario protégé tel qu'il se trouve dans une poche ou un sac à dos, le dispositif électronique active automatiquement une fonction de prévention de toucher accidentel, tel que le dispositif électronique éteignant l'écran, ne répondant pas à un déverrouillage d'écran, ne répondant pas à un réveil par soulèvement, ne répondant pas à la prise d'appel entrant par soulèvement, et désactivant l'AOD, de façon à éviter l'occurrence d'un toucher accidentel, réduire la consommation électrique du dispositif électronique, de façon à fournir à un utilisateur un environnement de fonctionnement convivial, et améliorer l'expérience de l'utilisateur. De plus, un capteur ultrasonore est utilisé à la place d'un capteur de proximité optique pour mettre en œuvre la fonction de prévention de toucher accidentel, le nombre de composants électroniques du dispositif électronique est réduit, une ouverture avant sur l'écran est évitée, l'espace de cadre de bord du dispositif électronique est rétréci, le rapport écran-corps du dispositif électronique est augmenté, et la résistance à la poussière et la résistance à l'eau du dispositif électronique sont augmentées.
PCT/CN2021/109122 2020-07-31 2021-07-29 Procédé de prévention de toucher accidentel et dispositif électronique WO2022022609A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010762072.5A CN114070928B (zh) 2020-07-31 2020-07-31 一种防误触的方法及电子设备
CN202010762072.5 2020-07-31

Publications (1)

Publication Number Publication Date
WO2022022609A1 true WO2022022609A1 (fr) 2022-02-03

Family

ID=80037230

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/109122 WO2022022609A1 (fr) 2020-07-31 2021-07-29 Procédé de prévention de toucher accidentel et dispositif électronique

Country Status (2)

Country Link
CN (1) CN114070928B (fr)
WO (1) WO2022022609A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115009222A (zh) * 2022-05-26 2022-09-06 支付宝(杭州)信息技术有限公司 一种无感入车方法及装置
CN115236072A (zh) * 2022-06-14 2022-10-25 杰能科世智能安全科技(杭州)有限公司 一种升降柱状态检测方法及其装置
CN115314590A (zh) * 2022-06-21 2022-11-08 华为技术有限公司 接近事件的判定方法和电子设备
CN116661670A (zh) * 2022-11-18 2023-08-29 荣耀终端有限公司 管理手势导航窗口的方法、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103327148A (zh) * 2013-06-28 2013-09-25 惠州Tcl移动通信有限公司 基于移动终端调整屏幕背光亮度的实现方法及移动终端
US20170004806A1 (en) * 2015-07-01 2017-01-05 Thomas Joseph Edwards Method and apparatus to enable smartphones and computer tablet devices to communicate with interactive devices
CN106413060A (zh) * 2016-10-24 2017-02-15 北京小米移动软件有限公司 屏幕状态控制方法及装置
CN107943267A (zh) * 2017-11-21 2018-04-20 北京小米移动软件有限公司 一种控制屏幕的方法和装置
CN111323783A (zh) * 2020-02-27 2020-06-23 Oppo广东移动通信有限公司 场景识别方法、装置、存储介质及电子设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10311570B1 (en) * 2018-12-07 2019-06-04 Sonavista, Inc. Medical image based distortion correction mechanism
CN110300230B (zh) * 2019-07-01 2021-03-19 腾讯科技（深圳）有限公司 应用控制方法、装置、存储介质以及终端
CN111191616A (zh) * 2020-01-02 2020-05-22 广州织点智能科技有限公司 一种人脸遮挡检测方法、装置、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103327148A (zh) * 2013-06-28 2013-09-25 惠州Tcl移动通信有限公司 基于移动终端调整屏幕背光亮度的实现方法及移动终端
US20170004806A1 (en) * 2015-07-01 2017-01-05 Thomas Joseph Edwards Method and apparatus to enable smartphones and computer tablet devices to communicate with interactive devices
CN106413060A (zh) * 2016-10-24 2017-02-15 北京小米移动软件有限公司 屏幕状态控制方法及装置
CN107943267A (zh) * 2017-11-21 2018-04-20 北京小米移动软件有限公司 一种控制屏幕的方法和装置
CN111323783A (zh) * 2020-02-27 2020-06-23 Oppo广东移动通信有限公司 场景识别方法、装置、存储介质及电子设备

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115009222A (zh) * 2022-05-26 2022-09-06 支付宝(杭州)信息技术有限公司 一种无感入车方法及装置
CN115236072A (zh) * 2022-06-14 2022-10-25 杰能科世智能安全科技(杭州)有限公司 一种升降柱状态检测方法及其装置
CN115314590A (zh) * 2022-06-21 2022-11-08 华为技术有限公司 接近事件的判定方法和电子设备
CN116661670A (zh) * 2022-11-18 2023-08-29 荣耀终端有限公司 管理手势导航窗口的方法、电子设备及存储介质
CN116661670B (zh) * 2022-11-18 2024-03-08 荣耀终端有限公司 管理手势导航窗口的方法、电子设备及存储介质

Also Published As

Publication number Publication date
CN114070928A (zh) 2022-02-18
CN114070928B (zh) 2022-10-18

Similar Documents

Publication Publication Date Title
CN113645351B (zh) 应用界面交互方法、电子设备和计算机可读存储介质
WO2021000807A1 (fr) Procédé et appareil de traitement pour un scénario d'attente dans une application
CN110114747B (zh) 一种通知处理方法及电子设备
WO2022022609A1 (fr) Procédé de prévention de toucher accidentel et dispositif électronique
WO2021036770A1 (fr) Procédé de traitement d'écran partagé et dispositif terminal
CN111543042A (zh) 通知消息的处理方法及电子设备
EP3958106A1 (fr) Procédé d'affichage d'interface et dispositif électronique
WO2021008589A1 (fr) Procédé d'exécution d'application et dispositif électronique
WO2022037726A1 (fr) Procédé d'affichage à écran partagé et dispositif électronique
EP4033339A1 (fr) Procédé d'affichage d'interface utilisateur et dispositif électronique
CN113452945A (zh) 分享应用界面的方法、装置、电子设备及可读存储介质
JP2023500656A (ja) 表示要素の表示方法及び電子機器
CN114995715B (zh) 悬浮球的控制方法和相关装置
CN110058729B (zh) 调节触摸检测的灵敏度的方法和电子设备
WO2022166435A1 (fr) Procédé de partage d'image et dispositif électronique
EP4184321A1 (fr) Procédé d'interaction entre des dispositifs basé une opération de pointage, et dispositif électronique
WO2021238740A1 (fr) Procédé de capture d'écran et dispositif électronique
CN115032640B (zh) 手势识别方法和终端设备
WO2022078116A1 (fr) Procédé de génération d'image à effet de pinceau, procédé et dispositif d'édition d'image et support de stockage
WO2021129453A1 (fr) Procédé de capture d'écran et dispositif associé
WO2023061054A1 (fr) Procédé de commande de geste sans contact, et dispositif électronique associé
WO2022042774A1 (fr) Procédé d'affichage d'image de profil et dispositif électronique
WO2022222702A1 (fr) Procédé de déverrouillage d'écran et dispositif électronique
EP4310645A1 (fr) Procédé et appareil pour activer une fonction dans une application
WO2024139257A1 (fr) Procédé d'affichage d'interfaces de programmes d'application et dispositif électronique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21849138

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21849138

Country of ref document: EP

Kind code of ref document: A1