WO2016165483A1

WO2016165483A1 - Method and terminal for implementing somatosensory function

Info

Publication number: WO2016165483A1
Application number: PCT/CN2016/074280
Authority: WO
Inventors: 陈卉
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-07-03
Filing date: 2016-02-22
Publication date: 2016-10-20
Also published as: CN106325487A

Abstract

Disclosed are a method and system for implementing somatosensory function. The method includes: the modem embedded in the terminal receives the data from the sensor and/or touch screen module in the terminal and carries out an operation processing after receiving a request of activating the somatosensory function from an application processor; the modem sends the processed data to the application processor, and the somatosensory function is implemented by the application processor.

Description

Method and terminal for realizing somatosensory function

Technical field

This application relates to, but is not limited to, the field of communication technology.

Background technique

Somatosensory and gesture functions are the trend of mobile phone sensors in the future. At present, most mobile phones on the market have corresponding body and gesture functions. This function requires the terminal device to perceive the change of the posture of the human body and the change operation of the gesture regardless of the state. Currently, mainstream body sensations mainly include basic class gestures, including pick up, shake, turning, and sports-related gestures, including walking, running, and riding. (biking) and climbing, etc.; gestures include a touch class and an empty operation class, the former device can recognize the figure drawn by the human hand on the touch screen, and the latter device recognizes the person's gesture on the premise that the human hand needs to touch the touch screen.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

In the process of implementing and using the somatosensory and gesture functions, you will encounter the following problems:

Since the sensor needs to work normally when the mobile phone enters the sleep standby mode normally, and it needs a certain ability to process the sensor data, this process requires a lot of resources, so power consumption is a primary problem.

In order to save the power consumption of the whole machine, many terminal manufacturers will use an external MCU (Micro Control Unit) or Qualcomm's built-in ADSP (advanced digital signal processor) as the Sensor Hub ( Sensor centralized controller) to solve the problem of saving AP (Application Processor) resources and reducing power consumption. The Sensor hub is a functional module dedicated to processing sensors.

Although, through external or built-in ADSP to solve the problem of large power consumption of the whole machine, but plus The way of the Sensor Hub will increase the cost of hardware and software, and the built-in ADSP solution is mainly tied to Qualcomm's solution, which has certain limitations on the flexibility of software and hardware development and the customization of requirements.

This document provides a method and terminal for implementing somatosensory functions to reduce power consumption.

A method of implementing a somatosensory function, comprising:

After receiving the request for opening the somatosensory function of the application processor, the modem built in the terminal receives the data of the sensor and/or the touch screen module in the terminal, and performs arithmetic processing;

The modem passes the processed data to the application processor, and the application processor implements a somatosensory function.

Optionally, the foregoing method further has the following feature: after receiving the request of the application processor to enable the somatosensory function, the modem further includes:

The modem registers a wake-up clock;

The modem is woken up when the wake-up clock passes a specified time;

The receiving data of the sensor and/or the touch screen module within the terminal includes the modem beginning to receive data of sensors and/or touch screen modules within the terminal after being woken up.

Optionally, the foregoing method further has the following features:

The modem stores the processed data, waits for the next wake up, or waits for the application processor to wake up to pass the stored data to the application processor.

Optionally, the foregoing method further has the following features:

If the application processor registers a specified event that wakes up the application processor, the modem wakes up the application processor after processing the data corresponding to the specified event.

Optionally, the above method also has the following features:

The sensor is one or more.

A terminal includes an application processor, a sensor and/or a touch screen module, and a modem, wherein

The modem includes an interaction unit and a processing unit:

The interaction unit is configured to: interact with the application processor, and receive the application office The request for opening the somatosensory function of the processor, and transferring the processed data of the processing unit to the application processor;

The processing unit is configured to: receive data of a sensor and/or a touch screen module in the terminal, and perform an operation process;

The application processor is configured to implement a somatosensory function by using the received data.

Optionally, the foregoing terminal further has the following features:

The processing unit is further configured to: register a wake-up clock after the interaction unit receives the request for the open sensory function of the application processor;

The processing unit is configured to wake up when the wake-up clock passes a specified time, and start receiving data of the sensor and/or the touch screen module in the terminal after being woken up.

Optionally, the foregoing terminal further has the following features:

The storage unit is configured to store the data processed by the processing unit.

Optionally, the foregoing terminal further has the following features:

The application processor is further configured to: register a specified event that wakes up the application processor;

The processing unit is further configured to wake up the application processor after processing the data corresponding to the specified event.

A computer readable storage medium storing computer executable instructions for performing the method of any of the above.

In summary, the embodiments of the present invention provide a method and a terminal for implementing a somatosensory function, which can save power consumption and save hardware and software costs.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

1 is a schematic diagram of a terminal according to an embodiment of the present invention;

2 is a schematic diagram of a Modem according to an embodiment of the present invention;

3 is a block diagram of an internal unit of a Modem side Sensor-Unit module according to an embodiment of the present invention;

4 is a flowchart of a method for implementing a somatosensory function according to an embodiment of the present invention;

FIG. 5 is an overall flow chart of a method for realizing a body feeling according to an embodiment of the present invention.

Embodiments of the invention

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

1 is a schematic diagram of a terminal according to an embodiment of the present invention. The terminal in this embodiment includes multiple sensors, a touch screen module, and the like, as follows:

The acceleration sensor module 101 is configured to: detect acceleration of the terminal device in the X, Y, and Z axis directions. Acceleration is used for a wide range of basic somatosensory actions, such as flipping, shaking, and for step counting functions. The acceleration data is a set of three x, y, z values in three axes. The three-axis data of acceleration can be combined with the three-axis data of the geomagnet and the three-axis data of the gyroscope.

The gyro sensor module 102 is configured to detect an angular change of the terminal device in a certain direction. Gyro data can be used as the input data of the tilt function. At the same time, the gyroscope data can be used for data fusion, GPS (Global Positioning System) assisted navigation, and human motion trajectory. The tilt function is mainly used for tilt detection. After the terminal detects the tilting motion of the tilt, the terminal needs to perform subsequent requirements. For example, the tilt of the terminal can be detected to wake up the lighting screen, and the change of the human body from the sitting posture to the standing posture can be detected.

The geomagnetic sensor module 103 is configured to detect a change in the magnetic field around the terminal device. The biggest use of geomagnets is the compass application. 9-axis data can be combined with acceleration and gyroscope data.

The proximity light sensor module 104 is a sensor that is integrated with the proximity sensor and the light sensor. The proximity sensor can detect the distance between the terminal device and the human body during the call; the light sensor can detect the intensity of the ambient light currently occupied by the terminal device.

The pressure sensor module 105 is configured to: detect the atmospheric pressure of the environment in which the terminal device is located. The data of the pressure gauge can be converted into a relative height value, which can be used to detect the current height of the human body, and is used for step detection when climbing a mountain or climbing a building. Pressure gauge data can also be correlated with acceleration, gyroscopes and geomagnetism The data of the meter is combined with 10-axis data, which can be used for the relatively new indoor navigation and indoor motion track detection functions.

Touch screen module 106: set to: implement a gesture function.

Sensor-Unit Module 107: The Modem side handles the sensor unit module, which is configured to: maintain, calculate, and store sensor data and complete interaction with the AP side, which is a core part of an embodiment of the present invention. Once the AP side has an event request, the module starts to manage the sensor. At the same time, a clock that periodically wakes up the modem is registered internally to implement data interaction between the modem and the sensor.

SMD (Shared Memory Driver) channel 108: is the way in which the AP and its subsystems interact with each other. The data exchange between the AP and the Modem can also be in the SMD mode. The detection processing result on the Modem side can be transmitted to the AP side. .

Sensor Hal (sensor hardware compatibility layer) 109: is the interaction part of the AP side and the Sensor-Unit on the Modem side. This module is set to: request and accept the operation processing result on the Modem side. Set the sampling rate and batch mode of each device of the sensor.

FIG. 2 is a schematic diagram of a Modem according to an embodiment of the present invention. As shown in FIG. 2, the Modem of this embodiment includes an interaction unit 21 and a processing unit 22:

The interaction unit 21 is configured to: interact with the AP, receive a request for opening the somatosensory function of the AP, and transmit the processed data of the processing unit 22 to the AP;

The processing unit 22 is configured to receive data of the sensor and/or the touch screen module in the terminal, and perform arithmetic processing.

The AP is configured to implement a somatosensory function by using the received data.

In an embodiment, the processing unit 22 is further configured to: after the interaction unit 21 receives the request for opening the somatosensory function of the AP, registering a wake-up clock;

The processing unit 22 is configured to wake up when the wake-up clock passes a specified time, and start receiving data of the sensor and/or the touch screen module in the terminal after being woken up.

In an embodiment, the terminal in this embodiment further includes:

The storage unit 23 is configured to store the data processed by the processing unit 22.

In an embodiment, the AP is further configured to: register a specified event that wakes up the AP;

The processing unit is further configured to wake up the AP after processing data corresponding to the specified event.

FIG. 3 is a block diagram of an internal unit of a Modem side Sensor-Unit module according to an embodiment of the present invention, including the following contents:

The sensor driving unit 221 is mainly for completing the loading of the driving of several sensors on the side of the Modem, coordinating and managing the data reported by each sensor, the optimal configuration of the power supply, and the like, and the hardware-related driving part. This section is the lower-level part of the BSP (Board Support Package) layer that is directly associated with the hardware.

IIC (Inter-Integrated Circuit) control unit 222, this part is also a relatively low-level BSP layer, the main function is to cooperate with the sensor driving unit 201 to complete sensor initialization, data reading and sensor control, etc. .

The data operation processing unit 223 mainly includes data processing algorithms, including several axial data fusions, recognition of somatosensory actions, and misjudgment of gestures. As long as the AP has a request for the corresponding event, this part of the operation on the Modem side will perform the algorithm related to the event. The computing power of this unit determines the recognition rate and false positive rate of somatosensory and gestures, and also determines the efficiency of the AP side acquisition action.

The above processing unit 22 can be subdivided into a sensor driving unit 221, an IIC control unit 222, and a data operation processing unit 223.

The storage unit 23, due to power consumption considerations, cannot be frequently awake to interact with the Modem side when the AP side is sleeping. Therefore, the data processed by the data operation processing unit 233 needs to be stored in the storage unit 23. The size of the storage space depends on the actual platform.

Interaction unit 21 with the AP:

This part mainly completes the interaction with the Sensor Hal layer of the AP side, and feeds back the corresponding operation result of the requested event to the AP.

In this embodiment, not only the AP can stand by, but the Modem can also enter the sleep normally. Because the Modem can be periodically woken up to process the sensor data, the Modem wake-up is much less expensive than the AP wake-up.

FIG. 4 is a flowchart of a method for implementing a somatosensory function according to an embodiment of the present invention. As shown in FIG. 4, the method in this embodiment includes:

Step 11: After receiving the request for opening the somatosensory function of the AP, the modem built in the terminal receives the data of the sensor and/or the touch screen module in the terminal, and performs arithmetic processing;

Step 12: The modem delivers the processed data to the AP, and the AP implements a somatosensory function.

FIG. 5 is an overall flowchart of a method for implementing a somatosensory according to an embodiment of the present invention, including the following steps:

Step 301: The AP side registers the somatosensory or gesture event required for monitoring through the Sensor Hal layer, and sets the adoption rate and the reporting rate while monitoring. The AP side transmits the event information to the Modem side.

Step 302: After receiving the event of step 301, the Modem side registers a clock that can wake up the Modem, and each wake-up time can be adjusted according to the size of the standby power consumption. After the wake-up clock is registered, the modem side performs step 303.

Step 303: According to the wake-up time defined in step 302, the Modem side is woken up, and the data of the sensor is read by the IIC control unit, and the data is data of raw data, if some devices support FIFO ( First In First Out (first in, first out), the data is reported to the Modem side according to a certain reporting rate.

Step 304: The sensor operation and storage unit on the Modem side is calculated according to a certain algorithm, and the raw data value reported in step 302 is used, and the operation processing result is given corresponding to the event in step 301, and is stored. At the same time, waiting for the next wake up, go to step 303.

Step 305: After the AP is awake, the Modem side will transmit data to the AP side through the SMD in batches to implement data interaction with the AP. In step 301, if the AP registers an event that needs to wake up the AP, the Modem side processes the value of the corresponding event, and then wakes up the AP to implement the function, so as to avoid losing the event data when the AP sleeps.

Step 306: After the required value is passed to the AP through step 305, the Sensor's Sensor Hal layer passes the result to the framework and hands it to the upper layer for processing. If the AP needs to monitor again, it needs to perform step 301 again.

The method and apparatus proposed by the embodiments of the present invention can be widely applied to various terminal devices. Than Such as mobile phones, desktop phones, and so on.

One of ordinary skill in the art will appreciate that all or a portion of the steps of the above-described embodiments can be implemented using a computer program flow, which can be stored in a computer readable storage medium, such as on a corresponding hardware platform (eg, The system, device, device, device, etc. are executed, and when executed, include one or a combination of the steps of the method embodiments.

Alternatively, all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.

The devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.

When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. The above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Industrial applicability

A method for processing a somatosensory action using a Modem module is provided in an embodiment of the present invention, which can save power consumption and save hardware and software costs. Embodiments of the present invention can be used in projects that require more customization at low cost and somatosensory. The implementation of the somatosensory and gesture functions of the embodiment of the present invention satisfies the user's dynamic and health experience for the mobile terminal, and can also be applied to the design of the wearable device.

Claims

A method of implementing a somatosensory function, comprising:

After receiving the request for opening the somatosensory function of the application processor, the modem built in the terminal receives the data of the sensor and/or the touch screen module in the terminal, and performs arithmetic processing;

The modem passes the processed data to the application processor, and the application processor implements a somatosensory function.
The method of claim 1, wherein: after the modem receives the request to turn on the somatosensory function of the application processor, the method further includes:

The modem registers a wake-up clock;

The modem is woken up when the wake-up clock passes a specified time;

The receiving data of the sensor and/or the touch screen module within the terminal includes the modem beginning to receive data of sensors and/or touch screen modules within the terminal after being woken up.
The method of claim 2 further comprising:

The modem stores the processed data, waits for the next wake up, or waits for the application processor to wake up to pass the stored data to the application processor.
The method of claim 3 further comprising:

If the application processor registers a specified event that wakes up the application processor, the modem wakes up the application processor after processing the data corresponding to the specified event.
The method of claim 1 wherein: said sensor is one or more.
A terminal comprising an application processor, a sensor and/or a touch screen module, further comprising a modem, the modem comprising an interaction unit and a processing unit, wherein:

The interaction unit is configured to: interact with the application processor, receive a request for opening the somatosensory function of the application processor, and deliver the processed data of the processing unit to the application processor;

The processing unit is configured to: receive data of a sensor and/or a touch screen module in the terminal, and perform an operation process;

The application processor is configured to implement a somatosensory function by using the received data.
The terminal of claim 6 wherein:

The processing unit is further configured to: register a wake-up clock after the interaction unit receives the request for the open sensory function of the application processor;

The processing unit is configured to wake up when the wake-up clock passes a specified time, and start receiving data of the sensor and/or the touch screen module in the terminal after being woken up.
The terminal of claim 7, further comprising:

The storage unit is configured to store the data processed by the processing unit.
The terminal of claim 8 wherein:

The application processor is further configured to: register a specified event that wakes up the application processor;

The processing unit is further configured to wake up the application processor after processing the data corresponding to the specified event.
A computer readable storage medium storing computer executable instructions for performing the method of any of claims 1-5.