TWI474001B - Methods, non-transitory storage devices, and fyftem of motion-based device operations - Google Patents

Description

Method for operating a motion-based device, non-transitory storage device and system

The present invention is generally directed to motion-based operations of mobile devices.

The mobile device can include a motion sensor configured to detect motion of the mobile device. The motion sensor measures the movement and rotation of the mobile device in a two- or three-dimensional space and provides a series of readings of the acceleration as an output. Based on the acceleration reading, the mobile device can determine if the device is in motion. The mobile device can use motion to control various functions or applications of the mobile device. For example, the mobile device can use the series of readings as input to the application. Based on motion sensor readings, the application can perform a variety of tasks.

The present invention describes methods, program products and systems for the operation of motion-based devices. The mobile device coordinates the operation of the motion sensor and the proximity sensor. The mobile device can use the motion sensor to determine a gesture event. The mobile device can use the proximity sensor to determine the proximity event. The mobile device may use the gesture action event and the proximity event to confirm each other and determine that the mobile device has moved closer to the target object after the specified gesture action. After confirmation, the mobile device can perform the specified task.

In general, in one aspect, motion-based device operation can include receiving program instructions configured to perform a task even after the mobile device has moved to a location close to an object. Such The operations may include: obtaining motion readings from one or more motion sensors of the mobile device; detecting a gesture event based on the motion readings, comprising determining that the motion readings indicate that the mobile device is moving toward the target object in one or more specified manners; Detecting a proximity event, comprising obtaining a proximity reading from a proximity sensor of the mobile device, the proximity reading indicating that the mobile device is positioned proximate to an object; determining that the mobile device has moved to a target object based on the gesture event and the proximity event The position; and the work to perform the task in response.

Motion-based device operation can be implemented to achieve the following advantages. Compared with the conventional mobile device, the false positive rate of the gesture recognition or the proximity determination can be reduced. When the mobile device moves in a manner similar to the specified gesture, the mobile device can designate the movement as a gesture after confirmation from the proximity sensor. The interrupt movement can be filtered out. Thus, for example, if the user moves the mobile device from the pocket to the ear, the mobile device is required to activate the voice input function until the mobile device reaches the ear.

In addition, the response time of the mobile device can be shortened compared to conventional mobile devices. The proximity sensor of the mobile device may need to be calibrated prior to proximity detection. The mobile device can calibrate the proximity sensor based on the motion sensor reading before the mobile device reaches an object. Thus, when the mobile device reaches the object, the proximity sensor can already be calibrated. If the task requires a proximity sensor input, the proximity sensor can appear to respond almost instantaneously from the user's point of view.

Details of one or more implementations of motion-based device operation are set forth in the accompanying drawings and the following description. Other features, aspects and advantages of motion-based device operation will become self-described, schema and patentable Obvious.

Overview of the operation of motion-based devices

1 is a diagram providing an overview of an exemplary motion-based device operation. Mobile device 100 is a device configured to perform tasks in response to a gesture. The gesture may include picking up the mobile device 100 from the initial position and placing the mobile device 100 in a user action proximate to the face 118.

In the example shown, the initial position of the mobile device 100 is the face up position on the table 102. The mobile device 100 is then picked up from the station and the mobile device 100 is moved to the face following the motion path 104. The mobile device 100 can track the motion path 104 using a motion sensor. The motion sensor can be configured to measure linear acceleration of the mobile device 100 over multiple axes (eg, X, Y, Z or pitch, yaw, roll) Value, angular rate value, or both, and produces a motion sensor reading. The motion sensor readings can include a time series corresponding to the motion vectors of the motion path 104.

Mobile device 100 can include a gesture recognition subsystem 110. The gesture recognition subsystem 110 is a component of the mobile device 100 that is configured to recognize gestures from various motions. For example, the self-initial position can be performed quickly or slowly by the left or right hand with or without interruption (eg, flipping the mobile device 100 for viewing the action of the display prior to placing the mobile device 100 close to the ear) The mobile device 100 is picked up and the mobile device 100 is placed in a gesture close to the face 118. The gesture recognition subsystem 110 can recognize gestures from such changes based on one or more motion patterns. The operation of the schematic action recognition subsystem 110 will be described below with reference to Figures 3 and 4A-4B. Other details.

The mobile device 100 can include a proximity detection subsystem 112. The proximity detection subsystem 112 is a component of the mobile device 100 configured to detect a proximity event. The mobile device 100 can use the proximity event to confirm the identified gesture and reduce false positives. The misjudgment that the movement of the mobile device is erroneously recognized as the occurrence of the designated gesture. For example, when the mobile device is removed from the pocket and placed on a table, the mobile device may erroneously determine that the gesture action of the pick-up mobile device was received. By using one or more other sensors of the mobile device 100, the proximity detection subsystem 112 can detect a false positive when the condition of the mobile device is inconsistent with the normal result of the gesture. For example, the mobile device 100 can identify an erroneous recognition of the gesture of moving the mobile device 100 to the face 118 when the proximity sensor fails to detect the face 118 after the move. The mobile device 100 can confirm the gesture action when the proximity detection subsystem 112 detects the proximity event shortly before or shortly after the gesture recognition subsystem 110 recognizes the gesture action (eg, at time 116).

Likewise, the mobile device 100 can use the gestures detected by the gesture recognition subsystem 110 to confirm the proximity event. For example, the mobile device 100 can detect a proximity event when the user places the hand close to the proximity sensor of the mobile device 100 or places the mobile device close to the ear. Based on the gesture action determined by the gesture recognition subsystem 110, the mobile device 100 can determine whether the proximity event is caused by the user placing the mobile device 100 close to the ear. The mobile device 100 can close the display screen approximately simultaneously when a gesture is placed to approach the ear. Therefore, the mobile device 100 can prevent the user from merely placing the hand close to the display firefly. The display screen is closed during the curtain.

After the confirmation of the gesture, the mobile device 100 can perform various tasks. For example, the mobile device 100 can disable the touch screen input device, or activate the voice input device, or both. Based on the characteristics of the confirmed gesture input, the mobile device 100 can set the voice input device to various input modes.

2 is a block diagram of an illustrative mobile device 100 configured to perform motion-based operations. The mobile device 100 can include a motion sensor 202. The motion sensor 202 can continuously monitor motion of the mobile device 100 (including linear acceleration, or angular rate, or both) and generate motion sensor readings 206. The mobile device 100 can include a schematic motion recognition subsystem 110 configured to receive motion sensor readings 206 and generate an identified gesture 220. The recognized gesture action 220 can include an identifier for the gesture (eg, "pick up" or "drop"). In some implementations, the identified gesture action 220 can be associated with a timestamp. The time stamp may indicate the start time of the gesture, the end of the gesture, or any time between them.

The mobile device 100 can include a proximity detection subsystem 112 that is configured to confirm the identified gesture 220. If the identified gesture 220 is a schematic action that typically causes a proximity between the mobile device 100 and the object, the proximity detection subsystem 112 can configure the proximity sensor 222 to detect the proximity. The proximity sensor 222 can include components of the mobile device 100 configured to detect the presence of the object without physical contact between the mobile device 100 and a nearby item. When the proximity detection subsystem 112 receives the identified gesture 220, the proximity detection subsystem 112 can operate the proximity sensor from the passive mode. Change to active mode. In active mode, the proximity sensor can detect nearby objects (eg, face 118) and produce a proximity output 224. The proximity output 224 can be a binary value (eg, "yes" or "no"), or a scale indicating the likelihood that the mobile device 100 is in proximity to the object or indicating the distance between the mobile device and the object.

Based on the proximity output 224, the proximity detection subsystem 112 can determine whether to acknowledge the identified gesture 220. If the proximity detection subsystem 112 confirms the identified gesture 220, the proximity detection subsystem 112 can notify the interface 226. Interface 226 can include a system function or an application programming interface (API) of an application. After receiving the confirmation via interface 226, the system function or application can perform the task based on the identified gesture action 220.

An exemplary schematic action recognition subsystem operation

3 is a block diagram illustrating an exemplary system for illustrating motion recognition. The system for gesture recognition may include a motion sensor 202 and a schematic motion recognition subsystem 110. The gesture recognition subsystem 110 can be configured to receive and process motion sensor readings 206. The gesture recognition subsystem 110 can include a dynamic screening subsystem 308. The dynamic screening subsystem 308 is a component that illustrates the configuration of the motion recognition subsystem 110 to perform dynamic screening of the motion sensor readings 206. Dynamic screening subsystem 308 can high pass screen each of motion sensor readings 206. The dynamic screening subsystem 308 can reduce the time dimension of the time series of motion vectors in the motion sensor readings 206.

In some implementations, the dynamic screening subsystem 308 can apply a screening threshold, which can be at least one of an acceleration value or an angular rate value. If the motion vector V exceeds the screening threshold on at least one axis (eg, axis X) Values, the dynamic screening subsystem 308 can process a series of one or more motion vectors V1...Vi temporally before the motion vector V to generate a new motion vector V' to replace the motion vectors V1...Vi. The dynamic screening subsystem 308 can generate the motion vector V' by calculating the average of the vectors V1...Vi. Thus, dynamic screening subsystem 308 can create normalized motion sensor readings with fewer items in the time series.

Additionally, the dynamic screening subsystem 308 can be configured to select a portion of the motion sensor reading 206 for further processing. This selection can be made based on the sliding time window 310. Motion sensor 202 can continuously generate motion sensor readings 206. The dynamic screening subsystem 308 can use the sliding window 310 to select segments of continuous data and generate normalized motion sensor readings 311 based on the selected segments.

The gesture recognition subsystem 110 can include a motion recognition subsystem 312. The motion recognition subsystem 312 is a component that is configured to identify the motion recognition subsystem 110 to determine whether the normalized motion sensor reading 311 matches a known motion pattern. Motion recognition subsystem 312 can receive normalized motion sensor readings 311 and access motion pattern data store 314. The athletic pattern data store 314 can include a storage device that stores one or more motion patterns 316. Each of the motion patterns 316 can include a series of motion vectors and is associated with a range of influence (SOI) that defines the error limit. The motion recognition subsystem 312 can compare each of the received normalized motion sensor readings 311 with the stored motion patterns 316 and identify the gesture based on the comparison.

Comparing the received normalized motion sensor readings 311 with the stored motion patterns 316 can include distance calculations. Motion recognition subsystem 312 can A distance calculation subsystem 318 is included. The distance calculation subsystem 318 is a component of the motion recognition subsystem 312 configured to calculate the distance between the normalized motion sensor reading 311 and each of the motion patterns 316. If the distance between the normalized motion sensor reading 311 and the motion pattern P is within the radius of the SOI of the motion pattern P, the motion recognition subsystem 312 can identify the match and recognize gesture 220. Motion recognition subsystem 312 can transmit identified gesture action 220 to proximity detection subsystem 112. Additional details of the operation of distance calculation subsystem 318 will be described below with reference to Figures 4A-4B.

4A-4B are diagrams illustrating an exemplary technique for matching motion sensor readings to a motion pattern. 4A illustrates an example data structure of a normalized motion sensor reading 311 as described above with reference to FIG. The normalized motion sensor reading 311 can include a series of motion vectors 402. Each motion vector 402 can include an acceleration reading or angular rates a _x , a _{y ,} and a _z for axes X, Y, and Z, respectively. In some implementations, each motion vector 402 can be associated with a time t _i that defines a time series. In some implementations, the normalized motion sensor reading 311 can use the order of the motion vector 402 to implicitly specify the time dimension of the time series. In such implementations, time can be omitted.

The distance calculation subsystem 318 (as described above with reference to FIG. 3) can compare each of the normalized motion sensor readings 311 with the motion patterns 206a, 206b, and 206c. This comparison can produce a match. If the normalized motion sensor reading 311 matches at least one of the motion patterns 206a, 206b, and 206c, the gesture is tentatively identified. The operation of the comparison is described in further detail below with reference to Figure 4B.

4B is a diagram illustrating the distance calculation operation of the distance calculation subsystem 318. To perform the comparison, the distance calculation subsystem 318 can calculate the distance between the normalized motion sensor reading 311 and the motion pattern (eg, the motion pattern 206a, 206b, or 206c). The distance calculation subsystem 318 can use the directed graph 410 to calculate the distance between the normalized motion sensor reading 311 and the motion pattern using a dynamic time warp technique. For convenience, the normalized motion sensor reading 311 will be designated as R and the motion pattern will be designated as P. The distance between R and P will be specified as D(R, P).

In the example shown, the normalized motion sensor reading 311 can include a time series of m normalized motion sensor readings RV(1) through RV(m). The motion pattern may include a time series of n motion vectors PV(1) through PV(n). In some implementations, the distance calculation subsystem 318 calculates the distance D(R, P) by using the directed graph 410. The directed graph 410 can include m x n nodes. Each node can be associated with a cost. The cost of the node (i, j) can be determined based on the distance between the motion vectors RV(i) and PV(j). The distance can be an Euclidean distance, a Manhattan distance, or any other distance between two vectors in a multidimensional space.

The distance calculation subsystem 318 can add a directed edge from the node (i, j) to the node (i, j+1) and from the node (i, j) to the node (i+1, j). Thus, the directed edges between all nodes can form a grid in which (in this example) multiple paths lead from node (1, 1) to node (m, n).

The distance calculation subsystem 318 can add the source node S and the directed edges from the S to the node (1, 1) and the directed edges of the target node T and the self node (m, n) to T to the directed graph 410. The distance calculation subsystem 318 can determine the shortest path between S and T and specify the cost of the shortest path as the distance D(R, P) between R and P.

In some implementations, the distance calculation subsystem 318 can perform optimizations on the comparison. The distance calculation subsystem 318 can perform optimization by applying the comparison thresholds 412 and 414. The comparison thresholds 412 and 414 can define a series of vector pairs, and the distance calculation subsystem 318 performs distance calculations between the series of vector pairs. By applying the comparison thresholds 412 and 414, the distance calculation subsystem 318 can exclude those calculations that are unlikely to result in a match. For example, the calculation of the distance between the first motion vector RV(1) of the normalized motion sensor reading 311 and the last motion vector PV(n) of the motion pattern is unlikely to result in a match, and thus, can be omitted from the calculation. its.

The distance calculation subsystem 318 can compare the distance D(R, P) with the SOI associated with the motion pattern P. If the distance is within the SOI, the distance calculation subsystem 318 can identify the match. The gesture can be tentatively identified.

FIG. 5 is a block diagram illustrating an exemplary proximity detection subsystem 112 of the mobile device 100. The proximity detection subsystem 112 can be configured to confirm a tentatively identified gesture. The proximity detection subsystem 112 can include a proximity sensor controller 502. The proximity sensor controller 502 is a component of the proximity detection subsystem 112 that can set the proximity sensor 222 to various operational modes based on the identified gesture 220 received from the gesture recognition subsystem 110.

To detect that the mobile device 100 is in proximity to an object, the proximity sensor 222 can emit an electromagnetic or electrostatic field and detect a change in the field. To detect the change, proximity sensor 222 can compare the field reading to the baseline. The baseline may be a reading of an electromagnetic or electrostatic field when no object is in proximity to the proximity sensor. If the offset between the reading and the baseline satisfies a threshold, a proximity event can be detected.

The proximity sensor controller 502 can configure the proximity sensor 222 to operate in a passive mode or an active mode. When the proximity sensor 222 is operating in the passive mode, the proximity detection subsystem 112 can store the representation of the proximity event and (eg, the time stamp of the occurrence of the proximity event) in the proximity event cache 504. The proximity detection subsystem 112 may send a signal to the interface 226 when the time difference between the timestamp of the identified gesture 220 and the timestamp of the occurrence of the proximity event is less than a threshold. When the proximity sensor 222 is operating in the active mode, the proximity detection subsystem 112 can send a signal to the interface 226 when the proximity sensor 222 detects a proximity event.

The proximity sensor controller 502 can set the proximity sensor 222 to the passive mode by default. When the proximity sensor controller 502 receives the recognized gesture 220 from the gesture recognition subsystem 110, the proximity sensor controller 502 can set the proximity sensor 222 to the active mode. When the proximity detection subsystem 112 sends a signal to the interface 226, the proximity sensor controller 502 can set the proximity sensor 222 back to the passive mode. In addition, when the proximity sensor 222 does not detect a proximity event after the proximity sensor has been detected after the proximity sensor 222 is set to the active mode, the proximity sensor controller 502 can connect the proximity sensor 222. Set back to passive mode.

6A-6B are diagrams illustrating a timetable for configuring a proximity sensor. Figure 6A illustrates the conventional operation of the proximity sensor of the mobile device. At time t1 (602), the mobile device can initiate a proximity sensor for requesting a proximity sensor input. The proximity sensor can emit an electromagnetic or electrostatic field. At time t2 (604), the proximity sensor can take sufficient readings to establish a baseline r1 that can be measured relative to the baseline r1. The time taken to get the baseline is (t2-t1). At time 606, the mobile device moves closer to an object. The electromagnetic or electrostatic field changes to r2. At time t3, the proximity sensor can detect the change. The proximity sensor can determine that the offset between r2 and r1 meets the proximity threshold and can determine that the mobile device has moved to the vicinity of the object. Thus, the perceived response time 608 is the time to establish the baseline (t2-t1) plus the time of the proximity detection (t3-t2).

Figure 6B illustrates the motion-based operation of the proximity sensor of the mobile device. At time t1' (622), the mobile device can enter the motion detection mode. The proximity sensor can be turned on and set to operate in passive mode. In passive mode, the proximity sensor establishes a baseline. At time t2' (624), the proximity sensor establishes a baseline r1. The time taken to get the baseline is (t2'-t1'). At time tc (626), the gesture recognition subsystem detects the gesture and after this detection, the gesture configuration system then sets the proximity sensor to the active mode. Subsequently, at time 627, the electromagnetic or electrostatic field changes to r2. At time t3' (628), the motion sensor can determine that the offset between r2 and r1 meets the proximity threshold and determines that the mobile device has moved to a position near an object. The time to establish the baseline (t2'-t1') can be masked by the time it takes for the motion of the mobile device (eg, the time it takes to pick up the mobile device and place the mobile device close to the ear). Thus, the perceived response time 630 can be a proximity detection time (t3'-tc) that can be hundreds of milliseconds shorter than the response time 608 perceived by the conventional mobile device (FIG. 6A).

Exemplary motion-based operations of mobile devices

FIG. 7A is a flow diagram illustrating an exemplary motion-based operation 700 of a mobile device. The mobile device can be the mobile device 100 as described above. The mobile device can receive (702) configured to detect that the mobile device has moved The program instruction is followed by the action device even after the position of the object is approached. The item can include at least a portion of a face. Program instructions may include operating system instructions, application instructions, or both. The program instructions can be configured to operate the mobile device in a touch input mode for receiving touch input or in a voice input mode for receiving voice input.

The mobile device can obtain (704) motion readings from the motion sensor of the mobile device. When the motion sensor from the mobile device obtains a motion reading, the mobile device can determine the proximity baseline of the proximity sensor. Determining the proximity baseline of the proximity sensor of the mobile device can include setting the proximity sensor to a passive mode (where the proximity sensor produces one or more baseline readings); and based on the proximity sensor in the passive mode The resulting baseline reading determines the near baseline.

The mobile device can determine (706) that the motion reading indicates that the mobile device is moving toward the object. Determining the motion reading indicating that the mobile device is moving toward the object can include comparing the motion reading to one or more pre-stored motion patterns, and based on the result of the comparing, determining that the mobile device is moving toward the object in a schematic motion. One or more pre-stored motion patterns can be associated with a gesture action of moving the mobile device toward the object. Each pre-stored motion pattern may correspond to a movement of the gesture.

In response to determining that the motion reading indicates that the mobile device is moving toward the object, the mobile device can obtain (708) a proximity reading from the proximity sensor of the mobile device. Obtaining the proximity reading can include setting the proximity sensor from the passive mode to the active mode, in which the proximity sensor produces one or more readings for comparison with the proximity baseline. Determining the motion reading indicates that the mobile device is moving toward the object Including the start time of the exercise. The proximity sensor is set to the active mode from the passive mode after a specified delay from the start of the motion.

Based on the motion readings and the proximity readings, the mobile device can determine (710) that the mobile device has moved to a location near the object. Determining that the mobile device has moved to a location proximate to the object may include determining that the proximity reading satisfies a specified proximity offset from the proximity baseline.

The mobile device can perform (712) tasks in accordance with the program instructions. Performing the task may include changing the input mode of the mobile device from the touch input mode to the voice input mode. Changing the input mode to the voice input mode includes configuring the mobile device to accept at least one of a voice command or dictation.

FIG. 7B is a flow diagram illustrating an exemplary motion-based operation 720 of a mobile device. The mobile device can be the mobile device 100 as described above. The mobile device can set (722) the proximity sensor to operate in the passive mode.

The mobile device can receive (724) the identified gesture action from the gesture recognition subsystem of the mobile device. The mobile device can determine (726) whether the proximity sensor detected a proximity event within a graceful period (eg, 100 milliseconds) prior to receiving the identified gesture.

If the mobile device determines that the proximity sensor detected a proximity event within a threshold period before receiving the identified gesture, the mobile device may confirm the proximity event. After confirmation, the mobile device can then perform (728) the task (eg, changing the input mode to the voice input mode). In addition to performing tasks, the mobile device can set (722) the proximity sensor back to the passive mode.

If the mobile device determines that the proximity sensor has not detected a proximity event within the threshold period before receiving the identified gesture, the mobile device may feel close proximity The detector is set (730) to operate in active mode. The mobile device can determine (732) whether the proximity sensor has detected a proximity event within a graceful period (eg, 100 milliseconds) after receiving the identified gesture. The mobile device may perform (728) the task if the proximity sensor has detected a proximity event within the threshold period after receiving the identified gesture. Otherwise, the mobile device can set (722) the proximity sensor to operate in the passive mode.

FIG. 7C is a flow diagram illustrating an exemplary motion-based operation 740 of a mobile device. The mobile device can be the mobile device 100 as described above. The mobile device can receive (742) a program command configured to perform a task even after the mobile device has moved to a location close to an object.

The mobile device can obtain (744) motion readings from one or more motion sensing devices of the mobile device. The motion sensing device can include at least one of an accelerometer, a gyroscope, a magnetometer, a light sensor, or a gravimeter.

The mobile device can detect (746) a motion event based on the motion reading. Detecting the gesture event may include determining that the motion reading indicates that the mobile device is moving toward the target object in one or more specified manners.

The mobile device can detect (748) a proximity event. Detecting a proximity event may include obtaining a proximity reading from a proximity sensor of the mobile device. A proximity reading may indicate that the mobile device is positioned proximate to the object. Detecting the proximity event may include setting the proximity sensor to operate in a passive mode in which the triggering of the proximity sensor causes an event notification of the proximity event. Detecting a proximity event may include detecting a proximity event when the proximity sensor is operating in a passive mode.

The mobile device can be determined based on the gesture event and the proximity event (750) The mobile device has moved to a position close to the target object. Determining that the mobile device has moved to a position close to the target object may include: determining to detect a proximity event during the threshold period before detecting the motion event, or detecting the threshold event after detecting the motion event When the event is close, the mobile device has moved to a position close to the target object. Determining that the mobile device has moved to a location proximate to the target object may include switching the proximity sensor to switch from the passive mode to the active mode immediately after detection of the gesture event. When operating in active mode, the proximity sensor trigger can cause an event notification of a proximity event, the backlight of the display to be turned off, or the touch input of the touch-sensitive input device to be turned off.

The mobile device can perform (752) tasks in accordance with program instructions. The task can include turning off the touch-sensitive display screen and switching the input mode of the mobile device between the touch input mode and the voice input mode.

Exemplary mobile device architecture

8 is a block diagram of an illustrative architecture 800 of a mobile device configured to perform motion-based operations. The mobile device can include a memory interface 802, one or more data processors, an image processor and/or processor 804, and a peripheral interface 806. The memory interface 802, the one or more processors 804, and/or the peripheral interface 806 can be separate components or can be integrated into one or more integrated circuits. Processor 804 can include one or more application processors (APs) and one or more baseband processors (BPs). The application processor and baseband processor can be integrated into a single processing chip. The various components in mobile device 100, for example, may be coupled by one or more communication busses or signal lines.

Sensors, devices, and subsystems can be coupled to the perimeter interface 806 to facilitate multiple functionality. For example, motion sensor 810, light sensor 812, and proximity sensor 814 can be coupled to peripheral interface 806 to facilitate orientation, illumination, and proximity functions of the mobile device. Motion sensor 810 can include one or more accelerometers configured to determine changes in the speed and direction of movement of the mobile device. A location processor 815 (eg, a GPS receiver) can be coupled to the perimeter interface 806 to provide geographic location. An electronic magnetometer 816 (e.g., an integrated circuit die) can also be coupled to the peripheral interface 806 to provide information that can be used to determine the direction of the magnetic north. Therefore, the electronic magnetometer 816 can be used as an electronic compass. Gravity meter 817 can be coupled to peripheral interface 806 to facilitate measurement of the local gravity field of the earth.

Camera subsystem 820 and optical sensor 822 (eg, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) optical sensor) can be used to facilitate camera functions such as recording photos and video segments.

Communication functions may be facilitated via one or more wireless communication subsystems 824, which may include radio frequency receivers and transmitters and/or optical (eg, infrared) receivers and transmitters. The particular design and implementation of communication subsystem 824 may depend on the communication network that the mobile device intends to operate. For example, the mobile communication device may include the operating system on CDMA, WiFi ^TM network or WiFi ^TM and Bluetooth ^TM network subsystem 824 designed. In particular, the wireless communication subsystem 824 can include a escrow agreement to enable the mobile device to be configured as a base station for other wireless devices.

The audio subsystem 826 can be coupled to the speaker 828 and the microphone 830 to facilitate voice recognition functions, such as voice recognition, voice replication, digital recording, and telephone function.

Input/output (I/O) subsystem 840 can include touch screen controller 842 and/or other input controllers 844. The touch screen controller 842 can be coupled to the touch screen 846 or the touch pad. Touch screen 846 and touch screen controller 842 can, for example, use any of a plurality of touch sensitive technologies and other proximity sensor arrays for determining one or more points in contact with touch screen 846 or Other components to detect contact and movement or their interruptions include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies.

Other input controllers 844 may be coupled to other input/control devices 848 such as: one or more buttons, rocker switches, thumb wheels, infrared ray, USB ports, and/or such as touch The indicator device of the pen. The one or more buttons (not shown) may include an up/down button for volume control of the speaker 828 and/or the microphone 830.

In one implementation, the pressing of the button lasts for a first duration to unlock the touch screen 846; and the pressing of the button lasts for a second duration longer than the first duration to power the mobile device 100 or Shut down. The user may be able to customize the functionality of one or more of the buttons. Touch screen 846 can, for example, also be used to implement virtual or soft buttons and/or keyboards.

In some implementations, the mobile device 100 can present recorded audio and/or video files (such as MP3, AAC, and MPEG files). In some implementations, the mobile device 100 can include the functionality of an MP3 player. Thus, the mobile device 100 can include a pin connector that is compatible with the iPod. Other input/output and control devices can also be used.

The memory interface 802 can be coupled to the memory 850. Memory 850 can include High speed random access memory and/or non-volatile memory, such as one or more disk storage devices, one or more optical storage devices, and/or flash memory (eg, NAND, NOR). The memory 850 can store an operating system 852 such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. Operating system 852 can include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system 852 can include a core (eg, a UNIX core).

Memory 850 can also store communication instructions 854 to facilitate communication with one or more additional devices, one or more computers, and/or one or more servers. The memory 850 can include graphical user interface instructions 856 for facilitating graphical user interface processing; sensor processing instructions 858 for facilitating sensor related processing and functions; and telephone instructions for facilitating phone related programs and functions 860; an electronic message delivery instruction 862 for facilitating electronic message delivery related programs and functions; a network browsing instruction 864 for facilitating network browsing related programs and functions; and a media processing instruction 866 for facilitating media processing related programs and functions GPS/navigation commands 868 for facilitating global positioning system (GPS) and navigation related procedures and functions; camera instructions 870 for facilitating camera related procedures and functions; magnetometer data 872 and calibration instructions for facilitating magnetometer calibration 874. The memory 850 can also store other software commands (not shown), such as security commands, network video commands for facilitating network video related programs and functions, and/or to facilitate online shopping related programs and functions. Online shopping instructions. In some implementations, the media processing instructions 866 are divided into programs and functions for facilitating audio processing, respectively. And audio processing instructions and video processing instructions for programs and functions related to video processing. The boot record and the International Mobile Equipment Identity (IMEI) or similar hardware identifier may also be stored in the memory 850. Memory 850 can include motion instructions 876. The motion instructions 876 can be computer program products configured to cause the mobile device to perform motion-based operations, the motion-based operations including gesture motion recognition operations and gesture motion confirmation operations, as described with reference to Figures 1 through 7. .

Each of the above identified instructions and applications may correspond to a set of instructions for performing one or more of the functions described above. These instructions do not need to be implemented as stand-alone software programs, programs or modules. Memory 850 can include additional instructions or fewer instructions. In addition, various functions of the mobile device can be implemented in hardware and/or in software, including in one or more signal processing and/or special application integrated circuits.

Exemplary operating environment

9 is a block diagram of an exemplary network operating environment 900 configured to perform a motion-based operational mobile device. Mobile devices 902a and 902b can communicate, for example, on one or more wired and/or wireless networks 910 during data communications. For example, wireless network 912 (eg, a cellular network) can communicate with a wide area network (WAN) 914 (such as the Internet) by using gateway 916. Likewise, access device 918 (such as an 802.11g wireless access device) can provide communication access to wide area network 914.

In some implementations, both voice communication and data communication can be established via wireless network 912 and access device 918. For example, mobile device 902a can place and receive telephone calls (eg, using Internet voice communication) VoIP protocol, sending and receiving email messages (eg, using Post Office Protocol 3 (POP3)), and capturing electronic files and/or strings on wireless network 912, gateway 916, and wide area network 914 Streams (such as web pages, photos, and videos) (for example, using Transmission Control Protocol/Internet Protocol (TCP/IP) or User Datagram Protocol (UDP)). Similarly, in some implementations, mobile device 902b can make and receive phone calls, send and receive email messages, and retrieve electronic files on access device 918 and wide area network 914. In some implementations, the mobile device 902a or 902b can be physically coupled to the access device 918 using one or more cables, and the access device 918 can be a personal computer. In this configuration, the mobile device 902a or 902b may be referred to as a "tethered" device.

Mobile devices 902a and 902b may also establish communications by other components. For example, wireless mobile device 902a can communicate with other wireless devices (e.g., other mobile devices 902a or 902b, cellular telephones, etc.) over wireless network 912. Likewise, mobile devices 902a and 902b can establish inter-segment communication 920 (e.g., a personal area network) by using one or more communication subsystems, such as Bluetooth ^(TM) communication devices. Other communication protocols and topologies can also be implemented.

Mobile device 902a or 902b can communicate with one or more services 930 and 940, for example, on one or more wired and/or wireless networks. For example, one or more athletic training services 930 can be used to determine one or more athletic patterns. The motion pattern service 940 can provide the one or more motion patterns to the mobile devices 902a and 902b for recognizing the gestures.

Mobile device 902a or 902b may also be in one or more wired and/or wireless networks Access other materials and content on the road. For example, a content publisher (such as a news website, a Really Simple Syndication (RSS) feed, a web site, a blog, a social networking site) can be accessed by the mobile device 902a or 902b. , developer network, etc.). This access can be provided by a web browsing function or an application (eg, a browser) call in response to a user touching a form, such as a web object.

A number of implementations of the invention have been described. However, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, each of the subsystems, components, or units described above can include hardware devices, software instructions, or both.

100‧‧‧ mobile devices

102‧‧‧

104‧‧‧Sports path

110‧‧‧Signal recognition subsystem

112‧‧‧ proximity detection subsystem

116‧‧‧Time

118‧‧‧ faces

202‧‧‧Sports sensor

206‧‧‧motion sensor readings

206a‧‧‧ sport style

206b‧‧‧ sports style

206c‧‧‧ sports style

220‧‧‧Identified gesture

222‧‧‧ proximity sensor

224‧‧‧ close output

226‧‧" interface

308‧‧‧Dynamic screening subsystem

310‧‧‧Sliding time window

311‧‧‧Normalized motion sensor readings

312‧‧‧Sports recognition subsystem

314‧‧‧Sports type data storage

316‧‧‧ sports style

318‧‧‧ Distance calculation subsystem

402‧‧‧Sports vector

410‧‧‧ directed graph

412‧‧‧Comparative threshold

414‧‧‧Comparative threshold

502‧‧‧ proximity sensor controller

504‧‧‧ Near event cache memory

602‧‧‧Time t1

604‧‧‧Time t2

606‧‧‧Time

608‧‧‧ perceived response time

622‧‧‧Time t1'

624‧‧‧Time t2'

626‧‧‧Time tc

627‧‧‧Time

628‧‧‧Time t3'

630‧‧‧ perceived response time

800‧‧‧ Architecture

802‧‧‧ memory interface

804‧‧‧ processor

806‧‧‧ peripheral interface

810‧‧‧Sports sensor

812‧‧‧Light sensor

814‧‧‧ proximity sensor

815‧‧‧ position processor

816‧‧‧Electronic magnetometer

817‧‧‧ Gravimeter

820‧‧‧ camera subsystem

822‧‧‧Optical sensor

824‧‧‧Wireless communication subsystem

826‧‧‧ Audio subsystem

828‧‧‧Speaker

830‧‧‧Microphone

840‧‧‧Input/Output (I/O) Subsystem

842‧‧‧Touch Screen Controller

844‧‧‧Input controller

846‧‧‧ touch screen

848‧‧‧Input/control device

850‧‧‧ memory

852‧‧‧Operating system

854‧‧‧Communication Directive

856‧‧‧Graphical User Interface Instructions

858‧‧‧Sensor processing instructions

860‧‧‧ telephone order

862‧‧‧ Electronic Message Delivery Instructions

864‧‧‧Internet browsing instructions

866‧‧‧Media Processing Instructions

868‧‧‧Global Positioning System (GPS)/Navigation Directive

870‧‧‧ camera instructions

872‧‧‧ magnetometer data

874‧‧‧ calibration instructions

876‧‧ ‧ sport instruction

900‧‧‧Network operating environment

902a‧‧‧ mobile device

902b‧‧‧ mobile device

910‧‧‧Wired and / or wireless network

912‧‧‧Wireless network

914‧‧‧ Wide Area Network (WAN)

916‧‧‧ gateway

918‧‧‧ access device

920‧‧‧Inter-level communication

930‧‧‧Sports training service

940‧‧‧Sports model service

1 is a diagram providing an overview of an exemplary motion-based device operation.

2 is a block diagram of an illustrative mobile device configured to perform motion-based operations.

3 is a block diagram illustrating an exemplary system for illustrating motion recognition.

4A-4B are diagrams illustrating an exemplary technique for matching motion sensor readings to a motion pattern.

5 is a block diagram illustrating an exemplary schematic action confirmation subsystem of a mobile device.

6A-6B are diagrams illustrating a timetable for configuring a proximity sensor.

7A-7C are flow diagrams illustrating an exemplary motion-based operation of a mobile device.

8 is a block diagram of an illustrative architecture of a mobile device configured to perform motion-based operations.

9 is a block diagram of an exemplary network operating environment configured to perform a motion-based operational mobile device.

The same reference symbols are used in the various drawings.

100‧‧‧ mobile devices

110‧‧‧Signal recognition subsystem

112‧‧‧ proximity detection subsystem

202‧‧‧Sports sensor

206‧‧‧motion sensor readings

220‧‧‧Identified gesture

222‧‧‧ proximity sensor

224‧‧‧ close output

226‧‧" interface

Claims (24)

A method of operating a motion-based device, comprising: turning on a proximity sensor of a mobile device when a mobile device enters one of the motion detection modes and setting the proximity sensor to Operating in a passive mode of operation, wherein in the passive mode of operation, the mobile device establishes a baseline of the proximity sensor, the baseline including a proximity reading indicating that the proximity sensor is not Detecting an object, the baseline being usable by the mobile device to determine a proximity to an object when determining that the proximity sensor's proximity reading meets a specified offset from the baseline; the mobile device is in the motion Acquiring a motion reading from one or more motion sensing devices of the mobile device during one of the operations in the detection mode; obtaining the proximity sensing before obtaining the motion reading and before recognizing a gesture from the motion reading a plurality of readings for establishing the baseline and establishing the baseline from the plurality of readings; after setting the proximity sensor to the passive mode of operation, identifying Illustrating an action, comprising determining that the motion reading indicates that the mobile device is moving toward a target object in one or more specified manners; after recognizing the gesture, performing the following action: setting the proximity sensor to an active mode of operation Operating in the middle; determining that the proximity sensor obtains one of the readings in the active mode of operation and the proximity sensor determines the gesture One of the differences between the baselines satisfies a threshold; in response, it is confirmed that the gesture is not a false positive; and then a task is performed in response to the acknowledged gesture, wherein the time taken to establish the baseline is by The mobile device is shielded during the time of the movement. The method of claim 1, wherein: performing the task comprises changing the input mode of one of the mobile devices from a touch input mode to a voice input mode. The method of claim 2, wherein changing the input mode to the voice input mode comprises configuring the mobile device to accept at least one of a voice command or a dictation. The method of claim 1, wherein the recognizing the gesture comprises: comparing the motion reading to one or more pre-stored motion patterns, the one or more pre-stored motion patterns and the mobile device toward the target The object moves with one of the gestures, each pre-stored motion pattern corresponds to one of the gestures; and the gesture is recognized based on the result of the comparison. The method of claim 1, wherein one of the proximity sensors triggers an event notification that causes one of the proximity events, one of the backlights of one of the displays to be turned off, and a touch-sensitive input device when operating in the active mode of operation One of the touch inputs is turned off. The method of claim 5, wherein the confirming that the gesture is not a misjudgment comprises determining that the reading obtained by the proximity sensor in the active mode of operation is obtained within a threshold time period after the gesture is recognized. The method of claim 1, wherein the target object comprises at least a portion of a human face. The method of claim 1, wherein the one or more motion sensing devices comprise at least one of an accelerometer, a gyroscope, a magnetometer, a light sensor, or a gravimeter. A non-transitory storage device for storing a computer program product, the computer program product being configured to cause one or more mobile devices to perform operations including: when a mobile device enters one of motion detection modes, Opening a proximity sensor of the mobile device and setting the proximity sensor to operate in a passive mode of operation, wherein in the passive mode of operation, the mobile device establishes a baseline of the proximity sensor, the baseline Including a proximity reading indicating that the proximity sensor does not detect an object, the baseline being usable by the mobile device to determine that the proximity sensor's proximity reading meets one of the baselines The shifting time determines a proximity to an object; and obtains a motion reading from one or more motion sensing devices of the mobile device during a time when the mobile device operates in the motion detecting mode; and when the motion reading is obtained Obtaining a plurality of readings of the proximity sensor for establishing the baseline and establishing the baseline from the plurality of readings before the motion reading identifies a gesture After the proximity sensor is set to the passive mode of operation, the gesture recognition, which includes determining the motion reading indicates that the mobile device One or more specified ways of moving toward a target object; after identifying the gesture, performing the following actions: setting the proximity sensor to operate in an active mode of operation; determining that the proximity sensor is in the active mode of operation One of the readings obtained and one of the baselines determined by the proximity sensor prior to identifying the gesture action satisfies a threshold; in response, confirming that the gesture is not a false positive; and then responding to the confirmation A gesture is performed to perform a task in which the time taken to establish the baseline is obscured by the time of the mobile device during exercise. The non-transitory storage device of claim 9, wherein: performing the task comprises changing the input mode of one of the mobile devices from a touch input mode to a voice input mode. The non-transitory storage device of claim 10, wherein changing the input mode to the voice input mode comprises configuring the mobile device to accept at least one of a voice command or a dictation. The non-transitory storage device of claim 9, wherein the recognizing the gesture comprises: comparing the motion reading to one or more pre-stored motion patterns, the one or more pre-stored motion patterns and the action A movement of the device toward the target object is associated with a gesture, each pre-stored motion pattern corresponding to one of the motions of the gesture; and the gesture is identified based on the result of the comparison. The non-transitory storage device of claim 9, wherein when operating in the active mode of operation, one of the proximity sensors triggers an event notification that causes one of the proximity events, one of the backlights of one of the displays is turned off, and one One of the touch inputs of the touch sensitive input device is turned off. The non-transitory storage device of claim 13, wherein the confirming the gesture is not a misjudgment comprises determining that the proximity sensor obtains the reading in the active mode within a threshold time period after identifying the gesture obtain. The non-transitory storage device of claim 9, wherein the target object comprises at least a portion of a human face. The non-transitory storage device of claim 9, wherein the one or more motion sensing devices comprise at least one of an accelerometer, a gyroscope, a magnetometer, a light sensor, or a gravimeter. A motion-based device operating system comprising: a mobile device configured to perform an operation comprising: opening the action when the mobile device enters one of a motion detection mode One of the devices is in proximity to the sensor and the proximity sensor is configured to operate in a passive mode of operation, wherein in the passive mode of operation, the mobile device establishes a baseline of the proximity sensor, the baseline including a proximity Reading, the proximity reading indicating that the proximity sensor does not detect an object, the baseline being usable by the mobile device to determine when the proximity sensor's proximity reading meets a specified offset from the baseline Close to an object; Obtaining a motion reading from one or more motion sensing devices of the mobile device during a time when the mobile device operates in the motion detection mode; and recognizing a motion from the motion reading when the motion reading is obtained Previously, the plurality of proximity sensors are used to establish a reading of the baseline and the baseline is established from the plurality of readings; after the proximity sensor is set to the passive mode of operation, the gesture is identified, including Determining that the motion reading indicates that the mobile device is moving toward a target object in one or more specified manners; after recognizing the gesture, performing the following actions: setting the proximity sensor to operate in an active mode of operation; The difference between one of the readings obtained by the proximity sensor in the active mode of operation and the baseline determined by the proximity sensor prior to identifying the gesture action satisfies a threshold; in response, it is determined that the gesture is not a Misjudge; then perform a task in response to the confirmed gesture, wherein the time taken to establish the baseline is by the line Apparatus in motion to shield the time to enabling the user to enter the elapsed between the motion detection mode and perform the task in the mobile device to establish the perception of time does not include the time it takes it to the baseline. The system of claim 17, wherein: performing the task comprises changing the input mode of one of the mobile devices from a touch input mode to a voice input mode. The system of claim 18, wherein changing the input mode to the voice input mode comprises configuring the mobile device to accept at least one of a voice command or a dictation. The system of claim 17, wherein the identifying the gesture comprises: comparing the motion reading to one or more pre-stored motion patterns, the one or more pre-stored motion patterns and the mobile device facing the mobile device The target object is moved in association with one of the gesture actions, each pre-stored motion pattern corresponds to one of the gesture actions; and the gesture is identified based on the result of the comparison. The system of claim 17, wherein one of the proximity sensors triggers an event notification that causes one of the proximity events, one of the backlights of one of the displays to be turned off, and a touch-sensitive input device when operating in the active mode of operation One of the touch inputs is turned off. The system of claim 21, wherein the acknowledging that the gesture is not a false positive comprises determining that the reading obtained by the proximity sensor in the active mode of operation is obtained within a threshold time period after the gesture is recognized. The system of claim 17, wherein the target object comprises at least a portion of a human face. The system of claim 17, wherein the one or more motion sensing devices comprise at least one of an accelerometer, a gyroscope, a magnetometer, a light sensor, or a gravimeter.