CN105074626B

CN105074626B - Detection and response to extras touch event

Info

Publication number: CN105074626B
Application number: CN201480019431.0A
Authority: CN
Inventors: D·艾弗拉哈米
Original assignee: Intel Corp
Current assignee: Intel Corp
Priority date: 2013-05-01
Filing date: 2014-04-21
Publication date: 2019-03-05
Anticipated expiration: 2034-04-21
Also published as: US9448637B2; WO2014179096A1; CN105074626A; US20140327655A1

Abstract

The technology for allowing that extras touch event (such as on the surface around mobile device hand strike or finger or pencil tapping) is detected and positioned is described herein.The application executed in the operating system of mobile device or the equipment can take various movements based on the position of touch event.Detection device is mobile and in response to touch event, and mobile device sensor (such as accelerometer) detection has occurred and that touch event.In some embodiments, touch event feature vector is generated according to sensor reading, and the touch event feature vector is passed to the classifier of determining touch event position.May further determine that other touch event information, such as timing, intensity and can cause touch event object profile.Detection and positioning to touch event allow the interaction area of mobile device to extend beyond the physical boundary of the equipment, and multiple users is allowed to use mobile device in a manner of more convenient simultaneously.

Description

Detection and response to extras touch event

Background technique

The size of the interactive space of most of current mobile devices is restricted.For current tablet computer and intelligence Energy phone, interactive space are generally limited by its touch screen and control button.

Detailed description of the invention

Fig. 1 shows detectable extras touch event and can execute the example of different movements according to the position of touch event Property mobile device.

Fig. 2 shows detectable extras touch event and different movements can be executed according to the position of touch event and timing EXEMPLARY MOBILE DEVICE.

Fig. 3 is the flow chart for the illustrative methods for detecting extras touch event and determining its position.

Fig. 4 is the block diagram that can determine the exemplary computer device of position of extras touch event.

Fig. 5 is the position for determining the extras touch event at mobile device and the illustrative methods of execution movement Flow chart.

Fig. 6 is the block diagram that the exemplary computer device of technology described herein can be achieved.

Fig. 7 is the example of the executable computer executable instructions as a part for realizing technology described herein The block diagram of property processor core.

Specific embodiment

Technology disclosed herein makes the interactive space of mobile computing device extend beyond the physical boundary of the equipment.It rings Should be in extras touch event, the touch event that can occur near the equipment, the sensor being built into the equipment detects Movement of the equipment on the surface that the equipment is just stopping.Movement detected can be used at least roughly determining touch event Position.For example, mobile phone can detect finger tapping on the surface around phone of user using inbuilt accelerometers, determine The position of tapping, and where foundation occurs tapping and executes movement.

The extension of the interactive space of equipment allows user using its whole hand or both hands rather than is only used only one or several A finger (finger is usually only interacted with touch screen or control button) interacts with the equipment.For example, on a cellular telephone The music application of execution may be in response to user and strike his hand on the surface around the equipment and play idiophonic sound. Where the application can beat his or her hand according to user and play different sound.Interactive space through extending also allows more Multi-user interacts with individual equipment simultaneously, or multiple users is at least allowed conveniently to interact with individual equipment simultaneously.For example, Consider the question-and-answer game executed on tablet computer, the surface which can be neighbouring in plate based on which user His hand of upper tapping and which of detect several users and be given the chance answered a question.If having to crowd around with user Around the plate by with the touch screen interaction of plate come compared with participating in, user may more facilitate with this game interaction.

Referring now to the drawings, wherein similar appended drawing reference is used to quote similar element from beginning to end.It is described below In, for illustrative purposes, a large amount of details are elaborated to provide the thorough understanding to it, however, can not have these Novel embodiment is practiced in the case where detail can be obviously.In other instances, show in block diagram form Well-known structure and equipment are out in order to description of them.Be intended that covering scope of the claims in all modifications, Equivalent and replacement.

As used herein, term " touch event " and " exceptional space touch event " refer to that object is set with movement The interaction on the surface where standby neighbouring equipment.Object can be the finger of such as user or whole hand, pencil, drumstick or refer to Show pen, and surface can be such as table top, desk or floor.Touch event can be various interactions as a result, such as Finger or the pencil palm of tapping or opening on desk tap on table top.If the detectable touching of the sensor of mobile device Event is touched, then the touch event occurs near the equipment.That is, if touch event causes the detectable movement of mobile device, Then the touch event is near the equipment.Correspondingly, mobile device nearby can (such as the equipment just stopping according to various factors The material of platform, the object for creating touch event, object tap the intensity on surface and the sensitivity of movable sensor) And change.

Fig. 1 shows detectable extras touch event and can execute the example of different movements according to the position of touch event Property mobile device 100.Mobile computing device (mobile device) 100 can be any kind of movement described herein and set It is standby, such as mobile phone (as shown in the figure), plate or laptop computer.Mobile device 100 is just resting on the surface 110 of platform And it is carrying out electronic drum application, which applies and beat the surface near the equipment in response to user and play various beat The sound of percussion instrument.When the hand 120 or 130 of user beats surface 110, creation can cause 140 He of vibration of the movement of equipment 100 150, which is detected by the built-in sensors of the equipment.

Sensor generates the output of the movement of instruction mobile device.Typically, sensor includes accelerometer, the acceleration family planning At the output for reflecting acceleration of the equipment on one or more planes (for example, X, Y and Z plane).In some embodiments, it moves Dynamic equipment may include detecting the more than one accelerometer of touch event.In the other embodiments still having, gyroscope can be with one A or multiple accelerometers are used in combination to detect touch event, and in some embodiments, gyroscope provides indicating equipment and surrounds one Or the output of the angular orientation of multiple axis (for example, X, Y and Z axis), to realize the shake to equipment, the measurement fallen and rolled of inclining.

Mobile device 100 can execute different movements according to the position of touch event detected.For example, mobile device Where 100 can tap neighbouring circumferential surface according to user and play different sound.For example, if hand is on surface It is tapped on 110 and causes the small drum sound to be played, then peripheral region can be divided into multiple regions, and foundation by mobile device 100 Where hand taps and plays different small drum sound.In another embodiment, it can be played according to touch event position different Idiophonic sound.For example, can be programmed to equipment 100, so that the left hand in response to user beats the equipment left side Surface, can play army's drum sound, and if the right hand of user beats the surface on the right of the equipment, can play ground flop flop make an uproar Sound.Region around mobile device 100 can be divided into any number of subregion, and any shape can be used in each subregion, As long as being distinguished between the touch event that user equipment can occur in individual subregion.For example, tool there are two with The splitting scheme of upper subregion allows user to pretend him or she and beats a whole set of frame drum, wherein according to user on the surface 110 where It beats his or her hand and plays and step on small cymbals, fried small cymbals, army's drum, various flop bulging and pucking sound.

The position of identified touch event may include for example relative to the direction of mobile device and/or away from mobile device Distance.For example, the direction can indicate touch event from mobile device particular edge (for example, top, the following, left side, The right) most nearby occur, or occur at specific angular coordinate.For example, with reference to Fig. 1, the direction of hand 120 can be confirmed as In left the half of peripheral region or left lower quadrant, at 200 degree at 7 positions or in angular coordinate system.The side of hand 130 To can be determined to be in right half, right lower quadrant, at 4 positions or at 315 degree.As a result, in some embodiments, should Direction can be based at least partially on the equipment how its peripheral region of subregion.

Fig. 2 shows detectable extras touch event and different movements can be executed according to the position of touch event and timing EXEMPLARY MOBILE DEVICE 200.Calculating equipment 200 can be tablet computer or any other movement described herein Equipment is carrying out question-and-answer game application, beats surface 210 with hand (for example, hand 220,221,222 or 223) wherein giving The chance that first user answers a question.By making interaction surface extend beyond the touch screen 230 of mobile device 200, it is possible to increase Add the quantity for the player that can play multi-player gaming on the mobile apparatus or at least makes multiple users' object for appreciation game more convenient.Pass through letter Singly tapping the surface 210 near the equipment can interact with equipment 200, and multiple users do not need to crowd around at touch screen 230 weeks It encloses to play game.

Fig. 3 is the flow chart for the illustrative methods 300 for detecting extras touch event and determining its position.Method 300 It can be operated (such as in software application grade or operating system grade) in any rank of mobile device.Typically, method 300 It operates in operating system grade, and is communicated with software application and/or operating system component.

310 are acted in processing, sensor values is read in the buffer (or other memory constructions or component) of the equipment. Although method 300 shows the sensor values as continued to read, as finally returned that indicated by the path in 300 flow chart , as long as the reading of sensor values is realized by application or operating system component, sensor values can synchronously, asynchronously or continuously It reads on ground.One or more nearest sensor values can be stored in buffer, and sensor values may include indicating that this sets The standby acceleration in one or more planes or the equipment surround the value of the acceleration of one or more axis.

320 are acted in processing, determines whether sensor values indicates the generation of touch event.Various methods can be used for determining biography Whether sensor value indicates touch event.For example, in some embodiments, sensor values and touch event threshold value can be compared Compared with wherein touch event is by being more than that one or more accelerometer readings of the threshold value indicate.It include multiple acceleration in sensor values In the embodiment of plane or the value of multiple sensors, from less than all planes and/or less than the sensing of all sensors Device reads the presence that can be used for determining touch event.

If sensor values indicates touch event, 330 are acted in processing, determines that touch event is extras touching Touch event.If NO, then this method returns to 310 and reads new sensor values.Determine whether touch event additionally sets Standby touch event may include determining user whether via with the equipment direct physical contact (such as via with touch screen Interaction presses control button) input is supplied to the equipment.If touch event is confirmed as extras touch event, 340 then are acted in processing, method 300 calculates the feature vector of touch event and this feature vector is passed to classifier.If Touch event is determined not to be extras touch event, then method 300 returns to 310 and reads new sensor values.? In some embodiments of method 300, if the presence of sensor values instruction touch event, save processing movement 300 and square Method 300 from 320 proceeds to 340.Processing movement 330 can be supplied directly to the situation of mobile device not expecting user's input for example In save.

340 are acted in processing, the vector including one or more touch event features is determined according to sensor values.Touch thing Part feature vector may include for example being obtained and carrying out Fourier transform (such as Fast Fourier Transform) to sensor values Frequency domain information, minimum value and maximum value etc..Touch event feature vector may include time domain, frequency domain, other kinds of information or Any combination thereof.

Once being determined, touch event feature vector is just delivered to classifier.In some embodiments, using engineering Principle is practised to train classifier, wherein one group of training example (or vector) is supplied to classifier to train classifier can determine The position of touch event.In some embodiments, classifier can be closest (kNN) classifier of k-.In kNN classifier, The given input touch event feature vector for indicating query point, found in training data on closest to query point K example.Then, it is voted based on the majority in k example and distributes class to input feature value.Such is identified touch thing Part position, in some embodiments, if training set appropriate is already provided to classifier, to touch event feature vector Classification can determine multiple while tapping position provided by multiple users.

Various methods can be used to create classifier.Shown by one group of training for providing known touch event to mobile device Example creates classifier, this is referred to as the training classifier.Training set can be created in various ways.In one embodiment, will Mobile device places on the surface and starts trained application.The application provides instructions to user, this apply upper, Yong Hu Touch event is created on the surface of phone.Training example may include the touching occurred at the different location around mobile device Touch event, the touch event with intensity, using a part of different objects (for example, pencil, stylus) or the hand of user The touch event of (for example, hand of single, multiple fingers, fist, opening) creation is supplied directly to mobile device (for example, touching Screen or control button) touch event, be supplied to the touch event of different platform (for example, wood, granite) and send out simultaneously Raw multiple touch events.

In some embodiments, when mobile device is moved to different surfaces, can re -training classifier to allow this to set It is standby more reliably to detect touch event.For example, and user his or her palmistry ratio is tapped on carpet, user is on wood desktop It taps his or her hand and usually generates different accelerometer readings at mobile device.Equipment can prompt user to provide one group of training Example so as to for example the equipment detect it be moved into new surface when, starting utilize touch event position application When or user request when re -training classifier.

Processing act 350, by the position of the one or more touch event such as classifier determined by pass to apply or Operating system component.

In some embodiments, equipment can determine the intensity of touch event, and reflection such as user uses object or body Position taps the dynamics on the surface near the equipment.In these embodiments, executable at least partly using (or operating system) Movement of the ground according to touch event intensity.Alternatively, the characteristic for the movement that the application is taken may depend on touch event intensity.Example Such as, in music application example, the volume for playing musical instrument sound may be based partly on touch event intensity.In another example, exist In game application, the intensity that user taps surface may make game role to take different movements (for example, if intensity is a certain Hereinafter, then the role can jump, and if the intensity is more than threshold value, which can take off threshold value).

In some embodiments, which can determine the profile of the object of creation touch event.The profile can be for creating It is distinguished between the various objects for the touch event built in one group of training example.In these embodiments, using (or operation be System) can movement at least partly be executed according to contour of object.Alternatively, the characteristic for the movement that the application is taken can be according to object wheel It is wide.For example, music application example is returned to, if user uses single compared with user taps surface using the palm that he opens Finger tapping surface may make the different tones or sound of the application plays percussion instrument (such as small drum).Similarly, if divided Class device can distinguish between the hand or drumstick (or pencil, pen or stylus) of user, then music application can make difference Musical instrument played, such as if touch event is caused by drumstick, pencil, pen or stylus the army of broadcasting drum or flop drum, And small drum is played if touch event is caused by the hand or finger touch-surface of user.

In some embodiments, touch event position is determined with rough precision.For example, being swum in previously described question and answer In Application Example of playing, if four players are playing game, mobile device, which is only required to determine that touch event is located at, encloses Which quadrant around the surface of the equipment.It, can be by less training example as a result, in the embodiment for only needing rough precision Mobile device is supplied to train classifier.It, can be by biggish one in the embodiment of relatively precision accuracy for requiring touch location Group training example is supplied to mobile device for classifier training.

In some embodiments, technology herein can determine the clocking information of touch event.Clocking information may include appointing The clocking information of what type, such as absolute time (for example, the date, when, minute, second), relative to another touch event or equipment Or the time (for example, in question-and-answer game, time) from there is a upper problem of application affairs.

Technology described herein may further determine that the clocking information of multiple overlapping touch events.That is, touch event when Between upper enough near-earths occur so that accelerometer can be still presented when it starts to respond subsequent touch event in preceding touching Touch the response of event.In some embodiments, the first touch event occur in a series of multiple overlapping touch events when Between can be confirmed as sensor reading for the first time and be more than time of touch event threshold value.Since the sensor reading of touch event is usual Rapidly drop to touch event threshold value hereinafter, the time that therefore second touch event Chong Die with the first touch event occurs can To be time that sensor reading is more than touch event threshold value next time.

Fig. 4 is the block diagram that can determine the exemplary computer device 400 of the position of extras touch event.Calculate equipment 400 include one or more sensors 410, buffer 420, one or more application 430 and operating system 440.One or Multiple applications 430 and operating system 440 operate on calculating equipment 400.One or more sensors 410 may include one or more The mobile other sensors of a accelerometer, gyroscope or detectable equipment.Buffer 420 stores one or more sensors 410 values generated.In other embodiments, sensor values can be stored in mobile device 400 or by mobile device 400 In other accessible memory constructions (such as cache memory).It may include utilizing touch event position using 430 One or more application.

Operating system 440 includes touch event determining module 450, vector generation module 460, classifier 470 and report Module 480.It is attached in calculating equipment to determine that touch event determining module 450 assesses the sensor values being stored in buffer 420 Closely whether there is touch event.Vector generation module 460 generates the touch event feature vector of one group of sensor values.The vector It may include the feature of sensor values, minimum, maximum, centre and average sensor values and frequency domain in such as certain window Information.Classifier 470 can be used for determining the position of touch event according to touch event feature vector.For example, classifier can be Based on position associated with the top n example being most closely matched with supplied touch event feature vector in training class Determine the kNN classifier of touch event position.Classifier 470 can be provided by the component and/or application 430 of operating system 440 One group of training example training.Reporting modules 480 can be reported to the other assemblies of application 430 or operating system 440 about touching Touch the information of event, such as touch event position, clocking information, intensity, object for creating touch event profile.

It should be understood that Fig. 4 shows an example of the one group of module and component that can be included in mobile computing device.? In other embodiments, mobile device can have module or component more than module or component shown in Fig. 4 or few.For example, Mobile device may include multiple classifiers corresponding with different types of surface (for example, wood, granite).In some implementations In example, the mobile device including multiple classifiers corresponding with different surfaces type may include the mould for automatically selecting classifier Block.For example, the equipment can determine the type on surface by image captured by analysis built-in camera come selection sort device.Another In one example, equipment 600 can will be led to by instructing user to carry out one or more specific tappings on desk come selection sort device The training touch event feature vector for crossing tapping generation is compared from the training vector for corresponding to different stored classifiers. In these embodiments, mobile device 600 may not need to collect one group of new instruction when it is moved to different types of surface Practice example.

In addition, individual module or individual module shown in Fig. 4 can be combined into shown in Fig. 4 for isolated module It may be logically divided into multiple modules.In addition, being illustrated as any of the module in the operating system for calculating equipment 400 in Fig. 4 It can be operated in another software layer (such as application layer).In addition, module shown in Fig. 4 can with software, hardware, firmware or A combination thereof is realized.Refer to for be programmed to perform the calculating equipment of method can be programmed to via software, hardware, firmware or A combination thereof executes this method.

Fig. 5 is the position for determining the extras touch event at mobile device and the illustrative methods of execution movement 500 flow chart.Method 500 can be executed by the tablet computer for for example executing the question-and-answer game that four players are playing.Player The region that surface near the plate for distributing to them must be beaten, is answered a question with " coming in ".Plate is detected using accelerometer The movement of plate.

Act 510 in processing, the movement based on such as one or more mobile device sensor mobile device detected come Determine the position of the extras touch event occurred near mobile device.The position includes the side relative to mobile device To.In this example, problem is proposed to only No. 3 players to know the answer.No. 3 players tap after reading problem distributes to him Desk surface.Plate accelerometer taps desk in response to the hand of player and detects the movement of plate, and determination is distributing To the position of the touch event occurred in the equipment near zone of No. 3 players.

520 are acted in processing, the position is based at least partially on and determines the movement executed at mobile device.In the example In, No. 3 players chance responded to problem is given in plate determination.

530 are acted in processing, the movement is executed at mobile device.In this example, tablet computer inquires No. 3 players It answers a question.

Technology described herein at least has the advantages that following exemplary.Pass through the touch thing near detection mobile device Part allows user to interact with mobile device, and the interactive space of the equipment extends beyond the physical boundary of the equipment.In addition, institute is public The technology opened extends the type of supported interaction.Typically, user is via single finger or the interaction of more fingers and mobile device Interaction (for example, typewrite on hard or soft keyboard, supply single finger or multi-finger gesture to touch screen), but it is described herein Technology can detect the interaction of whole hand, such as user taps the hand opened or fist of fiercelying attack on the surface.In addition, disclosed Technology allows multiple users to interact simultaneously with mobile device, or multiple users is at least allowed conveniently to hand over mobile device Mutually.As described above, multiple users, which can be gathered in around mobile device, plays game, without crowd around around the equipment with intelligence The interaction of the limited touch area of energy phone or tablet computer.In addition, disclosed technology provide low cost solution with The interactive space of extension device.It is modified to current mobile device to realize that it is soft that technology described herein may only need Part modification, because many current mobile devices have possessed the mobile inbuilt accelerometers and gyroscope of detection device.In addition, passing through It can be fallen on surface in the hand for making them or other objects with the surface interaction near the equipment, user there is no concern that making a mess of this Equipment.

Technology, skill and embodiment described herein can be executed by any one of various calculating equipment, these meters Calculating equipment includes mobile device (such as smart phone, handheld computer, tablet computer, laptop computer, media play Device, portable game controller, camera and video recorder), non-mobile device (such as desktop computer, server, solid Determine game console, smart television) and embedded device (being such as integrated to the equipment in vehicle).As used herein , term " calculating equipment " includes computing system, and including the equipment with multiple discrete physical assemblies.

Fig. 6 is the block diagram that the exemplary computer device of technology described herein can be achieved.In general, shown in fig. 6 group Part can be communicated with component shown in other, but for the ease of explaining, all connections are not shown.Equipment 600 be include first The multicomputer system of processor 602 and second processor 604, and be explained as including point-to-point (P-P) interconnection.For example, Point-to-point (P-P) interface 606 of processor 602 is coupled to the point-to-point interface 607 of processor 604 via point-to-point interconnection 605. It should be appreciated that any or all point-to-point interconnection illustrated in fig. 6 is alternatively embodied as multi-point bus, and shown in Fig. 6 Any or all bus out can be substituted by point-to-point interconnection.

As shown in fig. 6, processor 602 and 604 is multi-core processor.Processor 602 includes processor core 608 and 609, and Processor 604 includes processor core 610 and 611.Processor core 608-611 can be similar to the side discussed below in conjunction with Fig. 7 Formula otherwise executes computer executable instructions.

Processor 602 and 604 further comprises at least one shared cache memory 612 and 614 respectively.It is shared Cache 612 and 614 can store one or more components (such as processor core 608-609 and 610- of the processor 611) data (for example, instruction) utilized.Shared cache 612 and 614 can be the storage levels of equipment 600 (hierarchy) a part.For example, shared cache 612 can be also stored in memory 616 being locally stored Data are to allow the component of processor 602 quickly to access these data.In some embodiments, shared cache 612 It may include multiple cache layers with 614, such as grade 1 (L1), grade 2 (L2), grade 3 (L3), grade 4 (L4), and/or other high speeds are slow It deposits or cache layer (such as last level cache (LLC)).

Although equipment 600 is shown to have two processors, equipment 600 may include any number of processor.This Outside, processor may include any number of processor core.Various forms, such as central processing unit, control can be used in processor Device, graphics processor, accelerometer (such as figure accelerometer or digital signal processor (DSP)) or field programmable gate array (FPGA).Processor in equipment can be identical or different with other processors in the equipment.In some embodiments, equipment 600 may include and first processor, accelerometer, FPGA or any other processor isomery or asymmetric one or more Processor.In terms of including the range of advantage measurement of framework, micro-architecture, heat, power consumption characteristics etc., processing element in systems Between each species diversity may be present.These differences can effectively make to manifest itself by asymmetric and different between the processor in system Structure.In some embodiments, processor 602 and 604 resides in identical die package.

Processor 602 and 604 further comprises Memory Controller logic (MC) 620 and 622.As shown in fig. 6, MC 620 Control the memory 616 and 618 for being coupled to processor 602 and 604 respectively with 640.Memory 616 and 618 may include various types of The memory of type, such as volatile memory are (for example, dynamic random access memory (DRAM), static random access memory Or nonvolatile memory (for example, flash memory) (SRAM)).Although MC 620 and 622 is shown as being integrated into 602 He of processor In 604, but in an alternate embodiment, these MC can be the logic outside processor, and may include memory hierarchy One or more layers.

Processor 602 and 604 is coupled to input/output (I/O) subsystem 630 via P-P interconnection 632 and 634.It is point-to-point The point-to-point interface 636 of processor 602 is connected by interconnection 632 with the point-to-point interface 638 of I/O subsystem 630, and point-to-point The point-to-point interface 640 of processor 604 is connected by interconnection 634 with the point-to-point interface 642 of I/O subsystem 630.Input/defeated Subsystem 630 further comprises the interface 650 for making I/O subsystem 630 be coupled to graphics engine 652 out, which can be with It is high performance graphics engine.I/O subsystem 630 and graphics engine 652 are coupled via bus 654.Alternatively, bus 654 can be with It is point-to-point interconnection.

Input/output subsystem 630 is further coupled to the first bus 660 via interface 662.First bus 660 can be with It is peripheral component interconnection (PCI) bus, PCI high-speed bus, another third generation I/O interconnection bus or any other type Bus.

Various I/O equipment 664 can be coupled to the first bus 660.First bus 660 can be coupled to second by bus bridge 670 Bus 680.In some embodiments, the second bus 680 can be low pin count (LPC) bus.Various equipment can be coupled to Two lines bus 680, these equipment include such as keyboard/mouse 682, audio I/O equipment 688 and store equipment 690, such as firmly Dish driving, solid-state driving or other storage equipment for storing computer executable instructions (code) 692.Code 692 can Including the computer executable instructions for executing technology described herein.It can be coupled to additional group of the second bus 680 Part includes one or more communication equipments 684, the communication equipment can via use one or more communication standards (for example, IEEE 802.11 standards and its supplement) one or more wired or wireless communication links (for example, electric wire, cable, Ethernet connection, Radio frequency (RF) channel, infrared channel, Wi-Fi channel) provide equipment 600 and one or more wired or wireless networks 686 (example Such as, Wi-Fi, honeycomb or satellite network) between communication.In addition, equipment 600 may include the one of the movement of detectable equipment 600 A or multiple sensors 689, such as accelerometer or gyroscope.

Equipment 600 may include removable memory, such as flash card (for example, SD (secure digital) blocks), memory stick, subscriber Identity module (SIM) card.Memory (including cache 612 and 614, memory 616 and 618, Yi Jicun in equipment 600 Storage equipment 690) data and/or computer executable instructions for executing operating system 694 Yu application program 696 can be stored. Sample data includes that equipment 600 is sent to and/or received from multiple network services via one or more wired or wireless networks One of device or other equipment or the webpage used for equipment 600, text message, image, audio files, video data, classification Device training data or other data sets.Equipment 600 may also access external memory (not shown), such as external hard-drive or Person's storage based on cloud.

Operating system 694 can control the distribution and use of component illustrated in fig. 6, and support one or more application journey Sequence 696.Operating system 694 may include collecting reading from one or more sensors, determining whether sensor reading indicates The touch event in region around the equipment, the feature vector for determining touch event, the component for determining touch event position, with And one or more classifiers 697 for these positions to be supplied to application program 696.Various classifiers can be included in and set In standby 600, in terms of and for example various types of surfaces and the configuration of various sensors.For example, if mobile device 600 includes Two accelerometers, then the first classifier can be used for providing touch event position from single accelerometer for the data collected, and second Classifier can provide position for the data collected at two accelerometers.

Application program 696 may include that common moving calculates equipment application (for example, e-mail applications, calendar, contact person's pipe Manage device, web browser, information receiving and transmitting application) and other calculating applications, it such as utilizes in the region around equipment 600 The detection of the touch event of generation and its application of position.

Equipment 600 can support various input equipments (such as touch screen, microphone, camera, physical keyboard and trace ball) with And one or more output equipments (such as loudspeaker and display).Other possible input and output devices include piezoelectricity and its His tactile I/O equipment.Input or any of output equipment can inside equipment 600, it is external or removable with equipment 600 Except ground attaches.External input and output equipment can be communicated via wired or wireless connection with equipment 600.

In addition, calculating equipment 600 can provide one or more natural user interfaces (UI).For example, operating system 694 or answering Using 696 may include the speech recognition logic as a part of Voice User Interface, the Voice User Interface allow user via Speech command operation equipment 600.In addition, equipment 600 may include allowing user via the posture and equipment 600 of body, hand or face Interactive input equipment and logic.For example, detectable and interpreting user gesture is to provide input to game application.

Equipment 600 can further comprise one or more communication components 684.Component 684 may include being coupled to one or more The wireless communication components of a antenna are to support the communication between system 60 and external equipment.Wireless communication components can support various nothings Line communication protocol and technology, such as near-field communication (NFC), Wi-Fi, bluetooth, 4G long term evolution (LTE), CDMA (CDMA), Universal Mobile Telecommunications System (UMTS) and global system for mobile communications (GSM).In addition, radio modem can support with For within single cellular network, between multiple cellular networks or mobile computing device and public switched telephone network (PSTN) Between data and voice communication one or more cellular networks communication.

Equipment 600 can further comprise that (input/output end port can be for example at least one input/output end port USB, IEEE 1394 (firewire), Ethernet, and/or the port RS-232), including physical connector, power supply, Proximity Sensor, Compass and receiver of satellite navigation system (such as GPS receiver).GPS receiver can be coupled to GPS antenna.Equipment 600 can Further comprise one or more additional antennas, the additional antenna be coupled to one or more accessory receivers, transmitter and/ Or transceiver is to realize additional function.

It should be appreciated that Fig. 6 shows only one exemplary computer device framework.Calculating equipment based on replacement framework can be used for Realize technology described herein.For example, being located at discrete integrated electricity instead of processor 602 and 604 and graphics engine 652 On the road, calculating equipment may include that in conjunction with multiple processors, graphics engine and add-on assemble SoC (system on chip) is integrated Circuit.It can be via different from bus shown in fig. 6 or point-to-point configuration connecting element in addition, calculating equipment.Moreover, in Fig. 6 Shown component is not required or is included entirely, because component and can add shown in can be removed in an alternate embodiment Add other assemblies.

Fig. 7 is performed as realizing the exemplary of the computer executable instructions of a part of technology described herein The block diagram of processor core 700.Processor core 700 can be any kind of processor (such as microprocessor, embedded processing Device, digital signal processor (DSP) or network processing unit) core.Processor core 700 can be single thread core or multithreading Core because it can each core include more than one hardware thread contexts (or " logic processor ").

The memory 710 for being coupled to processor 700 is also shown in Fig. 7.Memory 710 can be described herein any Memory or any other memory well known by persons skilled in the art.Memory 710, which can store, to be held by processor core 700 Capable computer executable instructions 715 (code).

The processor core includes the front end logic 720 that instruction is received from memory 710.Instruction can be decoded by one or more The processing of device 730.Decoder 730 produces microoperation (such as using the fixed width microoperation of predefined format) and exports as it, Or generate other instructions, microcommand or the control signal of reflection original code instruction.Front end logic 720 further comprises posting Storage renames logic 735 and scheduling logic 740, which usually distributes resource and to the finger with conversion for execution Corresponding operation is enabled to be lined up.

Processor core 700 further comprises executing logic 750, which includes one or more execution units (EU) 765-1 to 765-N.Some processor core embodiments may include specific function or multiple function is dedicated multiple executes list Member.Other embodiments may only include an execution unit or may include an execution unit of executable specific function.It holds Row logic 750 executes operation specified by code command.After executing operation specified by code command and completing, back-end logic 770 make Retirement using resignation (retirement) logic 775.In some embodiments, processor core 700 allows unordered It executes, but requires that instruction is made to retire from office in an orderly manner.Various forms well known by persons skilled in the art can be used in retirement logic 770 (for example, resequencing buffer etc.).

Processor core 700 is during executing instruction at least in the output generated of decoder 730, register renaming logic 735 using hardware register and table and become in terms of executing any register (not shown) modified of logic 750 It changes.

Although being not shown in Fig. 7, processor may include its being located at together in integrated chip with processor core 700 His element.For example, processor may include add ons, such as store control logic, one or more graphics engines, I/O control Logic processed, and/or one or more caches.

Referring back to Fig. 6, network 686 can provide the various clothes based on cloud that can be used for realizing technology described herein Business.For example, for training the training example of classifier that can be provided by service based on cloud for common surface.In some embodiments In, touch feature vector generation can locally executed at mobile device, and be subsequently transmitted to it is based on cloud service for Position determines.That is, classifier can be realized remotely.

Any one of disclosed method can be implemented as computer executable instructions or computer program product.This finger Order can make computer execute any one of disclosed method.In general, as used herein, term " computer " refers to herein Mentioned in any calculating equipment or system or any other calculate equipment.Term " computer executable instructions " as a result, Refer to the instruction that can be executed by any calculating equipment described herein and mentioned or any other calculating equipment.

It computer executable instructions or computer program product and during realizing disclosed technology creates and uses Any data can be stored in one or more tangible computer readable storage mediums (such as optical medium disk (for example, DVD, CD), Volatile memory component (for example, DRAM, SRAM) or non-volatile storage component (for example, flash memory, dish driving)) on. Computer readable storage medium can be comprised in computer readable storage medium (such as solid-state driving, USB flash memory driving and Memory module) on.Alternatively, computer executable instructions can be by comprising for executing all or part of disclosed method Hard wire logic specific hardware components or by any combination of computer readable storage medium or hardware component execute.

Computer-readable instruction can be such as special-purpose software application or via web browser or other software application A part of the software application of (such as remote computation application) access.For example, can be on a single computing device or in network rings This software is executed in border using one or more network computers.It is further understood that disclosed technology is not limited to appoint What certain computer language or program.For example, can by with C++, Java, Perl, JavaScript, Adobe Flash, Or any other suitable programming language software for writing realizes disclosed technology.Equally, disclosed technology is unlimited In any certain computer or any kind of hardware.Certain details of suitable computer and hardware are known, and It does not need to be set forth in the disclosure.

Moreover, software-based embodiment can be updated by suitable means of communication, downloaded or remotely be accessed Any of (including for example for making computer execute the computer executable instructions of any one of disclosed method).This A little suitable means of communication include that such as internet, WWW, Intranet, cable (including fibre-optic cable), magnetic communication, electromagnetism are logical Believe (including RF, microwave and infrared communication), electronic communication or other such means of communication.

As in this application and used in the claims, the list of items that term "and/or" is added can refer to institute Any combination of list of items.For example, phrase " A, B, and/or C " can refer to A, B, C, A and B, A and C, B and C or A, B and C. As used in this application and in the claims, be added term " at least one of " a list of items can refer to it is listed Any combination of project.For example, phrase " at least one of A, B or C " can refer to A, B, C, A and B, A and C, B and C, or A, B and C.

Disclosed method, equipment and system must not be explained to be limited in any way.On the contrary, this disclosure relates to each A disclosed embodiment independent and with all novel and non-obvious features of various combinations each other and sub-portfolio and Aspect.Disclosed method, equipment and system is not limited to any particular aspects or feature or a combination thereof, disclosed implementation Example does not require any one or more specific advantages there are problems that yet or as solution.

Theory of operation, the principles of science or other theories that equipment or method herein with reference to the disclosure are presented are retouched It states to better understand and provides, and do not expect to be limited in range.Equipment and method in appended claims Those of be not limited to work in a manner of described in these theory of operation equipment and method.

It is described although the operation of some disclosed methods shows for convenience with specific, sequence order, It should be understood that the describing mode is covered and is rearranged, unless language-specific set forth herein requires particular sorted.For example, It can rearrange or be executed concurrently in some cases the operation of sequence description.Moreover, for simplicity, appended attached drawing The various modes that disclosed method can be used in combination with other methods may be not shown.

Following example is related to the additional embodiment of technology disclosed herein.

A kind of mobile device of example 1., comprising: one or more sensors；One or more classifiers, this or more A classifier is based at least partially on the reading of one or more sensors to determine that is occurred near mobile device additionally sets The position of standby touch event；And reporting modules, this report module is to the application report extras executed on the mobile apparatus The position of touch event.

The mobile device of 2. example 1 of example, wherein the position of extras touch event includes relative to mobile device Direction.

The mobile device of 3. example 1 of example, wherein the position of extras touch event includes at a distance from mobile device.

The mobile device of 4. example 1 of example, wherein one or more sensors include at least one accelerometer.

The mobile device of 5. example 1 of example, wherein one or more sensors include at least one gyroscope.

The mobile device of 6. example 1 of example, wherein the application, which can be based at least partially on reporting modules and be supplied to this, answers Extras touch event position executes movement.

The mobile device of 7. example 6 of example, if wherein the application is further able to extras touch event position and exists Then executed in first area near mobile device the first movement and if extras touch event position in mobile device The second movement is then executed in neighbouring second area, the first movement is different from the second movement.

The mobile device of 8. example 6 of example further comprises extras touch event determining module, extras touching The clocking information that event determination module determines extras touch event is touched, reporting modules are further additionally set to the application report The clocking information of standby touch event, the meter of extras touch event is further based in part on by the movement of the first application execution When information.

The mobile device of 9. example 6 of example, at least one of these classifiers further determine that extras touch thing The intensity of part, for reporting modules further to the intensity of the application report extras touch event, which can be at least partly Ground is supplied to the extras touch event position of the application based on reporting modules and the intensity of extras touch events is held Action is made.

The mobile device of 10. example 1 of example, at least one of one or more classifiers are further determined that for creating The profile of the object of extras touch event, for reporting modules further to the profile of the application report object, which can Be based at least partially on reporting modules be supplied to the application extras touch event position and object profile it is dynamic to execute Make.

The mobile device of 11. example 1 of example further comprises: vector generation module, which is based on one Or the reading of multiple sensors generates extras touch event feature vector so that one or more classifiers are for determining The position of extras touch event；And extras touch event determining module, the extras touch event determine mould Block determines whether the reading of one or more sensors indicates the generation of extras touch event.

The mobile device of 12. example 1 of example further comprises: vector generation module, which is based on one Or the reading of multiple sensors generates extras touch event feature vector so that one or more classifiers are for determining The position of extras touch event and the intensity of extras touch event；And extras touch event determining module, The extras touch event determining module determines whether the reading of one or more sensors indicates extras touch event Generation.

The mobile device of 13. example 1 of example further comprises: vector generation module, which is based on one Or the reading of multiple sensors generates extras touch event feature vector so that one or more classifiers are for determining The profile of the position of extras touch event and the object for creating extras touch event；And extras touch Event determination module, the extras touch event determining module determine whether the reading of one or more sensors indicates additionally The generation of equipment touch event.

A kind of method of example 14. detection and response extras touch event, this method comprises: based on such as one or more The movement of a mobile device sensor mobile device detected determines that the extras that occur near mobile device touch The position of event；It is based at least partially on the position determination movement to be executed；And the movement is executed at mobile device.

The method of 15. example 14 of example, wherein the position includes the direction relative to mobile device.

The method of 16. example 14 of example, wherein the position includes at a distance from mobile device.

The method of 17. example 14 of example, further comprises: determining the clocking information of extras touch event, wherein really Surely the movement to be executed is based further on clocking information.

The method of 18. example 14 of example further comprises: determining the clocking information of extras touch event and is moving The one or more that dynamic equipment nearby occurs adds the clocking information of extras touch event, wherein the determination movement to be executed It is based further on the clocking information based in part on extras touch event and one or more additional extras The clocking information of touch event, which of extras touch event and one or more additional extras touch events Occur earliest.

The method of 19. example 14 of example, wherein the movement to be executed is one of multiple movements, if multiple movements include should The first movement and if extras touch event is determined that position then executes in the first area near mobile device For the second movement then executed in the second area near mobile device, first event is different from second event.

The method of 20. example 14 of example, further comprises: determining the intensity of extras touch event, wherein to execute Movement be based further on the intensity of extras touch event.

The method of 21. example 14 of example further comprises: determining the wheel for creating the object of extras touch event Exterior feature, wherein the movement to be executed is based further on the profile of object.

The method of 22. example 14 of example, wherein determining that position includes: to read one from one or more mobile device sensors A or multiple sensor values；Determine the generation of one or more sensors value instruction extras touch event；And according to one A or multiple sensor values generate extras touch event feature vector；

Wherein determine that position includes that extras touch event feature vector is supplied to classifier.

The method of 23. example 22 of example, wherein determining one or more sensors value instruction extras touch event Comprise determining that at least one of one or more sensors value is more than extras touch event threshold value.

The method of 24. example 22 of example, wherein the classifier is one of multiple classifiers, and this method further comprises: base Surface type near mobile device selects the classifier from multiple classifiers.

The method of 25. example 24 of example, is based in part on the image on the surface near mobile device wherein at least to determine Surface type.

The method of 26. example 24 of example is wherein at least based in part on and carries out one on desk in response to instruction user Or multiple specific tappings and generate training extras touch event feature vector, will training extras touch event feature Vector is compared to determine surface type from the training vector for corresponding to different stored classifiers.

The one or more computers for executing any one of method described in 4-26 according to claim 1 of example 27. can Read storage medium.

A kind of equipment of example 28., the device including executing any one of method described in 4-26 according to claim 1.

The one or more computer-readable mediums with the instruction stored thereon of example 29., these instructions are for making to move Dynamic equipment executes method, this method comprises: the shifting based on such as one or more mobile device sensors mobile device detected Move the position to determine the extras touch event occurred near mobile device；Position determination is based at least partially on to want The movement of execution；And the movement is executed at mobile device.

One or more computer-readable mediums of 30. example 29 of example, wherein the position includes relative to mobile device Direction.

One or more computer-readable mediums of 31. example 29 of example, wherein the position include with mobile device away from From.

One or more computer-readable mediums of 32. example 29 of example further comprise: determining that extras touch thing The clocking information of part, wherein the determination movement to be executed are based further on clocking information.

One or more computer-readable mediums of 33. example 29 of example further comprise: determining that extras touch thing The clocking information of part and the one or more occurred near mobile device add the clocking information of extras touch event, The wherein determination movement to be executed is based further on the clocking information and one based in part on extras touch event Outside the clocking information of a or multiple additional extras touch events, extras touch event and one or more plus Which of equipment touch event occurs earliest.

One or more computer-readable mediums of 34. example 29 of example, wherein the movement to be executed is multiple movements One, it is multiple movement include executed if the position is in the first area near mobile device first movement and if Extras touch event is determined to be in the second movement then executed in the second area near mobile device, and first event is not It is same as second event.

One or more computer-readable mediums of 35. example 29 of example further comprise: determining that extras touch thing The intensity of part, wherein the movement to be executed is based further on the intensity of extras touch event.

One or more computer-readable mediums of 36. example 29 of example further comprise: determination is additionally set for creating The profile of the object of standby touch event, wherein the movement to be executed is based further on the profile of object.

One or more computer-readable mediums of 37. example 29 of example, wherein determining that position includes: from one or more Mobile device sensor reads one or more sensors value；Determine that one or more sensors value instruction extras touch thing The generation of part；And extras touch event feature vector is generated according to one or more sensors value；Wherein determine position Including extras touch event feature vector is supplied to classifier.

One or more computer-readable mediums of 38. example 37 of example, wherein determining that one or more sensors value refers to Show that extras touch event comprises determining that at least one of one or more sensors value is touched more than extras Touch event threshold.

One or more computer-readable mediums of 39. example 37 of example, wherein the classifier is one of multiple classifiers, This method further comprises: the classifier is selected from multiple classifiers based on the surface type near mobile device.

One or more computer-readable mediums of 40. example 39 of example, it is attached to be wherein at least based in part on mobile device The image on close surface determines surface type.

A kind of equipment of example 41. includes: for being set based on such as one or more mobile device sensor movements detected Standby movement determines the device of the position of the extras touch event occurred near mobile device；For at least partly The device of the determination movement to be executed based on the position；And the device for executing the movement at mobile device.

The equipment of 42. example 41 of example further comprises: for determine extras touch event clocking information and The one or more occurred near mobile device adds the device of the clocking information of extras touch event, wherein for true Surely the device for the movement to be executed is based further on clocking information and one based in part on extras touch event Outside the clocking information of a or multiple additional extras touch events, extras touch event and one or more plus Which of equipment touch event occurs earliest.

The equipment of 43. example 41 of example, wherein the device for determining extras touch event position include: for from One or more mobile device sensors read the device of one or more sensors value；For determining one or more sensors The device of the generation of value instruction extras touch event；And for generating extras according to one or more sensors value The device of touch event feature vector；Wherein determine that position includes that extras touch event feature vector is supplied to classification Device.

Claims

1. a kind of mobile device, comprising:

One or more sensors, it is described for detecting the movement of the mobile device as caused by extras touch event Extras touch event is on the surface that the mobile device is placed and in the mobile device near zone；

One or more classifiers, one or more of classifiers are for being based at least partially on one or more of sensings The position of device read to determine the extras touch event；And

Reporting modules, the reporting modules are used to touch to extras described in the application report executed on the mobile device The position of event,

Wherein, the near zone can be divided into multiple subregions by the mobile device, and each subregion is in multiple users One.

2. mobile device according to claim 1, which is characterized in that the position of the extras touch event includes phase For the direction of the mobile device.

3. mobile device according to claim 1, which is characterized in that the position of the extras touch event include with The distance of the mobile device.

4. mobile device according to claim 1, which is characterized in that the application can be based at least partially on the report It accuses module and is supplied to the extras touch event position of the application to execute movement.

5. mobile device according to claim 4, which is characterized in that if the application is further able to described additionally set The first movement is then executed in the first area near the mobile device for standby touch event position and if described additionally set Standby touch event position then executes the second movement in the second area near the mobile device, and first movement is different from Second movement.

6. mobile device according to claim 4, which is characterized in that further comprise that extras touch event determines mould Block, the extras touch event determining module are used to determine the clocking information of the extras touch event, the report Module is accused to be used to be held further to the clocking information of extras touch event described in the application report by the application Capable movement is further based in part on the clocking information of the extras touch event.

7. mobile device according to claim 4, which is characterized in that at least one of described classifier is for further Determine the intensity of the extras touch event, the reporting modules to described in the application report for further additionally setting The intensity of standby touch event, the application can be based at least partially on the institute that the reporting modules are supplied to the application The intensity of extras touch event position and the extras touch event is stated to execute movement.

8. mobile device according to claim 1, which is characterized in that at least one of one or more of classifiers For further determining that the profile of the object for creating the extras touch event, the reporting modules are for further To the profile of object described in the application report, the application can be based at least partially on the reporting modules and be supplied to The profile of the extras touch event position of the application and the object executes movement.

9. mobile device according to claim 1, which is characterized in that further comprise:

Vector generation module, the vector generation module are generated for the reading based on one or more of sensors Extras touch event feature vector is so that one or more of classifiers are for determining the extras touch event The position；And

Extras touch event determining module, the extras touch event determining module are one or more for determining Whether the reading of a sensor indicates the generation of extras touch event.

10. mobile device according to claim 1, which is characterized in that further comprise:

Vector generation module, the vector generation module are generated for the reading based on one or more of sensors Extras touch event feature vector is so that one or more of classifiers are for determining the extras touch event The position and the extras touch event intensity；And

11. mobile device according to claim 1, which is characterized in that further comprise:

Vector generation module, the vector generation module are generated for the reading based on one or more of sensors Extras touch event feature vector is so that one or more of classifiers are for determining the extras touch event The position and the object for creating the extras touch event profile；And

12. a kind of method of detection and response extras touch event, which comprises

Based on as caused by the extras touch event and by one or more mobile device sensor shiftings detected The movement of dynamic equipment, the volume for determining on the surface that the mobile device is placed and occurring in the mobile device near zone The position of external equipment touch event；

The near zone is divided into multiple subregions, each subregion is for one in multiple users；

It is based at least partially on the position, the determination movement to be executed；And

The movement is executed at the mobile device.

13. according to the method for claim 12, which is characterized in that the position includes the side relative to the mobile device To and/or distance.

14. according to the method for claim 12, which is characterized in that further comprise: determining that the extras touch thing The clocking information of part, wherein the determination movement to be executed is based further on the clocking information.

15. according to the method for claim 12, which is characterized in that further comprise: determining that the extras touch thing The clocking information of part and the one or more occurred near the mobile device add the timing of extras touch event Information, wherein the determination movement to be executed is based further on the institute based in part on the extras touch event State the clocking information of clocking information and one or more of additional extras touch events, the extras touching Which of event and one or more of additional extras touch events is touched to occur earliest.

16. according to the method for claim 12, which is characterized in that the movement to be executed is one of multiple movements, institute State it is multiple movement include to be executed if the position is in the first area near the mobile device first movement, with And to be executed if the extras touch event is determined to be in the second area near the mobile device Two movements, first movement are different from second movement.

17. according to the method for claim 12, which is characterized in that further comprise: determining that the extras touch thing The intensity of part, wherein the movement to be executed is based further on the intensity of the extras touch event.

18. according to the method for claim 12, which is characterized in that further comprise: determining and described additionally set for creating The profile of the object of standby touch event, wherein the movement to be executed is based further on the profile of the object.

19. according to the method for claim 12, which is characterized in that the determination position includes:

One or more sensors value is read from one or more of mobile device sensors；

Determine the generation of one or more of sensor values instruction extras touch events；And

Extras touch event feature vector is generated according to one or more of sensor values；

Wherein the determination position includes that the extras touch event feature vector is supplied to classifier.

20. according to the method for claim 19, which is characterized in that the classifier is one of multiple classifiers, the side Method further comprises: the classifier is selected from the multiple classifier based on the surface type near the mobile device.

21. according to the method for claim 20, which is characterized in that the table being based at least partially near the mobile device The image in face determines the surface type.

22. according to the method for claim 20, which is characterized in that be based at least partially in response to instruction user in desk The upper training extras touch event feature vector for carrying out one or more specific tappings and generating additionally sets the training Standby touch event feature vector is compared to determine the surface from the training vector for corresponding to different stored classifiers Type.

23. one or more computer-readable storage mediums for realizing any one of method described in 2-22 according to claim 1 Matter.

24. the equipment of a kind of detection and response extras touch event, including for executing described in 2-22 according to claim 1 Any one of method device.

25. the equipment of a kind of detection and response extras touch event, comprising:

For based on as caused by extras touch event and by such as one or more mobile device sensors shifting detected The movement of equipment is moved to determine on the surface that the mobile device is placed and occur in the mobile device near zone The device of the position of extras touch event；

For the near zone to be divided into the device of multiple subregions, each subregion is for one in multiple users；

The device for the movement to be executed is determined for being based at least partially on the position；And

For executing the device of the movement at the mobile device.