CN109416679A - Multiple electronic controls and follow-up mechanism for mixed reality interaction - Google Patents

Abstract

A kind of system for using multiple electronic controls and follow-up mechanism to be used for mixed reality interaction, including electronic control unit, it is via network and multiple electronic device communications, it receives user's interaction and is transferred to device via network, and interaction is received from other devices via network and is transferred to host apparatus and a kind of method for communication nested between multiple electronic control units and electronic system.

Description

Multiple electronic controls and follow-up mechanism for mixed reality interaction

Cross reference associated with application

The application is entitled " the MULTIPLE ELECTRONIC CONTROL AND submitted on June 21st, 2016 The U.S. Patent Application Serial Number 15/187,787 of TRACKING DEVICES FOR MIXED-REALITY INTERACTION " PCT application and require its priority, include herein to herein by its entire specification by way of being cited in full text.

Technical field

This disclosure relates to the field of electronic device, and relate more particularly to for the input of interaction in software application and The field of output method.

Background technique

In the field of electronic device, to enable user interaction is provided usually using discrete hand-held controller or from host Device such as personal computer or video game console receive feedback.These controllers can via it is wired or wireless connection and It is connected, and is usually only matched with individual host device in any given time.When user is desirable with multiple controllers, he Must be individually connected to host apparatus (such as using two discrete ports on computers by both keyboard and mouse even It is connected to personal computer).This requires a large amount of discrete communication connections between host apparatus and controller, and if host fills It sets or controller has unmatched communication hardware, they are incompatible and are not available.Extraly, generally for specific Purpose (such as specific type video-game or computer application) design controller, and require user in a special manner with They are interacted, in order to keep controller in a specific way to have access by hand functional to its institute.This can for user Be it is difficult use or even unsound, and limit the mode that they interact with device.

Extraly, user usually interacts with body-building device, whether watches static screen.For example, ought be on a treadmill User can choose viewing and show the static screen of TV programme when running.Element in TV programme is static, wherein Element is simultaneously not based on the interaction of user and TV programme and changes behavior, but alternatively executes scheduled movement.Active screen Example be video-game.User interacts with remote-control device and influences the activity of element in video-game.At most, user can be with The unrelated movable screen of the movement with body-building device intercorrelation is performed to interact.

Further, with the Quick Extended of virtual reality industry, new exchange method is explored, including various for game Controller, walking stick and based drive input unit including gloves and hand tracking based on camera.However, these Device is focused on in the interaction of the hand of user, and is had ignored and be can be used for improving interactive and feeling of immersion and also expand simultaneously The other parts of the body for a possibility that having opened up data collection.

It is desirable that a kind of device so that user by various controls and follow-up mechanism be connected to host apparatus and and they Communication device it is unrelated, also solve user can be interacted and the demand of the controller unrelated with their positions or movement, For being interacted with mixed reality electronic application.

Summary of the invention

Therefore, imagine in a preferred embodiment of the invention and implement can be with multiple main frames device by the present inventor It matches and can send and used the nested multiple electronic controls and follow-up mechanism for mixed reality interaction communicated.

Preferred embodiment in accordance with the present invention discloses a kind of for being used to mix using multiple electronic controls and follow-up mechanism The system for closing reality interaction includes electronic control unit, the processing including storing in memory and operating in networked computing device At least multiple programming instructions on device, and be configured to via network and multiple electronic device communications, and be configured to operate Multiple hardware controls elements, hardware controls element configuration is used to receive man-machine interactively from user, and is configured to send multiple friendships For mutual data at least part of multiple electronic devices, interaction data is based at least partially on the man-machine interactively from human user At least part, and be configured to receive the traffic from least part of multiple electronic devices, the traffic includes at least mixed Real data is closed, wherein mixed reality data include at least multiple sensor data readings, and are configured to the traffic extremely Few a part is transferred at least part of multiple electronic devices.

According to another preferred embodiment of the invention, it discloses a kind of in multiple electronic control units and electronic system Between nested communication method, including step, via including at least multiple storages in memory and operate in networking and calculate dress The programming instruction on processor set simultaneously is configured to via network and multiple electronic device communications and is configured to operate multiple First electronic control unit of hardware controls element, hardware controls element configuration are used to receive multiple man-machine interactivelies from human user And be configured to send at least part of multiple interactive information to multiple electronic devices, interactive information, which is based at least partially on, to be come From at least part of multiple man-machine interactivelies of human user, and it is configured to receive from least part of multiple electronic devices The traffic and at least part that at least part of the traffic is transferred to multiple communication devices, via network connection to host Electronic device；It is connected to the first electronic control unit using additional electronic control unit, additional electronic control unit includes Stored at least in memory and multiple programming instructions for operating on the processor of networked computing device and be configured to via Network and multiple electronic device communications, and be configured to operate multiple hardware controls elements, hardware controls element configuration is used for Multiple man-machine interactivelies are received from human user, and are configured to for multiple interactive information to be sent at least the one of multiple electronic devices Part, interactive information are based at least partially at least part of multiple man-machine interactivelies from human user, and are configured to The traffic is received from least part of multiple electronic devices and at least part of the traffic is transferred to multiple communication devices At least part；Multiple user's interactions are received at additional electronic control unit；Multiple interactive information are sent to the first electricity Sub- control device, interactive information are based at least partially at least part of user's interaction；And in the first electronic control unit At least part of interactive information is transferred to host electronic by place.

Detailed description of the invention

Detailed description of the invention several embodiments of the invention, and be used to explaining according to the embodiment hair together with specification Bright principle.Those skilled in the art should know that specific embodiment shown in the drawings is merely exemplary, and should not regard It limit the invention in any way or in this scope of the claims.

Fig. 1 is that the nature trunk according to the preferred embodiment of the invention that is used for is tracked and fed back with the equipment for electronic reciprocal Example hardware arrangement figure, illustrate using multiple tethers and moveable trunk harness.

Fig. 2 is the figure for illustrating various alternative tether arrangements.

Fig. 3 is that the nature tether according to the preferred embodiment of the invention that is used for is tracked and fed back with the equipment for electronic reciprocal Additional exemplary hardware layout figure, illustrate mobile to detect the angle of tether using angular transducer.

Fig. 4 A is the exemplary object for illustrating pairing according to the preferred embodiment of the invention, nested communication control device of wireless The figure that removing the work is set.

Fig. 4 B be illustrate it is according to the preferred embodiment of the invention via adapter communications, the nested communication wireless controlled of pairing The figure of the exemplary physical arrangement of device processed.

Fig. 5 is to illustrate the various exemplary physicals configuration for being configured to for various purposes, nested communication wireless controller Figure.

Fig. 6 is to illustrate preferred embodiment in accordance with the present invention for communication nested between control device and electronic system Illustrative methods flow chart.

Fig. 7 is to illustrate the nested communication control dress according to the preferred embodiment of the invention for using and having various electronic systems The block diagram for the exemplary system architecture set.

Fig. 8 is the block diagram for illustrating the component according to an embodiment of the present invention for being suitable for body-building system.

Fig. 9 A is the block diagram for illustrating the component of motion sensing apparatus according to an embodiment of the present invention.

Fig. 9 B illustrates motion sensing apparatus according to an embodiment of the present invention.

Fig. 9 C illustrates hand held controller according to an embodiment of the present invention.

Fig. 9 D illustrates the example of the hand held controller of handle of the installation according to an embodiment of the present invention to body-building device.

Figure 10 be illustrate it is according to an embodiment of the present invention for the interaction based on user and body-building system and answered with body-building With the flow chart of interactive illustrative methods.

Figure 11 is the block diagram for illustrating the exemplary hardware architecture for the computing device in the embodiment of the present invention.

Figure 12 is the block diagram for illustrating the example logic framework according to an embodiment of the present invention for client terminal device.

Figure 13 is to show the exemplary architecture arrangement of client according to an embodiment of the present invention, server and external service Block diagram.

Figure 14 is the another party for illustrating the exemplary hardware architecture for the computing device in each embodiment of the present invention Block diagram.

Specific embodiment

The present inventor has already envisaged in a preferred embodiment of the invention and implements for the multiple of mixed reality interaction Electronic control and follow-up mechanism, can with multiple main frames device match and can send used nesting communicate.

One or more different inventions can be described in this application.Further, for one or more described herein Invention, can describe several alternative embodiments；It should know, show these only for schematically illustrate purpose and not with any Mode limits the invention for being included at this or the claim shown at this.One or more inventions can be widely applicable for Several embodiments, it is such as obvious from the disclosure.In general, describing embodiment in detail enough so that those skilled in the art One or more present invention can be practiced, and should know can using other embodiments and can make structure, logic, Software, electrical and other variations and without departing from the range of specific invention.It therefore, one skilled in the art would recognize that can To practice one or more inventions using various modifications and deformation.The special characteristic of one or more inventions described herein can By referring to foring a part of this disclosure and wherein by showing the specific realities of one or more inventions in a manner of schematically illustrate Apply example specific embodiment or attached drawing and describe.However, should know, these features, which are not exclusively for use in, describes their institute's references In one or more specific embodiments or attached drawing.The disclosure neither the literal explanation of all embodiments of one or more invention again The list of the feature of the nonessential one or more inventions being present in all embodiments.

The title of the header and present patent application of provided paragraph is in the present patent application only for convenient, and does not answer It is regarded as limiting the disclosure in any way.

The device being in communication with each other need not be continuously in communication with each other, unless expressly specified otherwise.In addition, the device being in communication with each other It can directly communicate or be communicated indirectly by the communication device or intermediary of one or more logics or physics.

The description that embodiment and several components are in communication with each other, which is not implied by, needs all these components.On the contrary, can describe Various optional features are to illustrate the extensive various possible embodiments of one or more inventions and in order to more completely illustrate this hair In terms of bright one or more features.It similarly, although can be with successively sequence description processing step, method and step, algorithm etc. Deng these techniques, method and algorithm can usually being configured with alternative sequential working, unless distinguishingly state otherwise.In other words, Instruction is not needed with the sequence in itself for the random order for the step of can describing in the present patent application or order Execute step.The step of technique, can actually be executed with random order.Further, some steps can simultaneously be held Row is and (such as because a step describes after another step) non-concurrently occurs although describing or implying.In addition, In the accompanying drawings by its explanation to the diagram of technique be not implied by shown technique exclude other deformation and modification, be not implied by shown in Technique or its arbitrary steps are necessary for one or more invention, and it is preferred for being not implied by shown technique.In addition, It is primary that each embodiment usually describes step, but this is not meant as them and must occur once, or executes or transport every time They can only occur once when row technique, method or algorithm.Some steps can save in some embodiments or some situations Slightly or some steps can execute more than once in given embodiment or situation.

When being described herein single device or article, it is obvious that can make instead of single device or article With more than one device or article.Similarly, when being described herein more than one device or article, it is obvious that can be with Single device or article are used instead of more than one device or article.

The function or feature of device alternatively can be not explicitly described as having the function of one or more of this or feature again Other a devices and embody.Therefore, the other embodiments of one or more inventions need not itself include device.

Described herein or reference technology and mechanism sometimes for will clearly describe in the singular.However, should know, Specific embodiment may include the successive ignition of technology or the multiple instantiation of mechanism, unless in addition explaining.Technique in attached drawing Illustrate or box be construed as indicate include that can be held for the one or more of specific logical function or step in implementing process The part of module, segment or code that row instructs.Alternate embodiment includes the wherein example in the scope of embodiments of the invention Such as can with it is shown or described except sequence execute function, including substantially simultaneously or with reverse order, depend on involved And function, as will be understood by those of ordinary skill in the art.

Conceptual framework

Fig. 7 is to illustrate according to the preferred embodiment of the invention communicate using the nesting with various electronic systems 710 to control The block diagram of the exemplary system architecture 700 of device 701 processed, 702a-n.According to embodiment, control device 701 can connect to Multiple electronic systems 710 such as include but is not limited to personal computer 711, video game console 712, media center (example Such as household audio and video system) 713 or mobile device 714 (such as smart phone or tablet computing device).Connection can be via various Device occurs, but according to embodiment and as contemplated by the present inventor, and the ideal method of connection is via with radio Radio mechanism connection.Exemplary communication protocols or technology for the connection may include that (but being not limited to) cellular radio is logical Letter, BLUETOOTH^TM、ANT^TM, WiFi, near-field communication (NFC) or other connection mechanisms.

According to embodiment, multiple additional control device 702a-n can be via various connection mechanisms and main control unit 701 pairings.When connecting in this way, the input from the user via additional control device 702a-n interaction can be sent to Interaction then can be transferred to multiple connected electronic systems 710 by main control unit 701.If there is electronic system 710 mentions For if, feedback can then be received by main control unit 701, and be transferred to additional control device if necessary 102a-n (for example, if user is interactive via multiple additional control device 102a, and electronic system 110 is provided for the use The feedback of family or the device special designing).It should know, and not all control device needs to utilize similar connection mechanism.Example Such as, if main control unit 101 runs BLUETOOTH^TMWith WiFi radio for wirelessly communicating, additional control device 102a It can be via BLUETOOTH^TMConnection and WiFi connection can be used in the second additional control device 102b.In this manner, main control Device 101 can be considered as combining various attachment devices to promote in multiple electronic systems 110 and variation design and operation Multiple additional control device 102a-n between the device that communicates.

Detailed description of illustrative embodiments

Fig. 1 is that the nature trunk according to the preferred embodiment of the invention that is used for is tracked and fed back with the example for electronic reciprocal Property hardware layout 100 figure, illustrate using multiple tether 110a-n and moveable trunk harness 120.According to embodiment, Multiple tether 110a-n can fix or be integrally formed as a part of handle or railing 130, such as all in training equipment Such as treadmill, ellipsograph, the handle found on climbing machine.In alternative arrangement, can be used has whole tether 110a- The equipment of the special designing of n, but could be aware that, it may desire to the tether that can be fixed and freely remove The modularized design of 110a-n is with the structural detail for promoting to be used for various body-building equipments or building, according to the special use of user Way or situation.Tether 110a-n then can be bonded or be integrally formed to trunk harness 120, as shown in the form of belt, Can from user dress so that their bodies moving influence tether 110a-n and in various ways to they application stress.It answers This knows, although various physical layouts can be used, such as to clearly illustrate the belt design for trunk harness 120 Including but not limited to vest, similar to climb or a series of pendency harness shape bands of equipment, knapsack, designed for being worn on Below user's body or substitutes the ligature of clothing (such as medical setting for collecting precise information) or may be easy to be labelled to The fixture or attachment point of multiple special formation of user's clothing.Extraly, trunk harness 120 can use movable part structure It makes, such as moves and be not limiting as the interior band 121 of their movements to a certain degree in harness 120 with permission user.It can be with Various modes measure the movement of interior band 121 (or other moveable parts), are such as passed using accelerometer, gyroscope or optics Sensor, and the data may be used as the interaction with software application, in addition to the data collected as described below from tether 110a-n it Outside.

When the user is mobile, their body deviation post and orientation naturally.These offsets can be via tether 110a- N is detected and is measured, such as by tension on measurement tether 110a-n or the mode of strain to indicate body orientation, or passes through survey The small variation strained on amount tether 110a-n is to determine more accurate mobile body gesture such as when a user speaks, Huo Zheyong The stride at family or the specific characteristic of gait.Extraly, by changing the quantity and arrangement of tether 110a-n, it can detecte and measure More accurate or special mobile form is (such as, using the special cloth for the multiple tethers for being connected to user's body specific region It sets to detect superfine micro-shifting and employ in medical diagnosis or fitness).The data may be used as the interaction with software application, such as Virtual reality applications are inputted as user to control personage in game.In this arrangement, when the user is mobile, which can To convert into game personage or incarnation to transmit more natural interaction sense and in the presence of sense.For example, being swum in multiplayer's role playing In play, this can be used for promoting non-verbal communication and identification between player, because can pass through detection nature trunk in gaming Position and movement and the different style and posture for transmitting them.In body-building or health application, which can be used for tracking And monitor the posture or ergonomics amount of user, or for special body-building such as keep user, stretching, extension or It runs form for appropriate during treadmill and helps to train them.In medical applications, which can be used for helping to examine The disconnected injury or defect that can need to pay close attention to such as is fitted by the exception in detection movement or for the physiology of unidentified injury It answers and (such as when foot or knee of user's subconsciousness by their deviation of gravity center injuries, does not recognize to exist consciously and ask Topic).

Pass through the various arrangements of tether 110a-n and tether sensor (as described below, referring to Fig. 2-Fig. 3), it may be possible to So that user is interacted in a manner of various immerse with software application, and touch feedback is received from application.For example, passing through detection tether Rotation, tension, stress or angle, by using natural body is mobile and positioning such as tilts, jump, squat down, kneel at one's knees, turn It is curved or to deviate their centers in all directions dynamic in the software application for being configured to receive trunk tracking input to trigger Make, user can interact with application such as reality-virtualizing game or simulation.By the touch feedback for applying version and intensity (as described in more detail below, referring to Fig. 2), application can provide a user the physics instruction of software event, such as application tension It to resist movement, draws or is drawn to be moved along a direction or " wrenchs " user on tether, or change is for multiple tethers Feedback such as drawn and discharged with the order of variation or sequence to simulate more complicated effect such as (such as in game purposes feelings In shape) explosion, ride-on vehicles or walk through leaf.

It should know, while reference has been made to virtual reality applications, possibility can be according to the extensive various uses case of embodiment 's.For example, trunk tracking can be used for body-building and health application, to monitor the posture or gait of user when walking, and not Use additional virtual reality device or software.

Fig. 2 is to illustrate the figure of various alternative tether arrangements.According to each purposes case and hardware layout, tether 110a-n It can use the design of various purposes driving as shown.For example, " stretchable " tether 210 can be used in user's mobile phase Between measure strain, tether 210 is stretched or compressed (such as using piezoelectric material and measure Electrical change) at this time.The arrangement can fit For precise measurement, but for always mobile detection or can continue using shortage mechanical strength or durability.Alternative construction It can use non-deformed tether 220 such as wirerope or similar non-stretching material.It, can instead of the strain on measurement tether 220 To allow to a certain degree shifting of the tether 220 substituted in shell 222 (such as attachment point on trunk harness 120 or handle 130) It is dynamic, and can be such as via the position or mobile 221 of optical sensor measurement tether 220.In third exemplary arrangement, it is Rope 230 can be wound around wheel shaft or pulley 231, and can be released when the applied force during user is mobile.It can measure The rotation of pulley 231, or alternatively can use tensioner such as reel spring (not shown) and be when extending or retracting Restrict 230 when measurement tension on the apparatus or strain.In this manner, could be aware that, various mechanical devices can be used to promote Into tether and attachment for detecting and measuring nature trunk position and movement, and should know according to implementation disclosed herein Example can use various additional or alternative hardware layout.

Extraly, by using various hardware constructions, it becomes able to utilize only measurement mobile or " passive " tether of strain, And resistance or mobile " active " tether to provide a user touch feedback can be applied.For example, reel spring is utilized Or in the arrangement of skidding 231, spring or skidding 231 can be wound to withdraw tether and guide or prevent user if desired Movement.In this manner, the various new models of the interaction based on feedback become possible, and lead in virtual reality purposes case It crosses more natural physical feedback during they interact and improves user's engagement and immerse.

The feedback that various forms and intensity are applied by using various tethers arrangement, can be used various feedback kinds with sound Tactile output should be provided a user in software event.It is, for example, possible to use the tension on tether to simulate controlled shifting It is dynamic such as to relate to water or intensive leaf, it is walked upwards along inclined surface, magnetic force or gravity, or can be by directionally or non-directionally opening The other forms physical resistance of power simulation or impedance.The traction on tether, withdrawal can be used or pull unexpected to simulate Power, such as setback force from artillery fire, explosion are grabbed or are tapped by software entity such as object or personage, parachute is unfolded, jumps Pole is jumped, and is slided or is fallen, or can be using other transient forces or event for drawing or pulling sensory transmission.It is various by utilizing More complicated event communication can be sent to user by tactile feedback patterns, such as ridden on the horse back or in the car, stood In marine ship's deck, turbulent flow in aircraft, weather, or other virtual events of touch feedback displaying can be used.With the party Formula can make virtual environment and event to user's more immersion and tangible, by make user using nature body it is mobile and It positions and interacts, and by feel nature and for providing touch feedback in a manner of expected from user.For example, if user is logical Personage of the first person viewpoint in control game application is crossed, can be seemed when their personage is struck naturally, will deposit In physical sensation corresponding with event；However, this for traditional interactive device can not, detract immersing or really for user Any feeling.By providing the physical sensation with virtual events, experience becomes more moving, and encourages user more naturally Interaction, because their movement leads to nature and thinkable feedback, the subconsciousness for meeting them is expected and is avoided excessively " break and the immerse " moment, this then reduce user as be adapted to it is limited interaction plan result and take unusual row For or unhealthy posture.

Touch feedback can be provided to notify the non-gaming event of user, such as for Email or using update Desktop notice, or provide to the feedback of their postures to be trained for body-building or health.For example, when user can be encouraged to take exercise Or specific stance, posture or posture are maintained with the time span of setting (for example, being used for yoga training application), and if he Pose deviation tolerance interval, provide feedback to remind them to adjust their posture.This can be used for without using void Intend other characteristic aspects of reality and the movement as the traditional software application in non-dedicated computing hardware can be operated, be good for Body, health or human engineering application.For example, human engineering training application can be used in the user at their desks, work is spread Make day and monitor their body postures and provide tactile cue to correct the posture when detecting bad posture, helps user's dimension Hold the operating attitude of health with reduce due to bad posture bring fatigue or damage (for example, can with when the work at computer Bad posture associated repeated stress damage when making).

Fig. 3 is that the nature trunk according to the preferred embodiment of the invention that is used for is tracked and fed back for the additional of electronic reciprocal The figure of example hardware arrangement 300 illustrates mobile to detect the angle of tether 320 using angular transducer 312,321a-n. According to an exemplary arrangement, tether 310 can be fixed to or pass through rotary joint such as ball bearing 311 or the like, To allow free angle mobile.During movement, angle movement or deflection that bar, stick or tether outstanding are segmented 313 can be measured 312 (such as using optics, magnetism or other sensors) are to determine the corresponding angle of tether 310.In this manner, essence can be collected True angle measurement and be not blocked from motion range or introduce unnecessary mechanical complexity.

In alternative hardware layout, the vertical angle of the enabled tracking tether 320 of the use of angular transducer 321a-n, with It detects and optionally measures vertically moving or orienting for user's trunk.As 320 contact sensor 321a-n of tether, can step on Remember and substantially vertical movement (no matter that is, whether tether is in the tilted angle upward or downward) be used to detect.In order to more accurate Measurement, thus it is possible to vary the particular hardware of sensor 321a-n constructs, such as is switched using pressure-sensing to detect and be applied with how many Power simultaneously uses the measured value with the corresponding angle of determination (as the given known tether 320 constructed can be possible).It should know, Various hardware combinations can be used to provide the method or degree of desired angle detection or measurement, such as use conductive tether 320 and capacitive sensor 321a-n is to detect contact, or uses mechanical or Rubber dome switch (as typically used in keyboard structure In) detect and be physically contacted not use conductive tether 320.

Interactive possibility can be extended to include the more detailed and natural shifting of user's body by being detected or being measured using angle It is dynamic.For example, all tether 110a-n can simultaneously detect downward angle if user squats down.Extraly, by when can Can when combination from it is multiple using sensor input (such as using adaptable software with from its detect any sensor receipts Collect data, and failed call special sensor type is for operating), such as sensed by combination from tether 110a-n and hardware The data of device such as accelerometer or gyroscope, can be enhanced data accuracy or applicability, make to be able to achieve position or movement The similar or change type of detection or multiple methods of accurate degree.Similarly, when user jumps, then all tethers can be same When detect upward angle.However, could be aware that if user tilts in one direction and not all tether 110a-n will be examined Survey identical angle.For example, can detecte downward angle along the tether 110a-n of user institute inclined direction, and on the opposite side Those of will test upward angle (by user's trunk and therefore the orientation of dressed trunk harness 120).In this manner, It can promote more accurate trunk interaction by improving orientation and mobile detection and identification.Extraly, it could be aware that, sense Device 321a-n can be used for other angles measurement, such as to detection level angle.For example, if user is dressing non-rotating body Dry harness 120 can apply the tether 110a-n of similar stress to all attachments when they reverse their bodies.It does not use Angle detection, the precise characteristics of the movement will be unclear, but level angle is used to detect, it becomes able to identify all systems The 110a-n that restricts just is strained along similar direction (such as when viewed from above with clockwise mode, because user can be in validity period Between watch tether 110a-n), and therefore by interactive interpretation be twist motion (rather than such as user squat down or go down on one's knees, this can With apply similar stress to tether 110a-n but will be with different angle measurements).

Fig. 4 A is to illustrate that pairing according to the preferred embodiment of the invention, nested communication control device of wireless 400a-b are shown The figure of example property physical layout.According to embodiment, multiple control device of wireless 400a-b can such as be wrapped via wireless communication protocol Include (but being not limited to) BLUETOOTH^TM、WiFi、NFC、ZIGBEE^TM, cellular radio is (such as via existing GSM or other honeycombs Network, either by using honeycomb frequency to create ad hoc or " netted " network between the devices if necessary) or its His suitable wireless communication protocol or technology and with multiple electronic host devices 410 (for example including but be not limited to game console Device, personal computer, mobile device such as smart phone or tablet computing device, media apparatus, or can be via physics point From controller or multiple controllers and receive user interaction other electronic devices) " pairing ".According to embodiment, two physics Isolated controller 400a-b can be matched with host 410, so that user holds a controller in each hand to facilitate behaviour Vertical and ergonomic comfort.For example, user can be when sitting down with comfortable angle and distance hand held controller, so that with two hands Manipulation single controller, which is compared, may have a greater degree of comfort.For example, user can be in trained or whole body interactive game A controller is grabbed in period each hand, is such as such as used when running or interacting on a treadmill with virtual fitness training aids NINTENDO WII^TMWhen game console.Also should know, control device 400a-b can be filled optionally with more than one host 410 pairings are set, such as using single controller to interact with multiple computers or mobile device, such as shows or refer to for providing Draw instruction, wherein teacher can directly interact with the device of student.

Fig. 4 B is to illustrate pairing, the nested exemplary physical arrangement for communicating control device of wireless 400a-b, illustrate root According to the figure of the preferred embodiment of the present invention communicated via adapter 420.According to embodiment, can use communication adapter 420 with Promote the communication between multiple controller 400a-b and multiple main frames device 410, such as to connect multiple physical controllers And to host show they as single " Virtual Controller " (as that can be needed according to the particular software application interacted, Such as require Single Controller design video-game) or host with can in addition can not communicate (for example, if one fill It sets and only relies upon BLUETOOTH^TMFor connecting, and another uses WiFI, or if just uses two different communication protocols View) controller between bridge communications.Extraly, communication adapter can be operated by control device, such as multiple volumes wherein In the arrangement that outer or " auxiliary " controller is communicated with the controller labeled as " master ", the master controller is then logical with host apparatus Letter, such as lead in " nesting communicates " arrangement for the communication adapter that wherein master controller is used as between other controllers and host It crosses and receives the traffic from each additional controller and submit all traffics (as described below, referring to Fig. 5-to host apparatus Fig. 6).For example, master controller can operate communication adapter 420 in body-building tracker or wearable device (for example, such as FITBIT^TM,APPLE WATCH^TM, or MICROSOFT BAND^TMDevice) between bridge communications to multiple host apparatus 410, with Convenient for by body-building or health and fitness information (as usually tracked by wearable device) and video-game or other software application integration.It should Arrangement can be used in the bridge data between different vendor or operation different software or the device of operating system, such as with APPLE WATCH will be come from^TMThe health data and MICROSOFT XBOX of device^TMGame console is integrated, or comes from FITBIT^TMThe body-building information and SONY PLAYSTATION of body-building tracker^TMGame console is integrated or hardware, software and Other combinations of manufacturer.

For example, according to specially designed purposes such as with home game console (such as MICROSOFT XBOX^TMOr SONY PLAYSTATION^TM) interaction, special wireless protocols or hardware technology can be used, or hardware device can be used Control device 400a-b is configured to utilize many communication protocols depending on the specific use, and user can provide option to select to wish Hope the agreement or frequency used.For example, the control device 400a-b for being configured to desktop computer may be configured to BLUETOOTH is utilized in 2.4GHz frequency range^TMRadio communication.Hardware switch or button 401b can be optionally provided, by User can choose different frequencies if required for this, for example, the radio frequency within the scope of 5GHz be used for instead of BLUETOOTH^TMAnd it is used on WiFi network, such as to increase range or speed or avoid the interference from other devices (for example, many micro-wave ovens used in family or office can cause the interference in 2.4GHz frequency band).It is various wireless The using of communication device can make user by their selected any positions or in a manner of hold multiple control device 400a-b, It improves ergonomics and slows down discomfort or reduce use and usually require that user grabs them in a specific way Conventional control device can duplicating property stress damage risk (for example, it is typically required that the hand and arm of user be in clumsiness Alignment computer keyboard).

By may be used as volume using the movement or positioning of wireless connection according to the embodiment, multiple control devices of raising The outer method interacted with user, such as use via internal hardware sensor such as accelerometer or gyroscope and be based on posture Input method or location tracking.In this manner, the input of additional form can be enabled according to user preference, improve with it is specific Quantity or quality that control device may interact and optionally reduce demand for special hardware button.For example, can be with The enabled control device of posture is programmed using the special exercise posture for the pressing for simulating " volume increase " button, is eliminated pair In the demand for the hardware button for executing the function.

According to embodiment, multiple hardware buttons can be provided on control device 400a-b, or be can change physics and set Meter or arrangement.For example, can use diaphragm or based on the button of mechanical switch to provide tactile sensation if by user's excitation (as design specialized is common in hardware keyboards in such as game or Specialty Design), or can use capacitive base In touch button with when tactile sensation can not be required (as common in mobile application, such as in smart phone or flat Screen lights control on plate computing device) interaction is provided in more compact physical layout.

Hardware button or other control can depending on the specific use or desired function and with various cloth placement locations, such as count Word or the button 404 for simulating " trigger " type can be placed on the downside 400c of control device (to be shown in sectional view Out), such as the game application for wherein may desire to trigger incentive functions.Another exemplary arrangement can be in a specific way Many button 402a-n for navigation are set, in order to form the directional pad (D- pad) for various game or application, such as Video is played for navigation media player interface or with top-down visual angle, such as strategy or simulation Trivial Pursuit Unhinged.Work as user It can need to be rapidly selected and not check when actuator buttons, other buttons may include holding for game or the large-scale of application Easily stretch the button 401a taken.There is provided various identification features such as these buttons can be read text label, color or physical form For example including (but being not limited to) rectangular 401a, circle 401b, triangle 403a or star 403b control button.

According to embodiment, particular exemplary arrangement can be the controller for acting essentially as and individually " separating controller " and match It is right, wherein one hand of user grab a device and divide between two each function to promote two handed input nothing Two hands need to be used on single device.Control can be arranged in a joint manner, and the top of each device is arranged in arrow button Portion, function button are located at the center of arrow button to provide the easy to reach button for interaction (such as to shield On menu item on curtain " receiving " or " click ").Extraly, these combination controls arrangement can then with multiple internal hardwares Sensor coupling, such as accelerometer, gyroscope, hall effect sensor, or the capture and explanation that pass through sensing data Promote other hardware devices of physics interaction, such as to enable by using predetermined or customized posture or location tracking Interaction.

It can be arranged with each combination using various alternative or additional control device by nested communication protocol.For example, The purposes (such as particular game or application, or for specific device or console) planned according to them, user can be with " master " control device of their preferences is selected, and then matches " secondary " control device with it.It could be aware that, press in this way Button or other interaction features or the physical Design such as specific number of ergonomics shape or layout, style or arrangement can be wide Change generally, and user can based on they particular demands or purposes and select to their attractive specific arrangements and Combination.Various control devices arrangement is described referring to Fig. 5, and should be known, it can be according to the hobby of user interchangeably Using any specific control device as main or secondary control device.Extraly, by using various wireless communications as described above Technology, the control device that additional hardware device can be communicated with for nesting match.For example, body-building tracker (for example including But it is not limited to FITBIT^TM, MICROSOFT, BAND^TM, or APPLE WATCH^TMDevice) it can be assisted via wireless personal area network Discuss such as BLUETOOTH^TMOr ANT^TMAnd it matches, such as to provide the biostatistics of user or health and fitness information and via control The interaction of device 400a-b processed.

Fig. 5 is the various exemplary physicals for illustrating to be configured to nested communication wireless controller 500a-c for various purposes The figure of configuration.According to an exemplary embodiment, controller 500a can remotely be controlled with configuration optimization for use as media playback, example Such as connected loudspeaker or playback reproducer, or for home theater or media PC setting.According to the arrangement, can be set The interface on the screen of navigation medium device many button 501a-n, such as to select audio or video content from playback library. Many additional buttons 502a-n can be set in aided location, to provide additional function such as broadcasting/pause, stopping or dish Single function is in order to the additional information checked on selected media.Depending on the specific use, control device 500a may be used as Master controller, optionally with or without the secondary control device for nesting communication, the body-building follow-up mechanism of such as user is used for Heartbeat or other biological statistical variations or game console, which are monitored, during media viewing passes through media services for playing Interaction content that device or playback reproducer are shown (such as in ROKU^TMOr it is connected on the similar operation home theater device of television set and broadcasts Put game).

It is arranged according to another exemplary, particular game purposes can be used for Configuration Control Unit 500b, such as formed to have and be closed Suitable physical form and have setting for be most preferably suitble to analog type game control element (button, switch and other Interface hardware element).According to the configuration, control device 500b can have the control element of number or simulation " control stick " type 510 for the personage of control player or vehicle in 360 degree of three-dimensional virtual environments, such as third person movement or adventure In type game.It can be provided for ease of single-hand handling & purposes and multiple additional buttons 511a-n are set, in order to access player's Inventory executes in special game movement (such as jump, run, rolling or other movement controls), or according to particular game Or application characteristic and execute the movement of context-sensitive.For example, simulator orientation control device 500b can with as secondary The media remote control apparatus 500a of controller is matched, so that user uses media remote-control device 500a as their main control Device and navigation medium interface select game from media content inventory, and then using secondary control device 500b to play game. When Exit Game, user then can continue further navigation medium interface using their master controller 500a, without pair Long-range control or switch controller are positioned in different purposes, because controller can be matched and be communicated, or even working as one can be temporary When not in use.

It is arranged according to another exemplary, control device can be configured and be distinguishingly used as secondary control device, be used for example as configuration and use It matches in emulator controller 500b for usually " hand to be in flow controller and behaviour for the held by both hands used in flight simulation game On vertical pole " (HOTAS) setting throttling controller 500c.According to the configuration, control device 500c may include control stick or cap Sub switch control element 520 is used for using the thumb navigation of user or for controlling many game or application function, usually positions So that the thumb of user is aligned naturally in order to use with control element 520 when keeping control device 500c.Control dress Setting 500c may further include the ergonomics part 521 for being integrally formed or removedly attaching, for example, can be removed or Orientable rigidity or flexible panel are typically configured to be consistent with the hand of user or the palm and be aligned the hand of user for various The appropriate manipulation of control element.For example, control device 500c, which can be configured, to be had by rigid plastic in an exemplary arrangement Theme made of material, and have setting be suitble to user's palm with for during use suitably grab and alignment it is removable Except plastic front board 521.The configuration can also be designed for receiving various alternative rigid panels 521, so that user exchanges crawl Panel with the most comfortable cooperates their specific anatomical structure (such as the configuration is to configure the physical form of control device 500c It is used in firearm industry so that the fixture of user configuration pistol is for appropriate posture and aiming).Control device 500c can To further comprise multiple additional control element 522a-n, such as button or switch, such as be configured so that properly to be aligned When (such as grabbing panel by using ergonomics, as described above), the finger of user can be aligned in suitable position naturally It sets with the desired excitation for control element 522a-n.In this manner, could be aware that can use human engineering with various arrangements 521 configuration of the department of the Chinese Academy of Sciences point, and when the hand of user is properly aligned and grabs control device 500a-c for the control of best use The various arrangements of element processed, so that any specific control device 500a-c is adapted to be suitble to the anatomy broadness model of user It encloses for best use, and promotion ergonomic comfort's property and health.For example, by encouraging user to find most preferably Human engineering cooperation, can reduce the risk of repeated stress damage He other health concerns.

Fig. 6 be illustrate it is according to the preferred embodiment of the invention for communication nested between control device and electronic system Illustrative methods 600 flow chart.It, can be by control device and as the electronics of main control unit in initial step 601 System (such as such as personal computer, game console or home theater or media center) pairing, such as automated session (such as when electronic system or controller are first powered on) or manually (such as by guide user action to initialize pairing).? In optional next step 602, multiple additional control devices and main control unit can be matched, for example, for user with Using the control device in each hand, or for multiple users with the one or more control devices of each pairing to be grouped cloth It sets their own and uses (such as multiple users for playing video game together).In next step 603, main control unit Multiple users interaction can be sent to the electronic system of pairing, such as when user's push button or when making a choice.Optional Next step 604 in, either one or two of multiple additional control devices can send multiple users and be input to master device, and main Control device can then send the electronic system that user is input to pairing.In next step 605, the electronic system of pairing Can send it is multiple feed back to that main control amber is dirty to be controlled, such as the instruction of touch feedback (such as with excited vibration motor) or Information for being shown on the visual displays as operated by control device.In optional final step 606, main control dress Set can send it is multiple feed back to additional control device, for example, if from design for being connected additional control device Electronic system sends feedback.In this manner, could be aware that, all traffics pass through main control between control device and electronic system Device and promoted by it.

Fig. 8 is to illustrate the block diagram of the component of the applicable body-building system 800 according to one embodiment.Applicable body-building System 800 includes hardware device 802, host apparatus 810 and body-building server 818.

Hardware device 802 includes motion sensing apparatus 804, left hand held controller 806A and right hand held controller 806B. Motion sensing apparatus 804 is mounted on the body kinematics on user's body and detecting user, this allows user by executing movement It is interacted with the body-building operated on host apparatus 810 using 813.Illustrative motion sensing is described in detail referring to Fig. 9 A and Fig. 9 B Device 804.It is included that user can press to apply 813 buttons interacted with body-building by controller 806A, 806B.For the ease of connecing Closely, the one or both of hand held controller 806A, 806B may be mounted on the handle of one piece of trained equipment.Respectively refer to Fig. 9 C Illustrative hand held controller 806 and installation system are described with Fig. 9 D.

In an alternative embodiment, additional hardware device 802 can be used.For example, hardware device 802 also may include tool There is the sport footwear of the embedded type sensor or motion sensor that are directly attached to body-building device.In addition, more shown in fig. 8 Hardware device 802 can be omitted.For example, a hand held controller can be used only, or can be using comprising will usually be distributed The combination hand held controller of all buttons between two hand held controllers 806A, 806B substitutes two hand held controllers 806A、806B。

As a whole, hardware device 802 can be used for (such as step instrument, ellipsograph, treadmill, rowing the boat body-building device Machine etc.) for adaptation into game console, game console receives being physically entered (both movement and button press) and producing for user Raw corresponding input data is used to interact with body-building using 813.For example, input data can be used for controlling in body-building application virtually Personage or object.

In some embodiments, the multiple collection that can add hardware device 802 are bonded to system 800 to fill additional body-building Adaptation is set into additional game console.For example, multiple fixed bicycles can be adapted to the control for being used for bicycle contest application In device processed, multiple users is thus allowed to participate in the virtual race vied each other.

Input data from hardware device 802 is in communication link 808A, 808B, 808C (hereinafter collectively referred to as 808) On be sent to the host apparatus API 812 on host apparatus 810.Communication link 808 can be wired or wireless connection. In one embodiment, communication link 808 is to be based on bluetooth low energy (BLE) agreement, and hardware device 802 is BLE device. If in systems including the additional set of hardware device 802, additional communication link 808 is established to connect hardware device 802 To host apparatus 810.

In an illustrated embodiment, each hardware device 802 is attached directly to host apparatus 810 by communication link 808. In this embodiment, three hardware devices 802 are simultaneously connected to host apparatus on communication link 808A, 808B, 808C 810。

Hardware device 802 can be both connected on identical communication protocol (such as bluetooth low energy), or be led in different Believe on agreement.

In another embodiment, only motion sensing apparatus 804 via the first communication link 808A is attached directly to host Device 810.In this embodiment, two hand held controllers 806A, 806B via other two communication link 808B, 808C coupling It is bonded to motion sensing apparatus 804, and configures motion sensing apparatus 804 with will be from the input of hand held controller 806A, 806B Data relay to host apparatus 810.The embodiment is described in more detail referring to Fig. 9 A.

Host apparatus 810 includes that interface software 811, host apparatus API 812, and one or more body-building apply 813. As a whole, host apparatus 810 is the computing device for being able to carry out program instruction.Host apparatus 810 may, for example, be intelligent electricity Words, tablet computer, laptop computer or desktop computer.

Interface software 811 and host apparatus API 812 and interface software cooperating are for use as in hardware device 802, strong Body is using the interface between 813 and body-building server 818.When do not have connect hardware device 802 when, host apparatus API 812 with Fixed intervals initiate the scanning to hardware device 802.The upper button press of hand held controller 806A, 806B or motion sensing apparatus Movement will trigger respective device to broadcast and be used to connect (if not yet connecting) on 804.If the broadcast of device and host fill One of sweep spacing of API overlapping is set, then host apparatus API 812 initiates to the connection of respective device to attempt.In detection button The process that connection is established after pressing or movement allows 802 intuition of hardware device being connected to host apparatus 810.Host dress It sets API 812 and maintains body-building using the connection between 813 and hardware device 802, as long as keeping activating using 813.It is establishing With after the connection of one or more hardware devices 802, interface software 811 and host apparatus API 812 are received and processed and From the input data of hardware device 802 so as to using input data to control and apply 813.For example, input data can be by Using 813 using to control the virtual portrait or object in game.

Host apparatus API 812 can distinguish the input data from different hardware device, because can configure each hard The firmware of part device is to include identifier (such as binary system in the title for any input data for being sent to host apparatus 810 Numerical value).Therefore, motion sensing apparatus 806 can have the identifier different from hand held controller 806A, 806B.Interface software 811 then device-based identifier can establish the application state for representing each device, and host apparatus API 812 can be with By input data map to its respectively application state expression in identify primary hardware device.For example via table or index After input data is mapped to corresponding hardware device, data can be used as the input for applying 813 to body-building.Wherein Motion sensing apparatus 804 is configured so that the input data from hand held controller 806A, 806B to be relayed to the reality of host apparatus 810 It applies in example, motion sensing apparatus 804 may be configured to read in the inlet flow from two hand held controllers 806A, 806B Identifier, and single integrated inlet flow is relayed into host apparatus 810.The process is described in more detail referring to Fig. 9 A and Fig. 9 D.

In some embodiments, left hand and right hand held controller 806A are manufactured, 806 (such as is manufactured to be equivalent with reducing Cost) and identifier having the same.In these embodiments, can be difficult to from two hand held controller 806A, It is automatically distinguished between the input data of 806B, therefore interface software 811 can promote user manually to identify two hand-held controls Device (such as press the button on left hand held controller 806A first and then press the button on right hand held controller 806B) processed. The setting can be stored a part of the application state expression as device by interface software 811.

Interface software 811 can be used similar process and be connected to 810 (example of host apparatus with multiple set when hardware device Such as be used for multi-player gaming) when distinguish the input data from different hardware device.For example, interface software 811 can be shown pair In the prompt (such as being attached to fixed bicycle) of mobile first motion sensing apparatus 804, then display is for mobile second movement The prompt (such as being attached to elliptical exercise device) of sensing device 804, and then show similar prompt to identify in two body-building Respective hand held controller 806 on device.

In one embodiment, it cuts open to write using customization bluetooth GATT and will come from hand held controller 806A, 806B and movement The signal of sensing device 804 is wirelessly transmitted to host apparatus 810 as input data.Then by host apparatus API 812 Manage the data sent from hardware device 802.

The example of data flow presented below between hardware device 802 and host apparatus 810:

1. motion sensing apparatus 804 is used as server and host apparatus 810 is connected to client.

2. the data that host apparatus 810 (client) orders the special service feature of such as accelerometer of motion sensing apparatus 804 Notice.

3. obtaining the reading (such as accelerometer readings) from service features from motion sensing apparatus 804.

4. reading is sent to host apparatus 810 (client subscribers).For example, three axis accelerometer reading can be used as three words It saves and sends.

5. the decoding of interface software 811 reads and can be applied to reading using 813 via API 812.Reading is by applying 813 It is construed to input data and allows that user makes a choice or control/manipulation is using element (such as the object, personage in 813 Deng), or for application 813 in game presentation or other interaction elements make a response.

Body-building is to receive input data from hardware device 802 by host apparatus API 812 and be based on input using 813 Data execute the application of some class users interactions.For example, body-building device 813 can be exercise game or can be with significant Any non-gaming application that mode is interacted with body-building device (such as handed over using 813 with the hardware device 802 of installation on a treadmill Park is mutually walked through with simulation).

It can configure using 813 (including game) to encourage user to take exercise.When user takes exercise, on host apparatus 810 Record reflects the progress of user experience from the received data of hardware device 802.It can also be connected by network 816 using 813 To body-building server 818, user data and statistical data can store and access wherein.

How hardware device 802, which can be user with an example of the software interactive on host apparatus 810, to transport Dynamic sensing device 804 is attached to the pedal of fixed bicycle.The movement of accelerometer measures pedal in motion sensing apparatus 804, " virtually riding " is allowed to control using 813 reality.Should hand over how device 802 is that user will transport with another example of software interactive Dynamic sensing device 804 is attached to their buttocks, and motion sensing apparatus 804 is for dancing or tempers the computer games check movement.

Host apparatus API 812 may be configured to operate in any suitable operating system, such as iOS, Windows/Mac/Linux or Android.The different editions of host apparatus API 812 can be created for different operating system. Although being shown as the discrete entity in host apparatus 810, host apparatus API 812 is usually to compile in building to body-building to answer With the discrete code library in 813.This allows third party developer that can write body-building using 813, uses host apparatus API Function in 812 is to exchange data with hardware device 802 and body-building server 818.

Body-building server 818 includes cloud service module 820, workout data library 822 and application database 824.Body-building clothes Business device 818 and its component 820,822,824 can be embodied as single device or as cluster (such as the net of interconnection device Network server cluster).

Cloud service module 820 receives historical data such as body movement and game performance data from host apparatus API 812 And the storing data in suitable database 822,824.

Cloud service module 820 also allows user on any device and accesses their information at any time.Module Processing user is for relatively and being shared in such as social networking system of third-party server 826 such as FACEBOOK and TWITTER On score request.

Cloud service module 820 can also be with master control (host) multi-player gaming.In one embodiment, in asynchronous context Master control game.User setting challenge after, cloud service module 820 can be completed to user time window he " bout " or It simply waits until he completes to setting the goal.The information is stored in application database 824.Cloud service module 820 is executed in player Between the logic of game, challenge or target that creates and maintain their state and progress.

In an alternative embodiment, the also real-time multi-player gaming of master control of cloud service module 820, wherein real-time game data exist It is moved back and forth between player on different host apparatus 810.

Database 822,824 can for scalability and fault-tolerance the cluster-based storage across several servers.

Cloud service module 820 also executes function associated with processing motor unit.Motor unit is strong across different type The standard unit of body device measurement.The standard mobile signature of these motor unit origin autokinesis sensing devices and limit.One In a embodiment, motor unit is normalized to provide consistency between different type and the body-building equipment of producer.Normalization Table can be to provide cross-reference for retrieving and analyzing later to compare based on standardization between device.Implement at one In example, standard mobile signature is realized when accelerometer on motion sensing apparatus 804 and specific gravity threshold crossings.For example, mark Quasi- mobile signature can be defined as deviateing 1G from the moving average for the 20 accelerometer samples collected on 333ms.It is logical Often, motor unit can be limited, calorie or HR Heart Rate including burning based on any number of parameter.Additional software Function can be added to the accuracy that host apparatus API 812 or body-building improve motor unit using 813.For example, if combustion The calorie of burning is major parameter, then can add option to select the body exercising machine type just used.

User can create individual goal to realize the motor unit of setting number in given time period.For example, target It can be 50,000 motor units in one week.These individual goals can then be shared with the friend in social networks.

User may be in social networks team's creation cooperative target to realize the motor unit of set amount.For example, five Target can be set to realize 250k motor unit weekly in people's group.

Moving cell is also used as the normalized movable benchmark for benefiting from data for comparative purposes.Example Such as, moving cell can be used for the contest between team or friend in a network.Can initiate challenge and on social networks to The friend of user broadcasts.

In one embodiment, cloud service module 820 searches for user from the social networking system on third-party server 826 Socialgram and replicate socialgram on body-building server 818.Therefore, follow-up function associated with social gaming machinery (such as creation of challenge and cooperation target) can execute on body-building server 818.Fig. 9 A is to show to be implemented according to one The component of the motion sensing apparatus 804 of example.Motion sensing apparatus 804 includes sensor 902, communication module 908 and firmware 910。

The body kinematics of the detection user of sensor 902.In an illustrated embodiment, sensor 902 includes accelerometer 904 With gyroscope 906.It however can include additional or different sensors 902 in motion sensing apparatus 804.

Communication module 908 is interacted with communication link 808A to transmit between motion sensing apparatus 804 and host apparatus 810 Data.For example, communication module 908 can receive input data from sensor 902 and send input data to host apparatus 810 To be used by body-building using 813.In one embodiment, communication module 908 is connected via bluetooth low energy (BLE) agreement.Firmware The logic of 910 operations comprising having managed motion sensing apparatus 804.Such as firmware 910 may include logic to support monitoring Sensor 902 is simultaneously established if detecting that user has rocked motion sensing apparatus 804 to the communication chain of host apparatus 810 " rocking to connect " function of road 808A.

In the alternative embodiment referring to described in Fig. 8, two hand held controllers 806A, 806B are coupled to motion sensing apparatus 804 and motion sensing apparatus 804 is configured so that the input data from hand held controller 806A, 806B is relayed to host apparatus 810.In this alternative embodiment, configuration firmware 910 independently of host apparatus 810 to detect and be connected to hand held controller 806A, 806B and communication module 908 are establishing and are maintaining the connection after the connection of hand held controller 806A, 806B.With The connection of hand held controller 806A, 806B can be the identical communication protocol of communication link 808A (such as bluetooth low energy) or not Same communication protocol (such as RF connection).Meanwhile firmware 910 manages three connections (with two hand held controllers 806A, 806B Connection, and the communication link 808A with host apparatus 810) and by the input data from hand held controller 806A, 806B Relay to host apparatus 810.In some embodiments, firmware 910 can be configured so that sensor 902 and hand-held control will be come from The input data of device 806A, 806B are combined into individually integrated inlet flow.Therefore, the reception of host apparatus API 812 includes coming from The single inlet flow of the input of all three hardware devices.

Fig. 9 B illustrates the motion sensing apparatus 804 according to one embodiment, including shell or shell 912 and internal electricity Subassembly 913.As described above with reference to Figure 8, motion sensing apparatus 804 is attached to user's (such as around leg) and using biography Sensor 902 is recorded as input data and is sent to host apparatus 810 to measure the movement of user.As shown in fig. 9b, Motion sensing apparatus 804 is wrapped in small shell 912 to allow to be attached to user in many ways.The embodiment shown in In, motion sensing apparatus 804 is wirelessly coupled to host apparatus 810 and is battery powered.

Fig. 9 C illustrates the hand held controller 806 according to one embodiment.Hand held controller 806 includes handle 914 and hangs Band 916.One or more buttons 918 are mounted on handle 914.Other than button 918 or it is replaced, handle 914 can also To include directional pad or control stick.As described above with reference to Figure 8, hand held controller 806 is communicated with host apparatus 810.Shown Embodiment in, hand held controller 806 wirelessly communicate and battery power.

Internally, hand held controller 806 includes wireless device and microcontroller.In one embodiment, radio is It is connected to motion sensing apparatus 804 and sends the RF that input signal relays to host apparatus 810 to motion sensing apparatus 804 Wireless device.In an alternative embodiment, hand held controller includes bluetooth low energy (BLE) wireless device and system on chip (SoC).In some embodiments, any other electronic component of wireless device, microcontroller and hand held controller 806 It can be placed on single printed circuit board.

Fig. 9 D illustrates hand held controller 806A, 806B's of the handle according to the installation of one embodiment to body-building device Example.The suspender belt 916 of hand held controller 806A, 806B are around handle.In one embodiment, the bottom side of handle 914 It may include coherent substance to increase the friction between hand held controller 806A, 806B and handle.

In order to reduce manufacturing cost, some or all of components of two hand held controllers 806A, 806B can be equivalent. In one embodiment, two hand held controllers 806A, 806B are completely equivalent (such as identical mechanical covers, internal electricity Subcomponent and firmware).In this embodiment, firmware 910 or host apparatus API 812 can be difficult to differentiate between two hand-held controls Device, and interface software 811 can prompt user to identify each hand held controller as described above with reference to Figure 8.In another implementation In example, two hand held controller internal electronic elements having the same (such as identical microcontroller, wireless device etc.), but It is that can be placed in different shells to make by a controller identification as left hand held controller 806A and by another identification For right hand held controller 806B.For example, the button 918 on two hand held controllers can have different layouts or different face Color or shell can have symmetrical ergonomic features so that they are more suitable for the hand of user.Two hand-held controls Device 806 can also have with firmware with flash memory so that motion sensing apparatus 804 or host apparatus API 812 can be distinguished from two The input data of controller 806.For example, firmware on left hand held controller 806A can be configured in the title of its inlet flow It sends identifier (such as Binary Zero), and can be only fitted to the firmware on the second hand held controller 806B in its inlet flow Different identifiers (such as binary one) is sent in title.

Figure 10 is to illustrate according to an embodiment of the present invention to be used to answer with body-building with interacting for body-building system based on user With the flow chart of the exemplary processes of 813 interactions.User starts for hardware device 802 to be attached to body-building device or the body of user Body；Hand held controller 806A, 806B or motion sensing apparatus 804 can be held in user hand or be attached to body-building device, 813 are applied depending on specific body-building.

According to embodiment, when user interacts with body-building system, motion sensing apparatus 804 obtains 1005 real time kinematics numbers According to including the acceleration information from accelerometer 904 or the spin data from gyroscope 906.Body-building system is not limited to Fixed exercising machine such as treadmill, but also may include dynamic exercise mechanism, such as run on road.By motion sensing The data that device 804 obtains can be sent to host apparatus 810 having been established on communication link 808.

Based on the data for obtaining 1005, the interface software 811 on host apparatus 810 can determine that 1010 activity-specifics move Signature.Activity-specific movement signature can identify current active performed by the user.According to embodiment, movement signature be may be used as The benchmark mobile for the future that can be compared by body-building using 813.

According to embodiment, interface software 811 can generate one or more activity vectors 1015 based on movement signature.Activity Vector indicates measurement associated with the activity level of user.For example, activity vector can indicate the calorie to be burnt by user Number, the strength level of the distance or user that are covered on specific time interval.Activity vector is across different type body-building dress Set the standard unit of measurement.

According to embodiment, user is interacted with body-building using 813 based on the activity vector for generating 1015 by interface software 811 1020.The intensity of body-building is executed by changing user, user changes activity vector on a period of time.Activity vector Variation changes the state or behavior of the object in body-building is using 813, thus user is allowed to answer on host apparatus 810 with body-building With 813 interactions.For example, wherein the personage of user is run on the screen across handicap run if user is run video-game in object for appreciation Road, user is run on a treadmill or the intensity effect ridden on the bicycle task of user (using object in 813) exists The speed of screen (behavior presented by object) is crossed in game.Therefore by faster running, user changes activity vector in real time, Speed corresponding to user personage improves.User can also be interacted with the application based on the input from hand held controller 806.Example Such as, as the personage of user in video-game close to barrier when, user can press the button on hand held controller 806, So that the personage of user skips or avoiding obstacles.

Hardware structure

It is closed in general, the group in hardware or software and hardware can be implemented in technology disclosed herein.For example, they can To implement in operating system nucleus, in discrete consumer process, in the library data packet being tied in network application, in spy On the machine very constructed, on specific integrated circuit (ASIC), or on a network interface card.

It can be implemented in at least some of software/hardware hybrid embodiment of this disclosed embodiment by being stored in (it is construed as wrapping on the machine for the programmable resident network that computer program in reservoir is selectively activated or reconfigured Include the machine of the discernable network of interval connection).The network equipment, which can have, can configure or designed for utilizing inhomogeneity Multiple network interfaces of type network communication protocol.Some generic structures for these machines can be described herein in order to say It is bright it is possible thereby to implement one or more exemplary schemes of given functional unit.According to specific embodiment, disclosed herein At least some can be implemented of the feature or function of each embodiment is the associated one or more of one or more networks On general purpose computer, such as end-user computer system, client computer, network server or other server systems System, mobile computing device (such as tablet computing device, mobile phone, smart phone, laptop computer or other are suitable Computing device), consumer electronics device, music player or any other suitable electronic device, router, interchanger, or Other suitable devices or any combination thereof.In at least some embodiments, the feature of each embodiment disclosed herein Or at least some of function can be implemented in one or more virtualized computing environments (such as network query function cloud, at one or more The virtual machine of master control or other suitable virtual environments on a physical computing machine) in.

Referring now to fig. 11, block diagram is shown, illustrates to be suitable for implementing that disclosed feature or function is at least herein The exemplary computing devices 10 of a part.The computing machine listed in the previous paragraphs that computing device 10 may, for example, be it is any one It is a, or can even be executed according to program stored in memory based on software or hardware based instruction it is any its His electronic device.Can configure computing device 10 with multiple other computing devices such as clients or server in communication network On communicate, communication network such as wide area network, Metropolitan Area Network (MAN), local area network, wireless network, internet or be used for using known protocol Any other network of the communication, whether wirelessly or non-wirelessly.

In one embodiment, computing device 10 includes one or more central processing unit (CPU) 12, one or more Interface 15, and one or more buses 14 (such as peripheral components interconnect (PCI) bus).When in suitable software or firmware Under control when movement, CPU 12 can be responsible for implementing special function associated with the function of the computing device of particular arrangement or machine Energy.For example, at least one embodiment, can configure or design computing device 10 for use as CPU 12 is utilized, locally deposits The server system of reservoir 11 and/or remote memory 16 and interface 15.In at least one embodiment, CPU can be made 12 execute one or more different types of functions and/or operations under the control of software module or component, software module or Component for example may include operating system and any suitable application software, driving etc..

CPU 12 may include one or more processors 13, such as come from Intel, ARM, Qualcomm and AMD The processor of one of microprocessor series.In some embodiments, processor 13 may include that the hardware of special designing is such as special With integrated circuit (ASICs), electrically erasable programmable read-only memory (EEPROM), field programmable gate array (FPGA) etc. Deng for controlling the operation of computing device 10.In a particular embodiment, 11 (such as based non-volatile random access of local storage Memory (RAM) and/or read-only memory (ROM), including such as one or more levels cache) it can also form CPU's 12 A part.However, being coupled to many different modes of system 10 in the presence of wherein memory.Memory 11 can be used for various Purpose such as, caching and/or storing data, programming instruction etc..Should further it know, CPU 12 can be various cores One of system on chip (SOC) type of hardware may include additional hardware such as memory or graph processing chips, such as Qualcomm SNAPDRAGON^TMOr Samsung EXYNOS^TM, if this field becomes increasingly prevalent, such as filled for mobile It sets or integrating device.

As used herein, term " processor " is not limited only to processor, mobile processor or micro- place referred in the art as Those of device integrated circuit is managed, but is broadly directed to microcontroller, microcomputer, programmable logic controller (PLC), dedicated integrated Circuit and any other programmable circuit.

In one embodiment, interface 15, which provides, is used as network interface card (NICs).In general, NICs control is in computer network Data packet is sent and received on network；Other types interface 15 can for example support that other peripheries for computing device 10 are set It is standby.Have among the interface that can be provided, Ethernet interface, Frame Relay Interface, cable interface, DSL interface, token ring interface, Graphic interface etc..It is furthermore possible to also provide various types interface, such as, universal serial bus (USB), serial line interface, with Too network interface, FIREWIRE^TMInterface, THUNDERBOLT^TMInterface, pci interface, parallel interface, radio frequency (RF) interface, BLUETOOTH^TMInterface, near-field communication (such as using nearly magnetic field) interface, 802.11 (WiFi) interfaces, Frame Relay Interface, TCP/ IP interface, isdn interface, fastethernet interface, gigabit ethernet interface, serial ATA (SATA) or outside SATA (ESATA) interface, high resolution multimedia interface (HDMI), Digital Visual Interface (DVI), analog or digital audio interface are different Walk transmission mode (ATM) interface, HSSI High-Speed Serial Interface (HSSI) interface, point of sale (POS) interface, fiber data distribution interface (FDDIs), etc..In general, the interface 15 may include for the physical port with suitable media communications.In some situations In, they also may include independent processor (such as special audio or video processor, as this field is hard for high-fidelity A/V Part interface is common), and in some cases, including volatibility and or nonvolatile memory (such as RAM).

Although system shown in Figure 11 illustrates the computing device 10 for implementing one or more inventions described herein One particular architectures is at least part of unique apparatus frame that can implement feature or technology described herein on it absolutely not Structure.It is, for example, possible to use the frameworks with one or arbitrary number processor 13, and the processor 13 can reside in individually In device or it is distributed among arbitrary number device.In one embodiment, the manipulation of single processor 13 communication and routing meter It calculates, and discrete private communication processor can be provided in other embodiments.It in various embodiments, can be according to this hair Implement different types of feature or function in bright system, system includes that (such as operation client software is flat for client terminal device Panel assembly or smart phone) and server system (all server systems as described in more detail below).

Unrelated with network equipment configuration, system of the invention can use one or more memories or memory module (such as, for example, remote memory block 16 and local storage 11), is configured to storing data, for general-purpose network operations Program instruction or other information relevant to the function of embodiment described herein (or above any combination).Program instruction can For example including operating system and/or one or more application or to control its execution.Memory 16 or memory 11,16 It may be configured to storing data structure, configuration data, encryption data, legacy system operation information or described herein What his special or common non-program information.

Because can use the information and program instruction to implement one or more systems or method described herein, at least Some network equipment embodiments may include non-provisional machine-readable storage media, can configure or designed for storage program Instruction, status information etc. are for executing various operations described herein.The example of the non-provisional machine-readable storage media Including but not limited to, magnetic media such as hard disk, floppy disk and tape；Optical medium such as CD-ROM disk；Magneto-optical media such as light Disk and particular arrangement are to the hardware device for storing and executing program instructions, such as ROM device (ROM), flash memory (as common in mobile device and integrated system), solid-state driving (SSD) and " mixing SSD " storage driving, can be in list The physical unit that solid-state and hard drive are combined in a hardware device (such as becomes more next in the field about personal computer It is more universal), memristor, random access storage device (RAM) etc..It should know, which can be whole and not removable (the RAM hardware module that can be such as soldered on motherboard or in addition be integrated in electronic device) removed or they can be Removable such as commutative flash memory module (such as " thumb drives " or other designed for fast exchange physical storage device Medium can be removed), " heat is pluggable " hard drive or solid-state driving, can be removed optical storage disc or other are this removable Medium, and the entirety and can be removed storaging medium can interchangeably utilize.The example of program instruction includes such as can be by The object identification code that compiler generates, the machine code that can be such as generated by compiler or linker, such as can be by for example JAVA^TMThe syllabified code that compiler generated and can be used Java Virtual Machine or equipment executes, or containing can be by calculating The file for the more advanced code that machine is executed using interpreter is (such as with Python, Perl, Ruby, Groovy or any other foot The script that this language is write).

In some embodiments, the system according to the present invention can be implemented on distinct computing systems.Referring now to fig. 12, Block diagram is shown, the exemplary framework of one or more embodiments or its component on distinct computing systems is depicted. Computing device 20 includes processor 21, can run the one or more functions for performing the embodiment of the present invention or apply such as example Such as the software of client application 24.Processor 21 can execute computations under the control of operating system 22, such as The WINDOWS of Microsoft^TMThe version of operating system, Mac OS/X or the iOS operating system of Apple, (SuSE) Linux OS Some mutation, the ANDROID of Google^TMOperating system, home entertainment equipment such as Microsoft XBOX^TM、Sony PLAYSTATION^TMOr virtual reality hardware device such as Oculus RIFT^TM、HTC VIVE^TM、Samsung GEAR VR^TM, Etc..In many cases, one or more common services 23 can operate in system 20, and can help for Client application 24 provides public service.Service 23 may, for example, be WINDOWS^TMIt services, the user's space in Linux environment is public Service altogether, or for any other type public service framework of operating system 21.Input unit 28, which can be, to be suitable for connecing Receive any type of user's input, including for example keyboard, touch screen, microphone (such as voice input), mouse, touch pads, Trace ball, or any combination thereof.Output device 27 can be any type for being suitable for providing output to one or more users, No matter being for system 20 and may include one or more screens for example for visual output remotely-or locally, raise Sound device, printer, or any combination thereof.Memory 25 can be with any structure known in the art and framework, with by Device 21 is managed using for example with the random access storage device of runs software.Storage device 26 can be for storing number in digital form According to any magnetism, optics, machinery, memristor or electrical storage device (such as 1 those described above referring to Fig.1).Storage device 26 Example include flash memory, magnetic hard disk drives, CD-ROM and/or analog.

In some embodiments, system of the invention can be implemented such as with arbitrary number client and/or service On the distributed computing network of device.Referring now to fig. 13, block diagram is shown, is depicted for real on distributed computing network Apply at least part of exemplary architecture 30 according to the system in the embodiment of the present invention.According to embodiment, arbitrary number can be provided Mesh client 33.Each client 33 can run the software of client-side aspects for carrying out the present invention；Client can be with Including system 20 shown in such as Figure 12.It is furthermore possible to also provide arbitrary number server 32 is for handling from one or more The received request of client 33.Client 33 and server 32 can be in communication with each other via one or more electric networks 31, It can be internet, wide area network, mobile telephone network (such as CDMA or GSM cellular network), wireless network in various embodiments Network (such as WiFi, Wimax, LTE etc.) or local area network (or any network topology actually known in the art；The present invention is simultaneously Any one preferred network topology not on any other).Any known network protocol, which can be used, in network 31 implements, including Such as wiredly and/or wirelessly agreement.

In addition, in some embodiments, when needing to obtain additional information or being related to the extra data about specific call, Server 32 can call external service 37.Communication with external service 37 can for example be sent out via one or more networks 31 It is raw.In various embodiments, external service 37 may include that network that is related to hardware device or being itself mounted at thereon is enabled Services or functionalities.For example, implement in the embodiment on smart phone or other electronic devices in wherein client application 24, Client application 24 can external service 37 in cloud or on the one or more house properties for being deployed in specific enterprise or user It is upper to obtain the information being stored in server system 32.

In some embodiments of the invention, client 33 or server 32 (or both) can use can be with local disposition Or the one or more service-specifics or application remotely disposed across one or more networks 31.For example, can be by of the invention One or more embodiments use or quote one or more databases 34.It should be appreciated by those skilled in the art that database 34 It can be arranged with extensive various frameworks and use extensive various data access and manipulating mechanism.For example, in various embodiments one A or multiple databases 34 may include using the relational database system of structured query language (SQL), and others can be with It is referred to as those of " NoSQL " (such as Hadoop Cassandra, Google including alternate data memory technology such as this field BigTable etc.).In some embodiments, mutation database schema such as column directional data can be used according to the present invention Data in library, memory, Cluster Database, distributed data base or even flat data repository.Those skilled in the art answer This knows, known or Future Data library technology any combination suitably such as can be used, unless for particular implementation in this The special arrangement of the specified particular database technology of example or component.In addition, should know, term " database " as used herein can To be related to physical database machine, the clusters of machines as single database system, or patrolling in total Database management system Collect database.Unless providing particular meaning using term " database " for given, it should be construed as to imply that these words Any meaning, it is all these to be appreciated by those skilled in the art as the obvious meaning of term " database ".

Similarly, most of embodiments of the invention can use one or more security systems 36 and configuration system 35. Safety and configuration management are public information technology (IT) and network function, and each some amount usually with any IT or network System is associated.It should be appreciated by those skilled in the art that this field now or future known to any configuration or secure subsystem It can be unrestrictedly used in combination with the embodiment of the present invention, unless distinguishingly seeking special peace by the explanation of any specific embodiment Complete 35 or configuration system 35 or scheme.

Figure 14 shows the exemplary schematic of the computer system 40 of any each position as can be used for distributed throughout system. It is that can execute code to handle the example of any computer of data.Computer system 40 can be made various modifications and Change and without departing from the wider range of disclosed system and method herein.Central processor unit (CPU) 41 is connected to bus 42, memory 43, nonvolatile memory 44, display 47, input/output (I/O) unit 48 and network interface card (NIC) 53 are also connected to bus.I/O unit 48 can be generally attached to keyboard 49, pointing device 50, hard disk 52 and real-time clock 51. NIC 53 is connected to network 54, can be internet or local area network, and wherein local area network can have or not have to internet Connection.The a part of power supply unit 45 as system 40 is also shown, is connected to main alternation (AC) power supply in this example 46.The battery that may exist is not shown, be also not shown widely known but is not suitable for disclosed current system and method herein Specific innovative function many other devices and modification.Some or all of components shown in should knowing can to combine, such as In various integrated applications, such as system on chip (SOC) device of Qualcomm or Samsung, as long as its can fit For multiple performances or function being incorporated into single hardware device (for example, being swum in the mobile device of such as smart phone, video Play console, the interior computer system in automobile such as navigate or multimedia system or other integrated hardware devices in).

In various embodiments, the function of system or method for carrying out the present invention can be distributed in arbitrary number client Among end and/or server component.Such as, it is possible to implement various software modules are used to perform various functions in conjunction with the present invention, and And these modules various can be implemented to operate in server and/or client components.

The possibility for knowing above-mentioned various embodiments is modified range by those skilled in the art.Therefore, the present invention is wanted by right It asks and its equivalent form is limited.

Claims (7)

1. a kind of system for using multiple electronic controls and follow-up mechanism to be used for mixed reality interaction, comprising:

Electronic control unit including at least memory, processor and is stored on the memory and on the processor Multiple programming instructions of operation, programming instruction configuration to:

Via network and multiple electronic device communications；

Multiple hardware controls elements are operated, the hardware controls element configuration is used to receive man-machine interactively from user；

At least part of multiple interaction datas to the multiple electronic device is sent, the interaction data is based at least partially on At least part of man-machine interactively from the user；

The traffic is received from least part of the multiple electronic device, the traffic includes at least mixed reality data, Wherein the mixed reality data include at least multiple sensor data readings；And

At least part of the traffic is transferred to at least part of the multiple electronic device.

2. system according to claim 1, wherein the multiple electronic device includes at least virtual reality device.

3. system according to claim 2, wherein the multiple electronic device includes at least body-building device.

4. system according to claim 3, wherein the body-building device includes at least body-building follow-up mechanism.

5. system according to claim 4, wherein the body-building tracing system provides at least multiple body-building information at least The virtual reality device.

6. system according to claim 1, wherein the multiple electronic device includes at least multiple tethers, each described Tether include have one end far from human user body fixed position be attached to attachment point and the other end be attached to it is close The rope yarn of the attachment point of human user body, the nearly attachment point be configured to dress or be in addition attached to user the person or Clothing, and at least part of the remote attachment point includes the sensing for being configured at least measure the strain of attached tether Device.

7. a kind of method for communication nested between multiple electronic control units and electronic system, comprising steps of

Via the first electronic control unit, via network connection to host electronic, first electronic control unit includes At least multiple programming instructions on the processor of the computer installation of network connection are stored in memory and operated in, and configure use In via network and multiple electronic device communications, being configured to operate multiple hardware controls elements, the hardware controls element is matched It sets for receiving multiple man-machine interactivelies from human user, is configured to send multiple interactive information to the multiple electronic device At least partially, the interactive information is based at least partially at least one of the multiple man-machine interactively from human user Point, and be configured to receive the traffic from least part of the multiple electronic device and by least the one of the traffic It is partly delivered at least part of the multiple communication device；

It is connected to first electronic control unit using additional electronic control unit, the additional electronic control unit is extremely It is few to include storage in memory and operate in multiple programming instructions on the processor of networked computing device, be configured to via Network and multiple electronic device communications, are configured to operate multiple hardware controls elements, the hardware controls element configuration is used for Multiple man-machine interactivelies are received from human user, are configured to send multiple interactive information at least the one of the multiple electronic device Part, the interactive information are based at least partially at least part of the multiple man-machine interactively from human user, with And it is configured to receive the traffic from least part of most multiple electronic devices and by least part of the traffic It is transferred at least part of the multiple communication device；

Multiple user's interactions are received at additional electronic control unit；

Multiple interactive information are sent to first electronic control unit, the interactive information is based at least partially on the user Interactive at least part；And

At first electronic control unit, at least part of the interactive information is transmitted to the host electronic.