CN102023700B - Three-dimensional man-machine interaction system - Google Patents

Three-dimensional man-machine interaction system Download PDF

Info

Publication number
CN102023700B
CN102023700B CN2009100938891A CN200910093889A CN102023700B CN 102023700 B CN102023700 B CN 102023700B CN 2009100938891 A CN2009100938891 A CN 2009100938891A CN 200910093889 A CN200910093889 A CN 200910093889A CN 102023700 B CN102023700 B CN 102023700B
Authority
CN
China
Prior art keywords
data
coordinate system
dimensional
acceleration
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN2009100938891A
Other languages
Chinese (zh)
Other versions
CN102023700A (en
Inventor
吴健康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN2009100938891A priority Critical patent/CN102023700B/en
Publication of CN102023700A publication Critical patent/CN102023700A/en
Application granted granted Critical
Publication of CN102023700B publication Critical patent/CN102023700B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

The invention discloses a three-dimensional man-machine interactive device. The device acquires and outputs sensing data of the three-dimensional position and motion state of the three-dimensional man-machine interactive device from a sensor coordinate system; and a three-dimensional interactive data processing and fusing device receives sensing data, performs motion state parameter estimation in a user coordinate system, converts a motion state parameter into an interactive parameter in an application coordinate system, and further performs real-time three-dimensional interaction with an application and control object through an application interface. The three-dimensional man-machine interactive device exchanges data and control instructions with the application and control object through the three-dimensional interactive data processing and fusing device so as to realize system management, system initiation, and resetting of initial values of a direction, a speed, a position and a relationship between a user and the application coordinate system.

Description

A kind of three-dimensional man-machine interactive system
Technical field
The invention belongs to the motion perception and obtain technical field and human-computer interaction technique field, equipment, the system and method for relate to that movable information obtains, handles, analysis and man-machine interaction being used.
Background technology
Man-machine interaction at present has two big types, and one type is mouse, and it is widely used in computer system, with keyboard, realizes the mutual of people and computing machine.Another kind of is representative with the TV remote controller, and it does not possess mouse function.One aspect of the present invention is similar 3D mouse in three dimensions, can help the user to locate easily; On the other hand, it can perception and estimated service life person move displacement, speed, acceleration, direction and the dynamics of interactive device, this has important application at aspects such as simulated training, recreation.
Three-dimension interaction device and system relate to two kinds of technology.The one, detect human-computer interaction device or hold the motion of the body of this device, the 2nd, detected movable information through handling, analyze and conversion, is reached the needs of interactive application.At present, the technology of these two aspects can be summarized as follows:
Chinese patent 200710169770.9 " multifunctional golf push pole " utilizes two groups of photoelectric sending and receiving sensors to receive respectively from the mistiming of the next laser beam of ground return, calculates the average velocity of square stance, can judge that can ball enter the hole foundation is provided; Also be provided with three dimension acceleration sensor and gather the acceleration on X, Y, three directions of Z, and hit a ball to from push rod and to leave the situation of change of acceleration the whole process of ball, calculate stressed on X, Y, three directions of Z at last.United States Patent (USP) 7; 234,351, " Electronic measurement of the motion of a moving bodyof sports equipment " also is that degree of will speed up sensor and gyroscope embed sports apparatus such as golf clubs, tennis racket; Measure movable information, be used for training.Similarly, United States Patent (USP) is measured the three-dimensional position and the azimuth information of moving object with miniature acceleration sensor, gyroscope and magnetometer.Yet, measure moving object at three-dimensional movable information though above patent has been spoken of use miniature acceleration sensor, gyroscope and magnetometer.They do not mention; The acceleration that acceleration transducer is measured not is only to be the acceleration of moving object; Wherein gravitational acceleration component is very big, can not acceleration of gravity be separated from institute's measuring acceleration, and stressed on three directions of sensor coordinate system do not have much meanings; And from acceleration information will be through twice integration to the position, these two integration constants have become unknown number.Equally, gyroscope survey be angular velocity, to angle also will be through an integration.Moreover the microsensor data noise is very big, can't directly use at all.That is to say there be not the processing and the analytical approach of further sensing data, the invention in these patents can't realize.
Aspect interactive application; Chinese patent 200610127869.8; " based on the detection method of the intelligent interaction device and the operating position of action of moving body "; Method for testing motion is worn necklace based on movable body, through the crooked deformation that the angular transducer that is installed in the necklace produces, judges the movement position of movable body; Chinese patent 200610083199.4 " based on the Gamecontrol system and the method for stereoscopic vision "; Use the video camera of demarcating to obtain the image of user's region of interest; Carrying out the zone with posture feature space time information and correlativity thereof selects and feature extraction; Obtain three-dimensional position and attitude information and be mapped as the game control order,, realize the Real Time Control Function of simulation personage and scene walkthrough by system and game control interface.United States Patent (USP) 7; 262,760, " 3D pointing devices withorientation compensation and improved usability "; It is a kind of method; It is the motion and the position data of the interactive device that records, the coordinate from the coordinate conversion of interactive device to interactive object, thus be mutual mouse input data.In fact, from measuring exercise data to depending on application alternately to a great extent.Exchange method in above-mentioned three patents also has nothing in common with each other.
Along with the application of various 3-D technologies, particularly train, play, make, the development of medical treatment, virtual reality, mixed reality etc., three-dimension interaction has been proposed increasingly high requirement.People hope to have a kind of man-machine interaction of three-dimensional.One of which, it should be able to obtain the three-dimensional position and the positional information of interactive device exactly; Its two, for train, the needs of recreation etc., we require is not only positional information, also requires three-dimensional motion (speed, acceleration, track) and dynamics information.Its three, this human-computer interaction device should not receive distance limit, it can closely use with computing machine like common mouse, also can be like TV remote controller, even farther distance is mutual with institute control machine.Its four, we require a kind of interactive device can more than enough usefulness, satisfy the needs of training, recreation, manufacturing, medical treatment, virtual reality, mixed reality etc.The purpose of Here it is three-dimensional human-computer interaction device of the present invention and system.
Summary of the invention
For the nonlinear problem and the magnetic field of the non-linear and microsensor measurement that solves human-computer interaction device itself motion that prior art exists receives the surrounding environment influence problem, the purpose of this invention is to provide a kind of three-dimensional man-machine interactive system.
For reaching said purpose, the present invention provides three-dimensional man-machine interactive system, comprising: this system the whole sensing data that is defined as alternately under three kinds of coordinate systems obtain, handle, fusion and transfer process, wherein:
Three-dimensional human-computer interaction device obtains and exports the multiple sensing data of the three-dimensional motion state of three-dimensional human-computer interaction device in a plurality of microsensor coordinate systems of self;
Three-dimensional human-machine interaction data processing is connected with three-dimensional human-computer interaction device with fusing device; Receive many sensing datas that three-dimensional human-computer interaction device is gathered; And there are many sensing datas of noise and dynamic change to merge to a plurality of microsensor institute measuring tool; Obtain motion state and the location parameter of three-dimensional human-computer interaction device in user coordinate system, i.e. the accurate valuation of position, speed, acceleration, orientation, angular velocity, derivation movement locus, acting force and direction; And convert the interaction parameter of using in the coordinate system to;
Use with controlling object and be connected with fusing device with three-dimensional human-machine interaction data processing through application interface; The interaction parameter that receives the accurate valuation of motion state and location parameter and release thus; Be used for control and use and controlling object, handle and fusing device transmission command information to three-dimensional human-machine interaction data through application interface.
Preferably, three-dimensional human-computer interaction device comprises: a plurality of microsensors, central controlled processing unit, wireless communication chips and one group of function key, wherein:
A plurality of microsensors extract the three-dimensional position and the exercise data of three-dimensional human-computer interaction device itself, and through output three-dimensional position and motion simulation data after the conversion processing and amplifying;
Each road analog to digital converter of central authorities' controlled processing unit or numeral inlet are connected respectively to all the sensors data terminal in a plurality of microsensors; Central authorities' controlled processing unit receives by the sampling rate of setting and the three-dimensional position of a plurality of microsensor outputs and the simulated data of motion is controlled and handled; The sampled data that obtains a plurality of microsensors is deposited in the register of central controlled processing unit, and these sampled datas is sticked the instruction of sequential label and function key pack together and export sampled data and instruct with function key; Central authorities' controlled processing unit receives, explains and carries out from three-dimensional human-machine interaction data and handle and fusing device place computer commands;
Wireless communication chips is connected with central controlled processing unit, receives and export the function key instruction and the sampled data of central controlled processing unit; Wireless communication chips is set up three-dimensional human-computer interaction device and is belonged to communicating by letter of computing machine or computing equipment, the data between completion computing machine or the computing equipment and the transmission of order with three-dimensional human-machine interaction data processing and fusing device;
Function key is connected with central controlled processing unit, and the information of function key is sent to three-dimensional human-machine interaction data by central controlled processing unit with it packing and handles and fusing device through interrupting.
Preferably, said microsensor is multiple microsensor, and multiple microsensor is at least one or more than one three dimension acceleration sensor, three-dimensional gyroscope, three-dimensional magnetometer, ultrasonic range finder, laser range finder.
Preferably, said function key is initialization key or reset button, options button and function display key.
Preferably; Three-dimensional human-machine interaction data is handled and fusing device comprises that data preprocessing module, DOA estimation unit, speed and Displacement Estimation module, movement locus and dynamics analysis module, interaction parameter produce and function key administration module and application interface, wherein:
Data preprocessing module is connected with three-dimensional human-computer interaction device respectively; Data preprocessing module receives the microsensor of three-dimensional human-computer interaction device respectively and measures acceleration, angular velocity and magnetic-field measurement data in this microsensor coordinate system; Handle the data stream of acceleration, angular velocity and magnetic-field measurement in generation and the output transducer coordinate system through denoising and integrity detection;
The DOA estimation unit is connected with data preprocessing module; The DOA estimation unit receives and the data stream of the acceleration in the sensor coordinate system, angular velocity and magnetic-field measurement is handled, and produces and the orientation valuation of output under changes of magnetic field and the big situation of the acceleration of human-computer interaction device own;
Speed is connected with the acceleration estimation module of DOA estimation unit with the Displacement Estimation module, and the speed displacement estimation module is used the acceleration valuation from the human-computer interaction device of acceleration estimation module itself, obtains speed and displacement data through integration;
Movement locus is connected with the Displacement Estimation module with speed with the DOA estimation module with the dynamics analysis module; Accept orientation valuation, speed and displacement data that orientation estimation module and speed and Displacement Estimation module are come; Form human-computer interaction device each state constantly in user coordinate system; Comprise orientation, angular velocity, acceleration, speed, displacement and position, and and then release movement locus, make force direction and dynamics;
Interaction parameter produces and is connected with the dynamics analysis module with movement locus with the function key administration module; Interaction parameter generation and function management module are based on user coordinate system and the relation of using coordinate system; The position under user coordinate system, movement locus, make force direction and dynamics parameter; Under the control of function key; On request to azimuth, angular speed, acceleration, speed, displacement, the movement locus of human-computer interaction device, make force direction and dynamics is further calculated and processed, interaction parameter is used in output;
Application interface produces with interaction parameter respectively and is connected with controlling object with application with the function key administration module, and the application interaction parameter that draws in application interface reception and the forwarding application coordinate system is given and used and controlling object.
Preferably, said DOA estimation unit is by DOA estimation module, magnetic field estimation module, acceleration estimation module and put the initial value module and form, and is used for reducing skew, improves estimation accuracy, wherein:
The input end of DOA estimation module is connected with data preprocessing module, magnetic field estimation module, acceleration estimation module, the output terminal of putting the initial value module respectively; Acceptance angle speed data stream, receive the magnetic field estimation module field signal, receive acceleration estimation module acceleration of gravity signal, receive put the initial value module the initial orientation signal; The DOA estimation module is used Bayes's estimation method; With the initial orientation value of putting the initial value module is starting point; With gyrostatic angular velocity measurement data is main, merges the restriction to the position angle valuation of magnetic field and acceleration of gravity simultaneously, thereby realizes the low deviation valuation in orientation; Used non-linear valuation wave filter to reduce the non-linear issuable error of the non-linear and sensor measurement of motion; The DOA estimation module is respectively to movement locus and dynamics analysis module, magnetic field estimation module and acceleration estimation module output orientation valuation;
The magnetic field estimation module is connected with the DOA estimation module with data preprocessing module respectively; Receive the magnetic-field measurement data stream of data preprocessing module output, the orientation valuation of reception orientation estimation module, use non-linear estimation method to realize valuation is carried out in the magnetic field that changes with position and time; The result of magnetic field valuation is sent to the DOA estimation module and participates in next orientation valuation constantly;
The acceleration estimation module is connected with the DOA estimation module with data preprocessing module respectively; Receive the bearing signal of acceleration information stream and the output of DOA estimation module; The acceleration estimation module uses non-linear valuation wave filter to realize the valuation of this acceleration under sensor coordinate system of human-computer interaction device; From the acceleration analysis data, isolate gravitational acceleration component with the valuation result, be sent in the DOA estimation module, next valuation constantly of participating in the orientation is merged; Simultaneously, the orientation valuation of using the orientation estimation module to provide is transformed into the acceleration valuation in the sensor coordinate system in the user coordinate system, is sent to speed and Displacement Estimation module;
Put initial value module and DOA estimation module, speed and Displacement Estimation module and produce with interaction parameter and is connected, put system initialization submodule and coordinate system replacement submodule in the initial value module and through DOA estimation module, speed and Displacement Estimation module and interaction parameter generation and function key administration module the coordinate system of orientation, speed, position and user and application is concerned and put initial value with the function key administration module; Replacement initial value when receiving control message from keyboard and instruction interpretation signal under simple standard scene.
Preferably, application and controlling object have through the command information that application interface mails to three-dimensional human-machine interaction data processing and fusing device: Apply Names, the requirement of data sampling rate, help information, initialization value; Send data sampling rate control command to human-computer interaction device, regularly send the power values of the PMU that reads human-computer interaction device simultaneously, the working status parameter of decision-making system.
Preferably, the acceleration of gravity direction that records in the principal direction that system initialization submodule is used the measured magnetic field of magnetometer and the acceleration transducer is put the acquiescence initialization for sensor coordinate system and user coordinate system concern; Use initial scene to put the acquiescence initial value as user coordinate system and application coordinate system relation; According to the difference of initial scene, system initialization submodule has different three-dimensional human-computer interaction device initial positions and different orientation, speed and position initial values.
Preferably; Said three-dimensional human-computer interaction device be conveniently hold, wear, pin is tied or wear the shape of mode attached to certain part of people's health, this shape can be hand rectangular-shaped, lip pencil, commander bar-shaped, hold in hand spherical, implant thin slice in the cap, be anchored on the thin slice on pin, shank or the arm.
A kind of Three-Dimensional Dynamic positioning equipment of the present invention is used to train, plays, makes, outside the various three-dimensional applications of medical treatment, virtual reality, mixed reality, also can be applied in location, navigation and the control of machine.
Beneficial effect of the present invention: system of the present invention the whole sensing data that is defined as alternately under three coordinate systems obtain, handle, fusion and transfer process.These three coordinate systems are sensor coordinate system, user coordinate system and application coordinate system.They correspond respectively to human-computer interaction device, user and application.Three-dimensional man-machine interactive system is obtained the exercise data of three-dimensional human-computer interaction device in sensor coordinate system, in user coordinate system, estimate its motion state parameters, and further converts them to use in the coordinate system interaction parameter.The present invention allows that one or more users are arranged in the same application, and same user uses one or more human-computer interaction devices.
Different with prior art; Of the present inventionly considered microsensor; Particularly the characteristics of big, the dynamic change of the noise of minisize gyroscopes, micro accelerometer and miniature magnetic meter and their institute's angle measurement speed, acceleration and magnetic field data have been invented the high precision estimation method that can merge the kinematic parameter of multiple sensors data effectively.Estimated position, speed, acceleration, orientation, angular velocity, and derivation movement locus, acting force and direction etc. in real time, the mutual needs of various three-dimensional applications such as satisfied training, recreation, manufacturing, medical treatment, virtual reality, mixed reality.
The present invention have function key, system initialization and replacement, with use functions such as mutual and system management, the three-dimension interaction that therefore can realize ideal with different three-dimensional applications.
What will emphatically point out especially is that the estimation accuracy of motion and location parameter is the bottleneck of restriction plurality of applications always.The present invention satisfies on the application requirements improving motion and location parameter estimation accuracy, has taked the series of new techniques means.This comprises: for the nonlinear problem that the non-linear and microsensor that solves human-computer interaction device motion itself is measured, the present invention uses UKF (the Unscented Kalman Filter) wave filter that can successfully manage 3 nonlinear problems in valuation; Receive surrounding environment influence in reply magnetic field and during variation issue, we at first estimate magnetic field, are being sent to the magnetic field estimated value data fusion DOA estimation; During gravitational acceleration component in solving the accelerometer measures data, we at first estimate the acceleration of human-computer interaction device itself, gravitational acceleration component are separated again; The more important thing is that we have invented a whole set of multisensor (gyroscope, accelerometer and magnetometer) data fusion and have carried out the method for high precision orientation, speed and location estimate.
The present invention can realize hoping for these years with machine alternately as interpersonal the same hope easily.Actions such as the present invention can also hold, pin is held, wear, wear, make man-machine interaction in three dimensions to more wide application development.Simultaneously, as an energy measurement and the device of estimating three-dimensional motion and location parameter, the present invention also can be used in the navigation of various moving objects, the field such as mutual of virtual world and physical world.
Description of drawings
Fig. 1 is the structural drawing of the three-dimensional human-computer interaction device of the present invention and the overall logic block diagram of system;
Fig. 2 is position and the orientation synoptic diagram of the three-dimensional human-computer interaction device of the present invention in user coordinate system;
Fig. 3 is the structure and the system flowchart of the three-dimensional human-computer interaction device of one embodiment of the invention;
Fig. 4 is the three-dimensional human-machine interaction data processing of one embodiment of the invention and the process flow diagram of fusing device.
Embodiment
Specify each related detailed problem in the technical scheme of the present invention below in conjunction with accompanying drawing.Be to be noted that described embodiment only is intended to be convenient to understanding of the present invention, and it is not played any qualification effect.
In order to help to understand technology contents of the present invention, we narrate the application scenarios of two application " three-dimensional man-machine interactive system " of the present invention earlier.One of scene is, on the computing machine giant-screen, shown the medical science tomography of a human body brain and the three-dimensional reconstruction figure of nuclear magnetic resonance, and several doctors are discussing medical scheme.All hold one " three-dimensional human-computer interaction device 100 " of the present invention in each physician's hand, they can locate in three-dimensional mind map easily, indication, and can do certain operation at assigned address.This is that the remote control 3D mouse of expanding adds function key.Certainly, because location, indication and the function selecting etc. of three-dimensional human-computer interaction device 100 of the present invention in three dimensions, some interactive graphics (IG)s that it also can be used for other are drawn and editor etc.Here, we claim that the device that the doctor hands is " three-dimensional human-computer interaction device 100 ", claims that the doctor is " user "; Reconstruction, analysis and the operating system of Computerized three-dimensional human brain 3-D view are called " application "; In using residing computing machine, receive, handle, merge sensing data, derive the subsystem of using required interaction parameter, then be called " three-dimensional human-machine interaction data is handled and fusing device 200 ".Usually, each user has one or more three-dimensional human-computer interaction devices 100, and each application can have one or more users.
Again for example, in training of three-dimension virtual reality and mixed reality and the games system, can realize two Human To Humans, or the people is to visual human's training or recreation, as plays ball.Wherein two people can also can be separated by thousands of miles together.They do not need actual court, are object with the display screen.At this moment, " three-dimensional human-computer interaction device 100 " of the present invention is virtual paddle.Speed, acceleration, track, angle, the angular velocity of player (i.e. " user ") when brandishing racket (interactive device), wait can by " three-dimensional human-machine interaction data handle with fusing device 200 " from three-dimension interaction human-machine device 100 in the data of sensor derivation come out.These batting parameters just can be used for further deriving ball by the motion after hitting.This virtual match is carried out as in the reality, and victory or defeat has also just had.The record of the shot of ball of deserving the beating and the ball of not accomplishing fluently can access, and analyzes reason, finds out the place of improvement.Here, " application " be realization, demonstration and the management of virtual ball match.
Like Fig. 1 the structural drawing of the three-dimensional human-computer interaction device of the present invention and the overall logic block diagram of system are shown; The present invention is handled by three-dimensional human-computer interaction device 100 and three-dimensional human-machine interaction data and fusing device 200 is formed.Three-dimensional human-computer interaction device 100 obtains the three-dimensional position and the exercise data of this device itself, and user's instruction, issues three-dimensional human-machine interaction data and handles and fusing device 200.Three-dimensional human-machine interaction data is handled and fusing device 200 is handled and the three-dimensional position and the exercise data of analyzing three-dimensional human-computer interaction device 100; Estimate three-dimensional human-computer interaction device 100 three-dimensional position in three dimensions and orientation at any one time, totally 6 parameters; And and then based on application requirements, use this 6 parameters and time series thereof, derive the control parameter in the application space.
Three-dimensional man-machine interactive system comprises: this system the whole sensing data that is defined as alternately under three coordinate systems obtain, handle, fusion and transfer process, wherein:
Three-dimensional human-computer interaction device 100 obtains and exports the sensing data of the three-dimensional motion state of three-dimensional human-computer interaction device 100 in the sensor coordinate system of self; Three-dimensional human-machine interaction data is handled and fusing device 200 receives sensing data; And in user coordinate system, use sensing data to carry out the valuation of motion state and location parameter, and further convert motion state parameters to use in the coordinate system interaction parameter;
Three-dimensional human-machine interaction data processing and fusing device pass sensor institute measuring tool more 200 pairs have many sensing datas of noise and dynamic change (like angular velocity, acceleration and magnetic field sensing data) to merge; Realize the accurate valuation of motion state parameters; Estimated position, speed, acceleration, orientation, angular velocity, and derivation movement locus, acting force and direction in real time;
Three-dimensional human-machine interaction data is handled and is connected with controlling object 300 with application with fusing device 200 and then through application interface 260, and three-dimensional human-computer interaction device 100 is handled through three-dimensional human-machine interaction data and fusing device 200, application interface 260 and application and controlling object 300 realization system managements; Put initial value module 224 and put initial value according to the coordinate system relation of using through DOA estimation module, speed and Displacement Estimation module and interaction parameter generation and function key administration module 250 couples of orientation, speed, position and users and application.
1, sensor, user and application coordinate system
In two application examples in above-mentioned 3 d medical images application and virtual court, we have set three fundamentals of three-dimension interaction system: three-dimensional human-computer interaction device, user and application.An application can have one or more users, and each user also can use one or more three-dimensional human-computer interaction devices.Three-dimensional human-computer interaction device, user and application also have separately space or coordinate system, and they are respectively: sensor coordinate system, user coordinate system and application coordinate system.
Fig. 2 shows position and the orientation of the three-dimensional human-computer interaction device of the present invention with respect to user coordinate system.Because a certain human-computer interaction device 100 is used by a certain user, and each user has his coordinate system, claim " user coordinate system ".Different users have different coordinate systems, and user coordinate system is different from global coordinate system.In interactive application, application also has the coordinate system of oneself, and we are referred to as " application coordinate system ".The coordinate system of the human brain three-dimensional reconstruction image that doctors studied in the example of front for example, the coordinate systems in virtual court etc. all are the examples of " application coordinate system ", they are not shown in Fig. 2.Because for each different application, all just like where managing user coordinate system and the specific (special) requirements of using coordinate system.When a user brings into use a human-computer interaction device 100, and select or added after certain uses, just need to set " user coordinate system " (x, y, z) with use coordinate system and their relation.
All microsensors 110 in the human-computer interaction device 100 all are calibrated to a unified coordinate system, and we are referred to as " sensor coordinate system " (x ', y ', z ').The sensing data of all acquisitions is all in sensor coordinate system.This sensor coordinates ties up to coordinate points x in the user coordinate system, y, and the position of z is exactly the position x=(x of our human-computer interaction device 100 that requires; Y, z), and sensor coordinate system is with respect to the azimuth angle alpha of user coordinate system; β; γ is exactly the orientation q=(α, beta, gamma) of our human-computer interaction device 100 that requires.When the user brought into use also initialization or replacement human-computer interaction device 100, the initial position of three-dimensional human-computer interaction device 100 was x 0=(x 0, y 0, z 0), wherein to tie up to the coordinate points of the initial position in the user coordinate system be x to sensor coordinates 0, y 0, z 0With with initial orientation be q 0=(α 0, β 0, γ 0), wherein to tie up to the position angle of the initial orientation in the user coordinate system be α to sensor coordinates 0, β 0, γ 0
Setting three coordinate systems of three-dimensional man-machine interactive system; Be after sensor coordinate system, user coordinate system and the application coordinate system; The core of the invention of narration is below, in sensor coordinate system, obtains three-dimensional motion and position data, in user coordinate system, merges multi-sensor data; Estimate the motion and the location parameter of each three-dimensional human-computer interaction device, in using coordinate system, produce at last this is used significant interaction parameter or notion.Specifically; Three-dimensional human-computer interaction device 100 obtains the motion of itself and the sensing data of position in sensor coordinate system, DOA estimation unit 220, speed and Displacement Estimation module 230 and movement locus and dynamics analysis module 240 use sensing data to estimate orientation, position, motion and the dynamics parameter of three-dimensional human-computer interaction device 100 in user coordinate system.Because sensor coordinate system is to define with the sensor coordinates in the human-computer interaction device, orientation, position, motion and the dynamics parameter of three-dimensional human-computer interaction device 100 in user coordinate system is actually the Relation Parameters of sensor coordinate system and user coordinate system.Interaction parameter produces and function key administration module 250 and 260 parameters of using in the user coordinate system of application interface, and the relation of user coordinate system and application coordinate system, according to application requirements, produces corresponding interaction parameter or notion.Here, existing from the conversion of user coordinate system to the application coordinate system, the cluster of self moving parameter to movement concept also arranged.By moving sign language promptly is one of movement concept cluster example.
Between sensor coordinate system and the user coordinate system, user coordinate system and use between the coordinate system all be N:1 relation (N in this expression more than or equal to 1 integer.A plurality of in other words user coordinate systems are used coordinate system corresponding to 1).In the narration below and in the synoptic diagram, we are that example is explained with the simplest situation of 1:1 only.The situation of N:1 only is the simple popularization of the following 1:1 situation that is described in detail, and has no the difficulty of technology.
The relation of sensor coordinate system and user coordinate system, and the initial value of the relation of user coordinate system and application coordinate system is all set by " initialization/put initial value module 224 ".The present invention will provide two kinds of approach to obtain initial value, a kind of initialization that is to use default value, and another kind is the artificial replacement of setting.Detail will narrated initialization/detailed description when putting initial value module 224.
Still be applied as example with 3 d medical images.3 doctors respectively hold three-dimensional human-computer interaction device, have started this application.Here, for simplicity, choosing acquiescence sensor coordinates ties up to the calibration in the user coordinate system during system start-up, by the center of doctors with three-dimensional human-computer interaction device sensing display, is user coordinate system and the calibration of application coordinate system.After this, doctor's moving three dimension human-computer interaction device has been selected the position of a doubtful tumour, and has selected amplification with function key, presents to two doctors in addition to this parts of images.The another one doctor then further selects painted and rotation, does further research and discussion.Simultaneously, they also with three-dimensional human-computer interaction device as scalpel, operation is simulated.
After having accomplished discussion, be the quitting time.They select with the three-dimensional human-computer interaction device in their hand is club, has carried out a golf range game.
2, three-dimensional human-computer interaction device 100
Three-dimensional human-computer interaction device can be an arbitrary shape, and modes such as conveniently holding, wear, pin is tied or wear is attached to certain part of people's health.For example, it can or command bar-shapedly like hand-held TV remote controller shape, pen, holds spherical in hand, implant in the cap and be anchored on thin slice on pin, shank or the arm etc.
Say that from the angle of infotech three-dimensional human-computer interaction device 100 is micro embedded systems.It comprises: a plurality of microsensor 110, central controlled processing unit 120, wireless communication chips 130 and one group of function key 140.
In the three-dimensional human-computer interaction device 100:
Microsensor 110 can comprise part or all of following sensor: three-dimensional accelerometer, three-dimensional gyroscope, three-dimensional magnetometer, ultrasonic range finder, laser range finder etc.Said three-dimensional acceleration can select for use miniature three-dimensional accelerometer or other models then to repeat no more; Said ultrasonic range finder selects for use miniature ultrasonic stadimeter or other models then to repeat no more, laser range finder can select for use the miniature laser stadimeter then or other models repeat no more; These microsensors 110 are measured the data relevant with the orientation with the position of this three-dimensional human-computer interaction device 100 wherein: the miniature three-dimensional accelerometer measures is along the acceleration information of its 3-D walls and floor, and acceleration of gravity also is mixed in wherein.Three-dimensional gyroscope survey is along the angular velocity of its this human-computer interaction device of 3-D walls and floor, and three-dimensional magnetometer is then measured three magnetic-field components on the coordinate direction.Miniature ultrasonic stadimeter and miniature laser stadimeter then can be used for measuring the distance comparatively accurately of this human-computer interaction device 100 and the mutual computing equipment of institute or other object of reference.
The all the sensors data all are connected respectively to each the road analog to digital converter or the numeral inlet of central controlled processing unit 120 in the microsensor 110.Central authorities' controlled processing unit 120 is that whole three-dimensional human-computer interaction device 100 is controlled and handled.Usually, there are several analog to digital converters its inside (Analog to Digital Converter ADC), converts digital signal to the simulated data of microsensor 110 is amplified to the dynamic range of analog to digital converter through prime amplifier after.The action that central authorities' controlled processing unit 120 reads the output data of analog to digital converter is exactly the sampling to microsensor 110 data.Therefore; Central authorities' controlled processing unit 120 is with certain sampling rate each microsensor 110 data of sampling respectively; Deposit in the register of central controlled processing unit 120; And with sequential label and function key instruction on these data post together, wireless communication chips 130 or wire communication passage are sent in packing.
As central controlled processing unit 120, also according to the program that embeds in advance, control and change sampling rate, reset, data are done necessary pre-service as the time when adopting microprocessor or controller, or the like.
Wireless communication chips 130 is set up three-dimensional human-computer interaction device 100 and is handled with fusing device 200 with three-dimensional human-machine interaction data and belong to communicating by letter of computing machines or computing equipment, accomplishes data and the transmission of order between them.Under the not very long situation of transmission range, can use existing wireless communications technology, chip and agreements such as bluetooth, Zigbee.In three-dimensional human-computer interaction device 100, wireless communication chips 130 is connected with central controlled processing unit 120, receives the instruction and data of central controlled processing unit 120, data is sent to three-dimensional human-machine interaction data handles and fusing device 200; Through wireless communication chips 130, central controlled processing unit 120 receives, explains and carries out from three-dimensional human-machine interaction data and handle and fusing device 200 place computer commands (like switch application, changing sampling rate etc.).Handle and fusing device 200 place computer terminals at three-dimensional human-machine interaction data; Three-dimensional human-machine interaction data is handled with after fusing device 200 place computing machines and wireless communication chips 130 are communicated by letter by communication protocol foundation; Receive the data and the status information of three-dimensional human-computer interaction device 100, send instruction to three-dimensional human-computer interaction device 100.In some cases, can be without radio communication, and adopt the wire communication mode, handle with fusing device 200 and belong to computing machine or computing equipments as the data line of three-dimensional human-computer interaction device 100 is inserted three-dimensional human-machine interaction data from the USB mouth.
Function key 140 can comprise initialization or reset button, options button (like the left button of mouse), function display key (like the right button of mouse), etc.In three-dimensional human-computer interaction device 100, function key 140 is connected with central controlled processing unit 120, and function key 140 information are sent to three-dimensional human-machine interaction data by central controlled processing unit 120 with it packing and handle and fusing device 200 through interrupting.
3, three-dimensional human-machine interaction data is handled and fusing device 200
As shown in Figure 4, three-dimensional human-machine interaction data is handled with fusing device 200 generally with the form of software with application, is present in appliance computer or other application apparatus.For example, be present in jointly in the toy computer or in the game machine, be present in the computer system jointly with 3 d medical images demonstration and surgery simulation system with video-game.Certainly, three-dimensional human-machine interaction data is handled and fusing device 200 also can be present in the hardware of three-dimensional human-computer interaction device with three-dimensional human-computer interaction device 100 jointly, even solidifies in chip.
Three-dimensional human-machine interaction data is handled and fusing device 200 comprises the part or all of of following module: data preprocessing module 210, DOA estimation unit 220, speed and Displacement Estimation module 230, movement locus and dynamics analysis module 240, interaction parameter generation and function key administration module 250, application interface 260.Movement locus and dynamics analysis module 240 are modules that the movement locus and the dynamics of human-computer interaction device are analyzed; Interaction parameter produces and function key administration module 250 is the generation of mutual mouse parameter and the administration module of function key; Application interface 260 is the interfaces to system management, control and application;
210 pairs of data of gathering and sending from human-computer interaction device 100 of data preprocessing module are done pre-service.Pre-service comprises: check the sequential label of data, the deal with data loss situation is carried out denoising filtering etc.If loss of data is arranged, the data of taking the method for interpolation to come polishing to lose usually.
DOA estimation unit 220 comprises the DOA estimation module 221 of estimating the orientation with the integration of the measured angular velocity of gyroscope; Export the magnetic field estimation module 222 of estimating magnetic field and being used to correct the orientation with magnetometer; Come estimated acceleration and correct the acceleration estimation module 223 in orientation with the acceleration transducer data with acceleration of gravity, and the replacement initial value put initial value module 224.In DOA estimation module 221, the angular velocity that uses three-dimensional gyroscope to record carries out integration, tries to achieve three position angles (α, beta, gamma) of human-computer interaction device 100 with respect to initial orientation.
In order to improve precision, adopt following method:
A) because three-dimensional gyroscope records is the angular velocity measurement value, this measured value is the stack of actual value and noise, and the variation of angular velocity can be a linearity or nonlinear.In order to obtain actual angular speed, need to adopt Bayes's estimation method.Because the valuation equation is non-linear, we can adopt the Kalman filtering or the Unscented Kalman filtering of expansion.
B) three-dimensional gyroscope survey is angular velocity, and only the integration with three-dimensional gyro data comes the valuation position angle, deviation can occur.For this reason; We add the magnetic field that magnetometer records and (suppose that usually magnetic field is relatively stable or change slowly; The measured value of magnetometer is also relatively stable or change slowly) acceleration that records with accelerometer (in the value that accelerometer records, is acceleration of gravity greatly, removes after the acceleration of human-computer interaction device 100 own; The size and Orientation of acceleration of gravity is very stable); Form the common equation of motion and measure equation, implement the Dynamic estimation computing, try to achieve position angle valuation comparatively reliably.
C) in reality, magnetic field can change because of the existence of the consumer in the surrounding environment etc.But this variation is slowly.Therefore, when using magnetic field that the position angle of using gyro data to estimate is proofreaied and correct, valuation is carried out in estimation module 222 pairs of magnetic fields that slowly change in magnetic field.
D) same, acceleration estimation module 223 is also carried out valuation to human-computer interaction device 100 acceleration own, is designated as a.At any one time, the measured value of three-dimensional accelerometer should be acceleration of gravity, human-computer interaction device 100 acceleration own and measure the noise sum.Because acceleration, position angle and magnetic field all are the state variables of human-computer interaction device 100, can write out the measurement equation in Bayes's estimation method with this.The acceleration of human-computer interaction device 100 can carry out valuation with bayes method.Try to achieve after the valuation, we can obtain acceleration of gravity in human-computer interaction device 100 value under the coordinate system at that time.This value also is used for the position angle of using gyro data to estimate is proofreaied and correct simultaneously, reaches the purpose that reduces deviation.
In front, we have estimated the orientation of human-computer interaction device 100, promptly with respect to three position angles of user coordinate system, and 100 acceleration (a in user coordinate system of human-computer interaction device x, a y, a z).Like this, through integral operation, can draw its speed and displacement in user coordinate system.The three-dimensional acceleration data of human-computer interaction device 100 in user coordinate system that speed and Displacement Estimation module 230 are used from acceleration estimation module 223; And, obtain the speed (v of human-computer interaction device 100 in user coordinate system through a integration to acceleration through putting initial value module 224 acquisition speed initial value and position initial values x, v y, v z); Again to rate integrating, obtain displacement and the position of human-computer interaction device 100 in user coordinate system (x, y, z).
From speed and displacement that the three dimension acceleration sensor data estimation goes out, because the noise and the drift of miniature angular-rate sensor, measuring accuracy is not very high, but can satisfy the plurality of applications requirement.For the high application of accuracy requirement, can add supersonic sounding or laser ranging, measure the distance of human-computer interaction device 100 with respect to certain specific objective (like training and game object, screen etc.), compensate the precision of above measurement and valuation.
So far, at any time, we can estimate position, speed, acceleration, position angle, the angular velocity of human-computer interaction device 100 in user coordinate system, form thus this constantly state vector (x, y, a, q, ω).Like this, the movement locus of human-computer interaction device and dynamics analysis module 240 just can draw the human body (fist, pin etc.) of human-computer interaction device 100 or its representative or the track of apparatus (racket etc.) easily.Under the situation of given quality (or weight), can calculate momentum by operating speed, or the size and Orientation that uses acceleration to calculate power.
As use human-computer interaction device 100 to represent sports apparatus such as racket, or when human-computer interaction device 100 being embedded among the sports apparatus such as rackets, position that calculate here and motion state vector (x; Y; A, q is that sensor coordinate system is for user coordinate system ω).Like known users coordinate system and the relation of using coordinate system, as shown in Figure 2, the position angle q that is had a few along y ' axle in the sensor coordinate system (x ', y ', z ') is identical with angular velocity omega.Therefore; With the racket is example; When the distance of striking point on the ball and sensor coordinate system initial point known (as being designated as d); In user coordinate system, because known human-computer interaction device 100 coordinate origin positions, speed and acceleration, the position of striking point on the ball, speed and acceleration can directly be obtained with the method for solid geometry.Thus, in batting moment, by the motion state vector of racket striking point on the ball (x, v, a, q, ω) and the motion state vector of ball, we can obtain the motion of batting back ball, thus " truly " of accomplishing the motion of batting and ball in virtual training or the recreation simulates.
Simultaneously, we also can access the motion state vector sequence of corresponding shot according to balling effect, especially its movement locus and the process of having an effect are carried out quantitative test, can improve shot.Also can further write out the repetition training program of specific action; Balling effect is estimated in motion according to batting back ball; Again according to movement locus and sticking point and the process of having an effect of the human-computer interaction device 100 of batting process; And their relevant with balling effect, estimate the evaluation of current shot, give the suggestion of the ball-driving method that makes new advances.
Interaction parameter produces and the function of function key administration module 250 is to use the state parameter of autokinesis track and dynamics analysis module 240 to obtain the interaction parameter in the three dimensions of using coordinate system, and accomplishes the management of function key.For example, in the three-dimensional graphics images editing application, it mainly comprises the location and selects and continuous the picture.Under the situation of special requirement, also can include the character input.Like this, in the ordinary course of things, the function key among the present invention on the human-computer interaction device 100 includes only switch, initialization/replacement, selection, function demonstration.When opening human-computer interaction device 100 or using initialization or reset key, system is in original state, puts initial value for simultaneously position, speed and the position angle of human-computer interaction device 100.Since function key often with application interface, therefore be placed in this module and manage.
Because the data of three-dimensional human-computer interaction device can get its three-dimensional position and orientation after orientation estimation unit 220 and speed and Displacement Estimation module 230, as 3D mouse the time, only need the location.Therefore, we can use the three-dimensional position of acquisition to come to be the 3D mouse location, also can convert the three-dimensional position that obtains into three-dimensional coordinate and come to be the 3D mouse location.
The application interface 260 of system management, control be total system downwards with the interface of human-computer interaction device 100 and the interface with application and controlling object 300 of making progress.Application interface 260 parameters comprise self-application: Apply Names, the requirement of data sampling rate, help information, initialization value; From human-computer interaction device: function key assignments, mouse position, three-dimensional position, three-dimensional position.System management is understood application requirements with control, sends control commands such as data sampling rate to human-computer interaction device 100.Regularly read simultaneously the state parameter of human-computer interaction device 100, like power level etc., the duty of decision-making system.The application interface 260 of system management, control comprises with the interface parameters of human-computer interaction device: order, function key, three-dimension sensor data.
Further specify implementation detail with a specific embodiment at present.As previously mentioned, the present invention is not limited to this example.
Fig. 3 illustrates the structure and the system flowchart of the three-dimensional human-computer interaction device of one embodiment of the invention.Among Fig. 3, microsensor 110 adopts three kinds of microsensors promptly: three axis accelerometer 111, three-axis gyroscope 112 and three magnetometers 113.Central authorities' controlled processing unit 120 adopts microcontroller (for the ease of understanding memory; With a kind of form of central controlled processing unit 120 be microcontroller also the mark-on mark be 120), microcontroller 120 comprises: analog to digital conversion interface 121, I2C interface 122, timer 123, control chip 124, register 125, interrupt manager 126, serial port module 127.Also comprise a PMU 150, power management is human-computer interaction device 100 power supplies.Function key 140 is chosen as button 1, button 2 and button 3.Different according to product sensor model and manufacturer, the output simulating signal that has, the direct output digital signal that has.But the output simulating signal, all be amplified to the signal level that is complementary with analog to digital conversion usually.Therefore; Any sensor in the three axis accelerometer 111 of microsensor 110 among Fig. 3, three-axis gyroscope 112 and three magnetometers 113, its signal is that simulating signal is connected with I2C digital interface 122 with the analog to digital conversion interface 121 that digital signal is then received microcontroller 120.Microcontroller 120 is set the sampling rate of each microsensor 110 according to from using the order that (using and controlling object 300, as above 3 d medical images system applies in the example Fig. 4) comes, and leaves in the register 125.Microcontroller 120 and according to promptly deciding sampling rate reads three axis accelerometer 111, three-axis gyroscope 112 and three magnetometers 113 of microsensor 110 respectively from analog to digital converter 121 or I2C interface 122 sensing data; And sensing data added time tag; According to man-machine interactive system data communication protocol form, the sensing data that adds time tag is put into the dedicated cache of register 125.
Function key 140 gets into human-computer interaction device 100 with the mode of interrupt manager 126, has high priority because they are compared with data.Therefore, by next function key 140, human-computer interaction device 100 will respond at once, with man-machine interactive system command communication protocol format, puts into register 124 special-purpose buffer memorys to function key 140 through control chip 124.
Dedicated cache in the register 125 in the microcontroller 120 is used for storage according to packaged data of communication protocol and instruction, waits for that being sent to three-dimensional human-machine interaction data handles and fusing device 200.
Here; Through serial port module 127 sensing data in the register 125 is sent into wireless communication chips 130, wireless communication chips 130 realizes that by bluetooth module human-computer interaction device 100 and three-dimensional human-machine interaction data are handled and the radio communication of the principal computer at fusing device 200 places.This is because almost used computing machine all has bluetooth module and corresponding interface support.
Fig. 4 is the three-dimensional human-machine interaction data processing of one embodiment of the invention and the process flow diagram of fusing device 200.
The sensing data of human-computer interaction device 100 is at first through pre-processing module; Isolate the data of each microsensor 110; Form each microsensor 110 (three-dimensional accelerometer 111, three-dimensional gyroscope 112 and three-dimensional magnetometer 113) data stream, and check the integrality of these data.Particularly find event of data loss, find out the correct time and the position of losing, the row interpolation of going forward side by side is handled.Because the data noise of microsensor 110 is very big, also must carries out denoising to the data of each microsensor 110 respectively and handle.
As shown in Figure 4, three-dimensional human-machine interaction data is handled and fusing device 200 comprises that data preprocessing module 210, DOA estimation unit 220, speed and Displacement Estimation module 230, movement locus and dynamics analysis module 240, interaction parameter produce and function key administration module 250 and application interface 260 and application and controlling object 300.
DOA estimation unit 220 as shown in Figure 4 is by DOA estimation module 221, magnetic field estimation module 222, acceleration estimation module 223 and put initial value module 224 and form; Data stream through data preprocessing module 210 pretreated three-dimensional gyroscopes 112 is sent to DOA estimation module 221, carries out DOA estimation by integral operation; Be sent to acceleration estimation module 223 through the data stream of data preprocessing module 210 pretreated acceleration transducers 111 and carry out acceleration estimation, be sent to magnetic field estimation module 222 through the data stream of data preprocessing module 210 pretreated three-dimensional magnetometers 113 and carry out magnetic field and estimate.
DOA estimation module 221 is main with the angular velocity data of three-dimensional gyroscope 112, merges the measurement data and their reference values in user coordinate system in acceleration of gravity and magnetic field simultaneously, estimates the position angle.In order to improve estimation accuracy, use Bayes's estimation method.At this moment, as the motion state vector, azimuthal state equation is with position angle q:
q t＝q t-1Θ(Δt)+V q (1)
Q wherein tBe t azimuthal vector constantly, Δ t is the sample interval, and Θ (Δ t) is the multiplication matrix with Quaternion Representation, V qIt is system noise.According to the principle of work of three sensors,, following measurement equation is arranged because q ties up to the azimuthal Quaternion Representation in the user coordinate system with sensor coordinates shown in Fig. 2:
Z t g = H ( q t ) + V t g - - - ( 2 )
Z t a = qrot ( q t , g ) + V t a - - - ( 3 )
Z t m = qrot ( q t , m ) + V t m - - - ( 4 )
Here, Z t g, Z t a, Z t mBe respectively that gyroscope, accelerometer and magnetometer are engraved in the measured value in the sensor coordinate system when t.V t g, V t a, V t mBe respectively that gyroscope, accelerometer and magnetometer are in t measurement noise constantly.The gyrostatic measured value of equation (2) expression is the azimuthal function H of synchronization (q t) and measure noise with; Equation (3) is very little at human-computer interaction device's acceleration itself; Compare under the negligible hypothesis prerequisite with acceleration of gravity, think that the measured value of accelerometer is value and the noise sum of gravity acceleration g through transforming to sensor coordinate system in the user coordinate system.Here qrot (q, g) and qrot (q m) be illustrated respectively in gravity acceleration g and magnetic field m in the user coordinate system through transforming to sensor coordinate system, and this conversion is defined by current orientation q.Here, we suppose that magnetic field is constant.
If with the measured value of three sensors with vector Z tExpression then has
Z t = ( Z t g , Z t a , Z t m ) T - - - ( 5 )
Like routine, T representes transposition, Z tIt is column vector.State equation has been arranged and measured equation, just can use Bayes's estimation method.Bayes's valuation is asked maximum a posteriori probability p (q to iterate t| Z 1:t-1) method come valuation.P (q t| Z 1:t-1) be known sensor data stream Z 1:t-1Condition under azimuthal posterior probability.Bayes's estimation method comprise two the step iterate, obtain state initial value q 0After, at a last moment posterior probability p (q T-1| Z 1:t-1) the basis on prediction and calculate posterior probability p (q t| Z T-1):
p(q t|Z 1:t-1)＝∫p(q t|q t-1)p(q t-1|Z 1:t-1)dq t-1 (6)
In the equation (6), conditional probability p (q t| q T-1) represented the temporal correlation (also claiming the markov characteristic) of dynamic system, it is obtained by equation (1).
The likelihood ratio that the renewal equation of Bayesian Estimation uses the measurement equation to try to achieve upgrades the posterior probability that (6) formula is tried to achieve, thereby obtains t azimuthal posterior probability constantly, accomplishes azimuthal valuation thus:
p(q t|Z 1:t)＝p(Z t|q t)p(q t|Z 1:t-1)/p(Z t|Z 1:t-1) (7)
Here, p (Z t| Z 1-1) can be considered normalized factor.If each sensor is separate, equation (7) can be written as:
p ( q t | Z 1 : t ) p ( Z t g | q t ) p ( Z t a | q t ) p ( Z t m | q t ) p ( q t | Z 1 : t - 1 ) / p ( Z t | Z 1 : t - 1 ) - - - ( 8 )
Here, p (Z t g| q t), p (Z t a| q t), p (Z t m| q t) be respectively t constantly gyroscope, accelerometer and magnetometer measures value in the orientation q tLikelihood ratio.They are tried to achieve by equation (2), (3) and (4).Realized the fusion of three kinds of sensing datas in the equation (8), the mode of fusion is multiplying each other of three likelihood ratios, and their contribution is regulated by the variance of likelihood ratio.
When solving equation (6) and (7), we are some hypothesis of pair distribution function work often.The most simply be the hypothesis of making linear equation and Gaussian distribution, it can use simple kalman filter method.Here, because measurement equation and state equation are not linear,, in most of valuations of the present invention, all use the UKF (Unscented Kalman Filter) that can be similar to nonlinear function in order to improve estimation accuracy.Following list of references has provided detailed narration: Zhiqiang Zhang, Zheng Wu, Jiang Chen; And Jiankang Wu; UbiquitousHuman Body Motion Capture using Micro-sensors, Seventh AnnualIEEE International Conference on Pervasive Computing andCommunications, PerCom 2009; March 9-13,2009.At the initial value q that obtains state variable 0(obtaining by putting initial value module 224 places) afterwards, just can use estimation method to find the solution.
Above in narrated in the DOA estimation module 221 the whole orientation estimation method and the formula of three sensing datas of fusion of realizing.But the hypothesis of this method is, magnetic field is constant, and it is very little that the acceleration of three-dimensional human-computer interaction device 100 itself are compared acceleration of gravity, can ignore.This hypothesis is inapplicable under many circumstances: for example, because the existence of various electrical equipment, magnetic field always changes with the place in time.In plurality of applications (like motion simulation), the acceleration of three-dimensional human-computer interaction device 100 has bigger amplitude, compares with acceleration of gravity and cannot ignore.Therefore, the present invention has increased magnetic field estimation module 222 and acceleration estimation module 223, the acceleration of the magnetic field that changes with real-time estimation and three-dimensional human-computer interaction device 100 itself.
In order to reflect truth, in magnetic field estimation module 222, we think that magnetic field changes, but this variation that changes with respect to the orientation is slowly.At moment t, we estimate the orientation with the formula of front earlier, estimate magnetic field and acceleration again.When estimating the orientation, use a last moment magnetic field estimated result.For this reason, changing equation (4) does
Z t m = qrot ( q t , m t - 1 ) + V t m - - - ( 4 - 1 )
In order to estimate magnetic field, be written as the state equation in magnetic field:
m t＝m t-1+V m (10)
Here, m is the three-dimensional state vector in magnetic field, V mBe noise.Measuring equation then is:
Z t m = qrot ( q t , m t ) + V t m - - - ( 11 )
Equation (4-1) and (11) though form class seemingly, meaning is different, equation (4-1) is normal parameter with magnetic field, the orientation is a state variable, produces likelihood function p (Z t m| q t), add in the DOA estimation system of equations.And equation (11) is normal parameter with the orientation that has estimated, and magnetic field is state variable, produces likelihood function p (Z t m| m t).A pair of state equation (10) has been arranged and measured equation (11), formed the basis of Bayes's valuation, similar with DOA estimation, we can find the solution the valuation problem in magnetic field.The result of valuation is the magnetic field value m in the user coordinate system t, be sent to DOA estimation module 221 and do next orientation valuation constantly.
Equally, we will estimate this acceleration a in sensor coordinate system of three-dimensional human-computer interaction device in acceleration estimation module 223 t s, be sent to DOA estimation module 221, produce the measured value Z of acceleration of gravity under sensor coordinate system thus t a-a t sBecause the acceleration of human-computer interaction device own is false much smaller than the hypothesis of acceleration of gravity, therefore, measures equation (3) and be rewritten as:
Z t a - a t - 1 s = qrot ( q t , g ) + V t a - - - ( 3 - 1 )
In order to estimate this acceleration in sensor coordinate system of human-computer interaction device, the state equation of valuation formula is in the acceleration estimation module 223:
a t s = a t - 1 s + V a - - - ( 13 )
Here, a sBe the three-dimensional state vector of acceleration in sensor coordinate system (what must remind is, three state vectors, promptly orientation q, magnetic field m and acceleration a when adding subscript s, are illustrated in the sensor coordinate system, do not add to be illustrated in timestamp in the user coordinate system), V aBe noise.The measurement equation is:
Z t a = qrot ( q t , g ) + a t s + V a s - - - ( 14 )
State equation (13) and measurement equation (14) are formed Bayes's valuation system of equations.Adopt same method for solving, we can estimate this acceleration in sensor coordinate system of three-dimensional human-computer interaction device from the measurement data of accelerometer with from the orientation of DOA estimation module 221, are sent to DOA estimation module 221.Simultaneously, it is converted to the acceleration qrot of three-dimensional human-computer interaction device under the user coordinate system itself -1(q t, a t s), be sent to speed and Displacement Estimation module 230.
Speed and Displacement Estimation module 230 are carried out integration (being summation) to the acceleration of three-dimensional human-computer interaction device under the user coordinate system itself and are come estimating speed and displacement under discrete case.And use and to put 224 of initial value modules and put initial value, obtain the position.DOA estimation module 221 all is sent to movement locus and dynamics analysis module 240 with the orientation of estimating and speed and speed, displacement and the position of Displacement Estimation module 230 with estimation.
Movement locus is connected with Displacement Estimation module 230 with speed with DOA estimation module 221 with dynamics analysis module 240.Movement locus and dynamics analysis module 240 are accepted the data such as orientation, speed and position of orientation estimation module 221 and speed and Displacement Estimation module 230, form a personal-machine interactive device 100 t (x of state vector completely constantly in user coordinate system t, v t, a t, q t, ω t).With human-computer interaction device 100 at current time t and in the past position, speed, acceleration, position angle and angular velocity state vector the sequence [(x in each moment t, v t, a t, q t, ω t), t=0,1,2 ...], also be movement locus, do further to analyze, just can obtain force direction and dynamics etc. and have direct applied parameter, and be sent to interaction parameter generation and function key administration module 250.For example, set mass value, self-acceleration just can push away acting force and direction.
Interaction parameter produces and 250 position, motion, track and dynamics parameters under user coordinate system of function key administration module; According to user coordinate system and the relation of using coordinate system; Under the control of function key,, movement locus, make force direction and dynamics is further calculated and processed on request to position angle, angular velocity, acceleration, speed, displacement; Desired parameters is used in output, is sent to application interface 260.For example:
In the 3 d medical images application examples; Relation according to a certain doctor's user coordinate system and 3-D view application coordinate system; To convert position and the orientation of using under the coordinate system into from the position in this user coordinate system and the direction parameter of last " movement locus and dynamics module 240 ", thereby can in 3-D view, select interested point or zone.
In the example of virtual court; The relation of using coordinate system according to player's user coordinate system and virtual court is (because the player possibly run; This relation also is dynamic); To be the relevant parameter of using under the coordinate system from the position in this user coordinate system, orientation, track and the dynamics Parameters Transformation of last " movement locus and dynamics module 240 ", be sent to application interface.In application, dock with the kinematic parameter of ball, thereby draw striking point on the ball, the motion of batting back ball.
Fig. 4 also shows three-dimensional human-computer interaction device 100 and handles and fusing device 200 and application interface 260 through three-dimensional human-machine interaction data, realizes system management with application and controlling object 300.Application and controlling object 300 have through the command information that application interface 260 mails to three-dimensional human-machine interaction data processing and fusing device 200: Apply Names, the requirement of data sampling rate, help information, initialization value; Send control commands such as data sampling rate to human-computer interaction device 100, regularly send simultaneously the state parameter that reads human-computer interaction device 100, like the power level of PMU 150 etc., the duty of decision-making system.
Shown in Figure 4 put that initial value module 224 is respectively DOA estimation module 221, speed and Displacement Estimation module 230 and interaction parameter produces and function management key module 250 is put initial value, the relation between the mutual relationship of initialization or replacement sensor coordinate system and user coordinate system (being stored among DOA estimation module 221, speed and the Displacement Estimation module 230) and user coordinate system and the application coordinate system (being stored in interaction parameter generation and function key administration module 250).The mutual relationship of initialization or replacement sensor coordinate system and user coordinate system; Or say orientation estimation module 221 and speed and Displacement Estimation module 230 are put initial value also owing to such reasons: the orientation valuation is mainly got by the angular velocity integration that gyroscope records, and speed and displacement are to be got by the valuation integration of the acceleration of human-computer interaction device 100 own.Because the existence of integration constant, initial value is all put in the valuation of position angle, speed and displacement, or says the problem of system initialization.Two kinds of modes of putting initial value are arranged:
A kind of is initialization, and the total system initialization is accomplished during by start.At this moment, sensor coordinate system and user coordinate system, and the relation of user coordinate system and application coordinate system is all got default value.In putting initial value module 224, the initialize routine of an acquiescence is arranged, when starting human-computer interaction device, automatically perform.This initialize routine is when initialization sensor coordinate system and user coordinate system concern; The terrestrial magnetic field that records with magnetometer is the z axle; The acceleration of gravity direction be the y axle pair to; The initial displacement of human-computer interaction device, speed, acceleration are zero, come initialization sensor coordinate system and user coordinate system; When using coordinate system and concern, come initialization at the initialization user coordinate system with the rotine positioning of user and application.For example, in the virtual ball tennis, suppose that the user is in a side court central authorities.
Another kind is the replacement initial value.At this moment, we often select an initial position.When concerning like replacement doctor's in 3 d medical images is used user coordinate system and 3-D view coordinate system; We let the doctor point to the center that 3 d medical images showed with the three-dimensional human-computer interaction device in the hand; Press reset button, put the initial position setting that initial value module 224 will be adjusted this application according to Application Design.
When replacement sensor coordinate system and user coordinate system concern initialization such as replacement sensor coordinate system and user coordinate system when concerning initialization; Difference according to application requirements; System initialization routine can have different three-dimensional human-computer interaction device 100 initial positions, and different orientation, speed and position initial value also can be arranged.One simple system initialization routine can be:
Three-dimensional human-computer interaction device 100 is placed the most original position of user coordinate system, as from initial point along the x change in coordinate axis direction, and get stationary state;
Press " initial value of resetting/put " key;
Put initial value module 224 and see the mandatory initial value in orientation (0,90 °, 90 °) off to the DOA estimation module; To speed and the mandatory initial value 0 of Displacement Estimation module 230 rate of deliveries and position;
The above; Be merely the embodiment among the present invention, but protection scope of the present invention is not limited thereto, anyly is familiar with this technological people in the technical scope that the present invention disclosed; Can understand conversion or the replacement expected; All should be encompassed in of the present invention comprising within the scope, therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.

Claims (8)

1. a three-dimensional man-machine interactive system is characterized in that, comprising: this system the whole sensing data that is defined as alternately under sensor coordinate system, user coordinate system, application coordinate system obtain, handle, fusion and transfer process, wherein:
Three-dimensional human-computer interaction device comprises a plurality of microsensors, and a plurality of microsensors obtain and export the position of three-dimensional human-computer interaction device and the sensing data of motion in sensor coordinate system;
Three-dimensional human-machine interaction data processing is connected with three-dimensional human-computer interaction device with fusing device; Receive the sensing data that three-dimensional human-computer interaction device is gathered; And there is the sensing data of noise and dynamic change to merge to a plurality of microsensor institute measuring tool; Obtain location parameter and the motion state of three-dimensional human-computer interaction device in user coordinate system, i.e. the accurate valuation of position, speed, acceleration, orientation, angular velocity, derivation movement locus, force direction and dynamics; And convert the interaction parameter of using in the coordinate system to;
Use with controlling object and be connected with fusing device with three-dimensional human-machine interaction data processing through application interface; The accurate valuation of receiving position parameter and motion state and the interaction parameter of releasing thus; Be used for control and use and controlling object, handle and fusing device transmission command information to three-dimensional human-machine interaction data through application interface;
Said three-dimensional human-machine interaction data is handled and fusing device comprises that data preprocessing module, DOA estimation unit, speed and Displacement Estimation module, movement locus and dynamics analysis module, interaction parameter produce and function key administration module and application interface, wherein:
Data preprocessing module is connected with three-dimensional human-computer interaction device respectively; Data preprocessing module receives a plurality of microsensors of three-dimensional human-computer interaction device respectively and measures acceleration, angular velocity and magnetic-field measurement data in this sensor coordinate system; Handle the data stream of acceleration, angular velocity and magnetic-field measurement in generation and the output transducer coordinate system through denoising and integrity detection;
The DOA estimation unit is connected with data preprocessing module; The DOA estimation unit receives and the data stream of the acceleration in the sensor coordinate system, angular velocity and magnetic-field measurement is handled, and produces and the orientation valuation of output under changes of magnetic field and the acceleration situation of human-computer interaction device own;
Speed is connected with the acceleration estimation module of DOA estimation unit with the Displacement Estimation module, and speed and Displacement Estimation module are used the acceleration valuation from the human-computer interaction device of acceleration estimation module itself, obtain speed and displacement data through integration;
Movement locus is connected with the DOA estimation module of speed with Displacement Estimation module and DOA estimation unit respectively with the dynamics analysis module; The orientation valuation that speed that inbound pacing and Displacement Estimation module are come and displacement data and DOA estimation module are come; Form human-computer interaction device each state constantly in user coordinate system; Comprise position, speed, acceleration, orientation, angular velocity, and and then release movement locus, force direction and dynamics;
Interaction parameter produces and is connected with the dynamics analysis module with movement locus with the function key administration module; Interaction parameter generation and function management module are according to user coordinate system and the relation of using coordinate system; The position under user coordinate system, movement locus, force direction and dynamics parameter; Under the control of function key; On request orientation, angular velocity, acceleration, speed, displacement, movement locus, force direction and the dynamics of human-computer interaction device are further calculated and processed, interaction parameter is used in output;
Application interface produces with interaction parameter respectively and is connected with controlling object with application with the function key administration module, and the application interaction parameter that draws in application interface reception and the forwarding application coordinate system is given and used and controlling object.
2. three-dimensional man-machine interactive system as claimed in claim 1 is characterized in that, three-dimensional human-computer interaction device comprises: a plurality of microsensors, central controlled processing unit, wireless communication chips and one group of function key, wherein:
A plurality of microsensors extract the three-dimensional position and the exercise data of three-dimensional human-computer interaction device itself, and through exporting the simulated data of three-dimensional position and motion after the conversion processing and amplifying;
Each road analog to digital converter of central authorities' controlled processing unit or numeral inlet are connected respectively to all the sensors data terminal in a plurality of microsensors; Central authorities' controlled processing unit receives by the sampling rate of setting and the three-dimensional position of a plurality of microsensor outputs and the simulated data of motion is controlled and handled; The sampled data that obtains a plurality of microsensors is deposited in the register of central controlled processing unit, and these sampled datas is sticked the instruction of sequential label and function key pack together and export sampled data and instruct with function key; Central authorities' controlled processing unit receives, explains and carries out from three-dimensional human-machine interaction data and handle and fusing device place computer commands;
Wireless communication chips is connected with central controlled processing unit, receives and export the function key instruction and the sampled data of central controlled processing unit; Wireless communication chips is set up three-dimensional human-computer interaction device and is belonged to communicating by letter of computing machine or computing equipment, the data between completion computing machine or the computing equipment and the transmission of order with three-dimensional human-machine interaction data processing and fusing device;
Function key is connected with central controlled processing unit, and the information of function key is sent to three-dimensional human-machine interaction data by central controlled processing unit with it packing and handles and fusing device through interrupting.
3. three-dimensional man-machine interactive system as claimed in claim 2; It is characterized in that; Said microsensor is multiple microsensor, and multiple microsensor is following microsensor more than three kinds: three dimension acceleration sensor, three-dimensional gyroscope, three-dimensional magnetometer, ultrasonic range finder, laser range finder.
4. three-dimensional man-machine interactive system as claimed in claim 2 is characterized in that, said function key is initialization key or reset button, options button or function display key.
5. three-dimensional man-machine interactive system as claimed in claim 1 is characterized in that, said DOA estimation unit is by DOA estimation module, magnetic field estimation module, acceleration estimation module and put the initial value module and form, and is used for reducing skew, improves estimation accuracy, wherein:
The input end of DOA estimation module is connected with data preprocessing module, magnetic field estimation module, acceleration estimation module, the output terminal of putting the initial value module respectively; Acceptance angle speed data stream, receive the magnetic field estimation module field signal, receive acceleration estimation module acceleration of gravity signal, receive put the initial value module the initial orientation signal; The DOA estimation module is used Bayes's estimation method, is starting point with the initial orientation value of putting the initial value module, is main with gyrostatic angular velocity measurement data, merges the restriction to the orientation valuation of magnetic field and acceleration of gravity simultaneously, thereby realizes the low deviation valuation in orientation; Used non-linear valuation wave filter to reduce the non-linear issuable error of the non-linear and sensor measurement of motion; The DOA estimation module is respectively to movement locus and dynamics analysis module, magnetic field estimation module and acceleration estimation module output orientation valuation;
The magnetic field estimation module is connected with the DOA estimation module with data preprocessing module respectively; Receive the magnetic-field measurement data stream of data preprocessing module output, the orientation valuation of reception orientation estimation module, use non-linear estimation method to realize valuation is carried out in the magnetic field that changes with position and time; The result of magnetic field valuation is sent to the DOA estimation module and participates in next orientation valuation constantly;
The acceleration estimation module is connected with the DOA estimation module with data preprocessing module respectively; Receive the orientation valuation of acceleration information stream and the output of DOA estimation module; The acceleration estimation module uses non-linear valuation wave filter to realize the valuation of this acceleration under sensor coordinate system of human-computer interaction device; From the acceleration analysis data, isolate gravitational acceleration component with the valuation result, be sent in the DOA estimation module, next valuation constantly of participating in the orientation is merged; Simultaneously, the orientation valuation of using the orientation estimation module to provide is transformed into the acceleration valuation in the sensor coordinate system in the user coordinate system, is sent to speed and Displacement Estimation module;
Put initial value module and DOA estimation module, speed and Displacement Estimation module and produce with interaction parameter and is connected, put system initialization submodule and coordinate system replacement submodule in the initial value module and through DOA estimation module, speed and Displacement Estimation module and interaction parameter generation and function key administration module the coordinate system of orientation, speed, position and user and application is concerned and put initial value with the function key administration module; Replacement initial value when receiving control message from keyboard and instruction interpretation signal under simple standard scene.
6. three-dimensional man-machine interactive system as claimed in claim 1; It is characterized in that application and controlling object have through the command information that application interface mails to three-dimensional human-machine interaction data processing and fusing device: Apply Names, the requirement of data sampling rate, help information, initialization value; Send data sampling rate control command to human-computer interaction device, regularly send the power values of the PMU that reads human-computer interaction device simultaneously, the working status parameter of decision-making system.
7. three-dimensional man-machine interactive system as claimed in claim 5; It is characterized in that; The acceleration of gravity direction that records in the principal direction that system initialization submodule is used the measured magnetic field of magnetometer and the acceleration transducer is put the acquiescence initialization for sensor coordinate system and user coordinate system concern; Use initial scene to put the acquiescence initial value as user coordinate system and application coordinate system relation; According to the difference of initial scene, system initialization submodule has different three-dimensional human-computer interaction device initial positions and different orientation, speed and position initial values.
8. three-dimensional man-machine interactive system as claimed in claim 1; It is characterized in that; Said three-dimensional human-computer interaction device be conveniently hold, wear, pin is tied or wear the shape of mode attached to certain part of people's health, this shape be hand rectangular-shaped, lip pencil, commander bar-shaped, hold in hand spherical, implant laminar in the cap or be anchored on laminar on pin, shank or the arm.
CN2009100938891A 2009-09-23 2009-09-23 Three-dimensional man-machine interaction system Expired - Fee Related CN102023700B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2009100938891A CN102023700B (en) 2009-09-23 2009-09-23 Three-dimensional man-machine interaction system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2009100938891A CN102023700B (en) 2009-09-23 2009-09-23 Three-dimensional man-machine interaction system

Publications (2)

Publication Number Publication Date
CN102023700A CN102023700A (en) 2011-04-20
CN102023700B true CN102023700B (en) 2012-06-06

Family

ID=43865075

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2009100938891A Expired - Fee Related CN102023700B (en) 2009-09-23 2009-09-23 Three-dimensional man-machine interaction system

Country Status (1)

Country Link
CN (1) CN102023700B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10010269B2 (en) 2015-08-05 2018-07-03 Wistron Corporation Calibration method and electronic device
US10088496B2 (en) 2015-08-27 2018-10-02 Wistron Corporation Calibration method and sports equipment

Families Citing this family (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102955566A (en) * 2011-08-31 2013-03-06 德信互动科技（北京）有限公司 Man-machine interaction system and method
JP5648647B2 (en) * 2012-03-21 2015-01-07 カシオ計算機株式会社 Image processing apparatus, image processing method, and program
TWI444851B (en) * 2012-04-18 2014-07-11 Au Optronics Corp Three-dimensional interactive system and method of three-dimensional interactive
CN102707305B (en) * 2012-06-06 2014-07-16 山东大学 Tumble detecting and positioning system and method
EP2687950A1 (en) * 2012-07-20 2014-01-22 BlackBerry Limited Orientation sensing stylus
KR101307664B1 (en) * 2012-09-11 2013-09-12 박희재 A client device implementing middleware to link between heterogeneous exergame equipment and variable game contents and management method thereof
WO2014129105A1 (en) * 2013-02-22 2014-08-28 ソニー株式会社 Head-mounted display system, head-mounted display, and control program for head-mounted display
CN103488291B (en) * 2013-09-09 2017-05-24 北京诺亦腾科技有限公司 Immersion virtual reality system based on motion capture
CN103546690A (en) * 2013-10-30 2014-01-29 天彩电子（深圳）有限公司 Method for obtaining and displaying motion data of action camera
CN103877726B (en) * 2014-04-10 2017-09-26 北京蚁视科技有限公司 A kind of virtual reality components system
CN104199540B (en) * 2014-08-01 2019-01-22 惠州Tcl移动通信有限公司 Communication system, wearable device and its control method
CN105327505B (en) * 2014-08-20 2019-01-01 广东小天才科技有限公司 Motion sensing game control equipment and control method thereof
WO2016033717A1 (en) * 2014-09-01 2016-03-10 北京诺亦腾科技有限公司 Combined motion capturing system
CN105487692A (en) * 2014-12-22 2016-04-13 哈尔滨安天科技股份有限公司 Controller switching method and system based on three-dimensional display space
CN104958897A (en) * 2015-06-25 2015-10-07 郭斌 Movement track and movement speed collecting device and system
CN105068679A (en) * 2015-07-22 2015-11-18 深圳多新哆技术有限责任公司 Method and device for regulating position of virtual object in virtual space
CN105068653A (en) * 2015-07-22 2015-11-18 深圳多新哆技术有限责任公司 Method and apparatus for determining touch event in virtual space
CN105096384B (en) * 2015-07-22 2018-10-26 深圳多新哆技术有限责任公司 Adjust the method and device of virtual article position and attitude angle
TW201709025A (en) * 2015-08-26 2017-03-01 巨擘科技股份有限公司 Device for integrating position, gesture, and wireless transmission
CN105354820B (en) 2015-09-30 2018-05-22 深圳多新哆技术有限责任公司 Adjust the method and device of virtual reality image
CN105573496B (en) * 2015-12-11 2018-10-23 山东大学 A kind of division methods of working regions Kinect and application
CN105759952A (en) * 2015-12-28 2016-07-13 北京像素软件科技股份有限公司 Method and device for generating input information according to postures of four limbs
CN105536144B (en) * 2016-01-23 2018-07-10 中山大学 A kind of device with the human-computer interaction function of game and massage
WO2017185222A1 (en) * 2016-04-26 2017-11-02 深圳市优宝创科技有限公司 System and method for motion trajectory collection and analysis based on ball games
CN105955467A (en) * 2016-04-26 2016-09-21 四川川大智胜软件股份有限公司 Multi-man man-machine interaction method based on mobile terminal
CN105976000A (en) * 2016-04-28 2016-09-28 赵阳 Positioning system and method
CN105955510A (en) * 2016-04-28 2016-09-21 赵阳 Man-machine interaction system and method
CN106094540B (en) * 2016-06-14 2020-01-07 珠海格力电器股份有限公司 Electrical equipment control method, device and system
CN106468951A (en) * 2016-08-29 2017-03-01 华东师范大学 A kind of intelligent remote control systems based on the fusion of both hands ring sensor and its method
CN107945602A (en) * 2016-10-12 2018-04-20 阿里巴巴集团控股有限公司 A kind of equipment operation examination/Training Methodology, apparatus and system
CN106621320A (en) * 2016-11-29 2017-05-10 维沃移动通信有限公司 Data processing method of virtual reality terminal and virtual reality terminal
CN106780754B (en) * 2016-11-30 2021-06-18 福建北极光虚拟视觉展示科技有限公司 Mixed reality method and system
CN107506038B (en) * 2017-08-28 2020-02-25 荆门程远电子科技有限公司 Three-dimensional virtual earth interaction method based on mobile terminal
TWI676915B (en) * 2017-09-13 2019-11-11 新普科技股份有限公司 Method of generating control commands for an electronic device based on signal accumulation amount of sensors
CN107783653A (en) * 2017-11-06 2018-03-09 广州新维感信息技术有限公司 A kind of virtual reality exchange method and system
CN108592902B (en) * 2018-05-08 2020-07-31 清华大学 Positioning equipment, positioning method and system based on multiple sensors and mechanical arm
EP3588474A1 (en) * 2018-06-28 2020-01-01 West & Berg Holding AB Real time golf swing training aid
CN112107852A (en) * 2020-09-16 2020-12-22 西安万像电子科技有限公司 Cloud game control method and device and cloud game control system
CN115218645A (en) * 2021-04-15 2022-10-21 中国科学院理化技术研究所 Agricultural product drying system
CN113406646B (en) * 2021-06-18 2023-03-31 北京师范大学 Method and equipment for three-dimensional positioning based on multi-direction ultrasonic ranging and IMU (inertial measurement Unit)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101086681A (en) * 2006-06-09 2007-12-12 中国科学院自动化研究所 Game control system and method based on stereo vision
CN201514612U (en) * 2009-09-23 2010-06-23 吴健康 Three-dimensional dynamic positioning equipment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101086681A (en) * 2006-06-09 2007-12-12 中国科学院自动化研究所 Game control system and method based on stereo vision
CN201514612U (en) * 2009-09-23 2010-06-23 吴健康 Three-dimensional dynamic positioning equipment

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10010269B2 (en) 2015-08-05 2018-07-03 Wistron Corporation Calibration method and electronic device
US10088496B2 (en) 2015-08-27 2018-10-02 Wistron Corporation Calibration method and sports equipment

Also Published As

Publication number Publication date
CN102023700A (en) 2011-04-20

Similar Documents

Publication Publication Date Title
CN102023700B (en) Three-dimensional man-machine interaction system
CN201514612U (en) Three-dimensional dynamic positioning equipment
CN101579238B (en) Human motion capture three dimensional playback system and method thereof
CN102667672B (en) Acceleration motion identify method and system thereof
US9604101B2 (en) Golf swing analysis device and golf swing analysis method
US20050032582A1 (en) Method and apparatus for determining orientation and position of a moveable object
CN201431466Y (en) Human motion capture and thee-dimensional representation system
CN104225897A (en) Motion analysis method and motion analysis device
EP2717241A2 (en) Golf swing analyzing apparatus and method of analyzing golf swing
CN104548555A (en) Movement analysis method, and movement analysis apparatus
CN104225899A (en) Motion analysis method and motion analysis device
CN104197987A (en) Combined-type motion capturing system
CN105233483A (en) Motion analysis method and motion analysis device
Burchfield et al. A framework for golf training using low-cost inertial sensors
EP3078398A1 (en) Analysis device, analysis method and recording medium
JP6054331B2 (en) Improved fitting system for golf clubs
CN106573168A (en) Motion analysis method, motion analysis device, motion analysis system, and program
KR20160106670A (en) Movement analysis method, movement analysis device, movement analysis system and program
Cho et al. Motion recognition with smart phone embedded 3-axis accelerometer sensor
US20180216959A1 (en) A Combined Motion Capture System
CN113058246A (en) Wheelchair curling track identification, positioning, tracking and motion state detection system
CN106102845A (en) Motion analyzing apparatus, MAS, method of motion analysis, the display methods of motion analysis information and program
CN110787433A (en) Golf ball sensing system for golf simulator
KR20160076485A (en) Exercise analysis device, exercise analysis system, exercise analysis method, display device, and recording medium
CN104587662A (en) motion analyzing apparatus andmotion analyzing method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: Three-dimensional man-machine interactive system

Effective date of registration: 20140326

Granted publication date: 20120606

Pledgee: Yancheng City Lake District New District Management Committee

Pledgor: Wu Jiankang

Registration number: 2014990000203

PLDC Enforcement, change and cancellation of contracts on pledge of patent right or utility model
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20120606

Termination date: 20210923