CN101069214A - System and method for tracking facial muscle and eye motion for computer graphics animation - Google Patents
System and method for tracking facial muscle and eye motion for computer graphics animation Download PDFInfo
- Publication number
- CN101069214A CN101069214A CNA2005800407214A CN200580040721A CN101069214A CN 101069214 A CN101069214 A CN 101069214A CN A2005800407214 A CNA2005800407214 A CN A2005800407214A CN 200580040721 A CN200580040721 A CN 200580040721A CN 101069214 A CN101069214 A CN 101069214A
- Authority
- CN
- China
- Prior art keywords
- emg
- performing artist
- eog
- electrode
- muscle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a motion tracking system enables faithful capture of subtle facial and eye motion using a surface electromyography (EMG) detection method to detect muscle movements and an electrooculogram (EOG) detection method to detect eye movements. Signals corresponding to the detected muscle and eye movements are used to control an animated character to exhibit the same movements performed by a performer. The motion tracking electrode is affixed to a skin surface of a performer at plural locations corresponding to respective muscles, and a processor operatively coupled to the plurality of pairs of EMG and EOG electrodes. The programming instructions further provide processing of the EOG data and mapping of the processed EOG data onto the animated character. As a result, the animated character will exhibit the same muscle and eye movements as the performer.
Description
Related application data
Present patent application requires the right of priority of following temporary patent application according to 35U.S.C. § 119 (c): (a) sequence number 60/616,049 title of submitting on October 4th, 2004 is " system and method for capturing facial and eye motion " (SYSTEM AND METHOD FORCAPTURING FACIAL AND EYE MOTION); And the sequence number of (b) submitting on October 1st, 2004 _ _ _ _ title is " tracks facial features moves under the situation that the facial face of no vision catches " (TRACKING OF FACIAL FEATUREMOVEMENTS WITHOUT VISUAL FACE CAPTURE).
Technical field
The present invention relates to three-dimensional picture and animation, more particularly, relate to the face and the motion tracking system of eye motion that do not use video camera and can catch the performing artist in making the computer graphical animation, to use.
Background technology
Use motion capture system to catch the motion of real object, and it is mapped on the object of computing machine generation.This type systematic usually is applied in the making of motion picture and video-game to create the numeral as the people of the source data of creating computer graphical (CG) animation.In exemplary systems, performer wears has the suit of the marker (for example having the little reaction equation marker that is attached to health and limbs) attached to diverse location, and digital camera writes down moving of performing artist from different perspectives when these markers of irradiation.System is marker position (for example as volume coordinate) and the direction in each frame of analysis image to determine that performing artist's cover is loaded onto then.By the position of trace labelling device, the space representation of marker on the system creation different time, and the numeral of the performing artist in the structure motion.Then with this sports applications in digital model, then it is applied texture and the CG that finishes that plays up to make performer and/or performance represents.This technology is used for making incredible realistic animations in many popular movies by special effects companies.
Motion capture system also is used to follow the tracks of performer's the motion of facial characteristics with the expression of the facial movement of creating the performer and expression (for example laugh, cry, smile etc.).The same with body motion capture, marker is attached to performer's face and camera record performer's expression.Because related more big muscle in moving than health, facial moving relates to less relatively muscle, so the facial markers device is usually much smaller than the corresponding body marker, and video camera has more high resolution than the video camera that usually is used for body motion capture usually.Usually video camera is moved copline with performer's physics and align, limit the maintenance video camera simultaneously and focus on performer's the face.Facial motion capture system can be integrated in the helmet or physical attachment in other equipments of performer, so that shine the facial markers device uniformly and with the near minimum of the degree that relatively moves between video camera and the face.
Motion capture system is to realize real-time visual with respect to the advantage of traditional animation techniques (for example key frame processing).Production team can be in real time or near checking the space representation of performing artist's motion in real time, thereby make the performer can change the physics performance so that catch optimal data.And the nuance that the physics that the motion capture system detection uses other cartoon techniques to reproduce easily moves obtains to reflect more accurately the data that nature moves thus.As a result of, utilize the animation of the source material establishment of adopting the motion capture system collection will show more lively outward appearance.
Although motion capture system has these advantages, the shortcoming of conventional motion capture system is that they can't catch eye motion.Because marker can't be attached at performing artist's eyes,, motion capture cameras moves less than eyes so detecting.Must add these eyes in follow-up CG animation process then moves.Except making that animation process more bothers, the animation product shortcoming verisimilitude that is produced is because it can't comprise that the trickle eyes that produce during the performance move.
Another shortcoming that depends on the conventional motion capture system of video camera is, because of blocking performing artist's exercise data with the interference of other objects of for example stage property or other performers.Exactly, if the part of health or facial markers device is blocked in outside the visual field of digital camera, then collect data less than relevant this health or facial parts.This causes blocking or the cavity in the exercise data.Block though can during post-production after a while, use conventional computer graphics techniques to fill up, the quality of fill data shortcoming actual motion data, thus cause the animation defective that can be identified by spectators.For fear of this problem, conventional motion capture systems limit the quantity of object that can disposable seizure, for example be limited to a performing artist.This also often makes exercise data look true to nature inadequately, because the quality of performing artist's performance usually depends on the interaction with other performers and object.And, be difficult to the performance of these separation to look that the mode of nature combines.
Outside the amusement industry, many other situations are arranged, expectation seizure or tracks facial muscle and/or eyes move, and do not have the dependence to optical camera.For example, automatic speech recognition equipment, gate control system, the electronics storage and retrieval system that is used for personal profiles and/or medical treatment/dentist's screening system all can be utilized this type of analytical technology.Utilize the speech recognition system of facial-feature analysis in the noise circumstance that is difficult to use separately the sound speech recognition (for example on the military aircraft or in the factory), to be used widely.Each environment of these potential application is short of the effective parts that are used for accurately individual facial characteristics being converted to useful electronic data at present.Constantly change in the situation of face orientation especially existing problems with respect to checkout equipment this person.
Therefore, expectation provides a kind of motion tracking system that overcomes these and other shortcomings of prior art.More properly, expectation provides a kind of subtle facial that can catch the performing artist faithfully and eye motion and does not use the motion tracking system of video camera.
Summary of the invention
The present invention overcomes the shortcoming of prior art by the motion tracking system that can catch subtle facial and eye motion faithfully is provided.The present invention uses surface electromyography (EMG) detection method to detect muscle and move, and uses electroculogram (EOG) detection method to detect eyes and move.The signal controlling animated character that use is moved corresponding to detected muscle and eyes presents identical the moving that the performing artist carries out.
More particularly, the embodiment of motion tracking animation system comprises a plurality of electromyograms (EMG) electrode pair, and these a plurality of electromyograms (EMG) electrode pairs are suitable for being attached at performing artist's skin surface a plurality of positions corresponding with each muscle; And the processor that in operation, is coupled to these a plurality of EMG electrode pairs.This processor comprises programming instruction, and these programming instructions are carried out the function of obtaining the EMG data from a plurality of EMG electrode pairs.The EMG data comprise with performance during performing artist's muscle move corresponding electric signal.Programming instruction also comprises, and the digital model of EMG data to provide muscle to move is provided, and this digital model is mapped on the animated character.As a result of, the animated character will present the muscle identical with the performing artist and move.
In an embodiment of the present invention, a plurality of electroculograms (EOG) electrode pair is suitable for being attached at performing artist's skin surface and performing artist's eyes position adjacent place.Processor is coupled to a plurality of EOG electrode pairs in operation, and comprises programming instruction, and these programming instructions are carried out the function of obtaining the EOG data from a plurality of EOG electrode pairs.The EOG data comprise with performance during performing artist's eyes move corresponding electric signal.Programming instruction also provides the processing of EOG data, and with handled EOG data map to the animated character.Making the animated character present the eyes identical with the performing artist like this moves.
By detailed description of the preferred embodiment hereinafter, with face and the motion tracking system of eye motion that those skilled in the art can more fully be understood to catch the performing artist, and realize attendant advantages of the present invention and purpose in Computergraphic animation, to use.Hereinafter will at first these accompanying drawings will be described briefly below with reference to the accompanying drawing of institute's attached sheet.
Description of drawings
Fig. 1 is the block diagram of diagram according to the motion tracking system of the embodiment of the invention;
Fig. 2 is the front view that comprises human face's myoarchitecture of exemplary electrodes placement location;
Fig. 3 is used to use electromyogram (EMG) signal system to make the method for animation frame according to the embodiment of the invention;
Fig. 4 is the method that is used to handle the EMG signal;
Fig. 5 is the method that is used to handle calibration poses;
Fig. 6 is the side view that the human eye of the voltage potential (voltage potential) that exists between cornea and the eyeground is shown;
Fig. 7 A and Fig. 7 B illustrate the synoptic diagram that moves corresponding electric field measurement with eyes;
Fig. 8 is the photo that is pasted with the performing artist of facial muscles and eyes potential electrode; And
Fig. 9 illustrates that use is from the facial muscles of performing artist's acquisition of Fig. 8 and the animated character that the eyes mobile data is played up.
Embodiment
As described further below, the present invention satisfies the requirement of the motion tracking system of the subtle facial can catch the performing artist under the situation of not using video camera faithfully and eye motion.In describing in detail hereinafter, similar element number is used for describing the illustrated similar components of one or more accompanying drawings.
At first with reference to figure 1, block diagram illustration is according to the motion tracking system 100 of the embodiment of the invention.Motion tracking system 100 comprises motion tracking processor 108, and this motion tracking processor 108 is made as by the electrode interface 112 that is fit to and communicates by letter with a plurality of eye motion electrode pairs with a plurality of facial muscles electrode pairs.Motion tracking processor 108 can also comprise programmable calculator, and this programmable calculator has the data storage device 106 that is made as the data file that can store association.As known in the art, can use network that one or more computer workstations are coupled to motion tracking processor 108 so that a plurality of graphic designs personnel can use the data file of storage in the process of creating the computer graphical animation.Motion tracking processor 108 can also comprise the Computergraphic animation system that commercial packages for example provides, and this Computergraphic animation system can be created 3D figure and the animation that is used for show business, for example Alias|Wavefront
TMMaya software product line or other similar products sold.Should be appreciated that Computergraphic animation system can comprise the computer hardware and the software systems of separating fully with motion tracking processor 108, perhaps, can be used as a part and the motion tracking processor 108 integrated (for example " plug-in unit ") of total hardware and software system.
The muscle electrode pair comprises electrode 120
1, 122
1To 120
N, 122
N, they pass through electric conductor 116 separately
1, 118
1To 116
N, 118
NBe coupled to electrode interface 112.Also ground-electrode 126 is coupled to electrode interface 112 by electric conductor 124.In a preferred embodiment of the invention, the muscle electrode pair comprises surface electromyography (EMG) electrode, the voltage difference that the depolarization ripple of propagating along muscle surface that these surface electromyography (EMG) electrode measurement takes place when muscle is crooked is caused.The detected signal of surface electrode is usually in the scope of 5mV.(or vertically align with the pulse that should get rid of) should align these electrodes with the direction of the expectation of electric pulse.
The eye motion electrode pair comprises electrode 136
1, 138
1With 136
2, 138
2, they pass through electric conductor 132 separately
1, 134
1With 132
2, 134
2Be coupled to electrode interface 112.Ground-electrode 144 also is coupled to electrode interface 112 by electric conductor 142.In a preferred embodiment, detect eye motion by obtain and measure electroculogram (EOG) from eyes.
Fig. 2 illustrates has people's anatomical model 200 that the facial muscles system that facial muscular position puts is specified in the demonstration that is used for the EMG electrode pair.Use the EMG electrode pair to catch data from each respective muscle group.For example, electrode pair location 1L is corresponding to the muscle of being responsible for lifting left inner brow, and electrode pair location 2L is corresponding to the muscle of being responsible for lifting the left-external side eyebrow, and electrode pair location 3 is corresponding to the muscle of the eyebrow of being responsible for loweing, or the like.(for example use cosmetic glue) by conventional methods electrode pair is attached at performing artist's skin surface appointed positions.The moving of corresponding muscle that should be realized that the performing artist will produce the electric signal that passes through the detected association of counter electrode.Because everyone muscle has trickle difference, thus may need the test of certain amount accurately to search and the electrode of locating each performing artist to realize having most data aggregation.
As known in the medical domain, upper facial muscles is responsible for changing the outward appearance of eyebrow, forehead and last palpebra inferior.Shrink the static insertion point of the forehead muscle isostension ground (isotonically) of portion on cranium on the face, thereby the surface texture of making (being skin) can vertically be swelled with the direction of muscle and be wrinkling.Lower facial muscles is not made of on the same group a plurality of, comprises musculus orbicularis oris along the greater zygomatic muscle that shrinks towards the angled direction of cheekbone and lip, circular naturally or oval and orbicular muscle of eye that stretch around eyes, difficult to articulate stretching, the buccinator muscle that flatly shrinks towards ear and other muscle of controlling multiple other actions.The muscle of mouth has especially complicated muscle interaction.Musculus orbicularis oris is the sphincter that is not attached to bone.Three main muscle are promptly put on face flesh (M.Levator), levator labii superioris (Labii Superioris) and wing of nose flesh from the top combination, and frontalis is in combination of the host node place of mouth and contraction flatly simultaneously.Depressor supercilii (M.Depressor), depressor muscle of angle of mouth (Anguli Oris), depressor muscle of lower lip (M.Depressor Labii Inferioris) and mentalis (Mentalis) is all oblique and vertically shrink.Facial expression is moved by the complexity of these upper and lower facial muscles and combination and forms.
Fig. 2 also illustrates the exemplary designated of EOG electrode pair.Electrode pair location E01 adjacent with eyes and E02 are suitable for detecting with an eyes left side/related signal moves right.The electrode pair location E03 of eye below and top and E04 be suitable for detecting with eyes on/down mobile related signal.Ground electrode location GD is positioned on the forehead.Note ground-electrode can be attached on the health not can interference with facial muscle application and any position of action, for example neck, shoulder, ancon etc.Forehead is represented the position that makes things convenient for of ground-electrode, is can interference with facial muscle action because it approaches other electrodes.Notice that second ground-electrode is absolutely necessary for the EMG electrode, this second ground-electrode can be positioned at the zone identical with the EOG ground-electrode.Because the eyes that dipole is represented the performing artist move parallel, so only need to collect/motion down from eyes wherein.The same with the EMG electrode, the EOG electrode (for example uses cosmetic glue) by conventional methods and is attached at performing artist's skin surface appointed positions.
Briefly, the side cross-sectional, view of human eye 600 is shown with reference to figure 6.Cornea 602 is positioned at the front surface of eyes, and the surface of transmission from the light of object reflection is provided.Cornea 602, crystalline lens 604 and vitreum 612 are with light deflection, refraction and focusing.Eyeground 608 provides the indent of eyes, and this indent comprises retina 606.To retina 606, retina converts light to the electric pulse that is sent to brain through optic nerve 610 to crystalline lens 602 with the image focusing of object.As known in the art, there is voltage potential between cornea 602 and the eyeground 608.This voltage potential can be measured electric field and makes the variation of this electric field form with the mobile relevant dipole of eyes by sending.
Fig. 7 A and Fig. 7 B diagram electrode pair 136
1, 138
1Two dipoles that surround.These two dipoles are corresponding to performing artist's eyes 600L, each of 600R.Shown in Fig. 7 A, when dipole (being eyes) during towards the place ahead, electrode 136
1, 138
1Between the electric field measured be empty.In Fig. 7 B, dipole is towards the right, the electrode 136 of inducting thus
1, 138
1Between positive electric field.Electrode 136
1, 138
1Between provide the suitable positional information of eyes with respect to the electric field change of ground connection.With respect to eyes with nominal the place ahead approximately ± move in 20 ° the scope, this electric field change keeps quite linearity.
Turn back to Fig. 1 now, motion tracking processor 108 provides the signal Processing and the amplification of the signal that EMG and EOG electrode detection are arrived.With respect to the EMG electrode, signal Processing can comprise rectification to the received signal.Because original signal is a two-phase, so its mean value is zero.Rectification makes electric current only for unidirectional, therefore will stride across the negative content " upset " of the signal of zero axle, thereby makes whole signal for just.The phase component of the signal that receives from the EOG electrode is vital for the direction of determining eyes, so can carry out rectification for these signals.Signal Processing can also comprise that to the received signal filtering and linear envelope detect.The EMG signal is actual to be the mixing of many signals and some noises.These voltages also rise and descend at different rates or frequency, thereby form frequency spectrum.Motion tracking processor 108 can be with mixed signal filtering, to eliminate for example electrical noise of motion artifact (movement artifact).Most of EMG are present between frequency range 20 and the 200Hz.Motion artifact has the frequency less than 10Hz, and noise has the frequency range of the EMG scope that is higher than expectation.Therefore, signal can be eliminated motion artifact and noise by bandpass filter.Electrical interface 112 and/or motion tracking processor 108 can also amplify the signal that receives they are risen to the level that is suitable for signal Processing.In an embodiment of the present invention, electrical interface can be provided by the Myopac that RunTechnologies company makes.
Fig. 3 diagram realizes according to the demonstration of the face tracking method of the embodiment of the invention.This method is included in step 302 at first and obtains the EMG signal, and is comprising electrode being placed facial privileged site, as discussed above.For example, electrode can be placed on the selected facial muscles group.Thus, when the performing artist made facial expression, related facial muscles were according to the mobile generation muscle movement voltage potential of facial muscles.Handle the EMG signal that is obtained in step 304, show signal preferably with generation.The EMG signal Processing comprises a plurality of EMG passages of each frame of process facial expression.Then in the one group of calibration expression or the attitude of step 306 generation definition n dimension " expression space ".Can represent calibration expression or attitude with mathematical way by the matrix of creating the matrix product of representing calibration poses and EMG channel value.In order to produce each calibration expression, the performing artist makes the muscle groups action of some isolation so that the data that calibration is collected.For example, the performing artist moves up eyebrow, and measures the signal of the EMG electrode of auto correlation.Calibration factor can be applied to detected signal, move so that realize the correspondence of digital model.Position that can also mobile a little EMG electrode is so that detect stronger signal or avoid and the crosstalking of adjacent EMG sensor.Then to the process of each other muscle groups duplication similarity.
In step 308, will be from the EMG signal of subsequent performances and calibration expression match to generate the weighting coefficient of each expression component.Weighting coefficient (x) asks x to generate by separating expression formula F (x)=B, and wherein F is that group calibration expression that generates in step 306, and B is a plurality of EMG passages at each frame of step 304 processing.In step 310,, generate facial simulation 3D computer graphics model for each facial expression component.The 3D computer graphics model represents to use not the same facial expression of parameter on the same group.For example, can use geometric coordinate to represent facial expression as parameter.And for example, can use muscle strength to represent facial expression as parameter.When in step 312 during with the combination of weighted components and 3D computer graphics model, the result is the facial expression geometric figure with the facial expression geometric figure coupling of original performance.The final geometirc graphical presentation of facial expression be dispose one group of parameter and weighted components the 3D computer graphics model and, and it is played up animation frame in step 314.
Fig. 4 be more detailed icon step 304 being used to of describing handled the block diagram of the circuit of EMG signal as mentioned.In step 402, obtain the EMG signal, use for example described above substantially places the electrode on the facial muscles.The simulation EMG conversion of signals that will obtain by A/D converter 404 becomes digital signal.Hi-pass filter 406 is with the EMG signal filtering, and rectifier 408 is with the EMG signal rectification.From the EMG signal of rectification, deduct noise by noise reduction circuit 410 then.Quadrature by the signal behind 412 pairs of noise reductions of integrator, and it is carried out low-pass filtering by low-pass filter 414.At last, handle the EMG channel value by processor 416.
Fig. 5 be more detailed icon step 306 being used to of describing generates the process flow diagram of the process of calibration expression as mentioned.Obtaining of the EMG signal of step 502 execution calibration poses.Handle the EMG signal that is obtained in step 504.Then in the one group of calibration expression or the attitude of step 506 generation definition n dimension " expression space ".As mentioned above, represent calibration expression or attitude by the matrix of creating the matrix product of representing calibration poses and EMG channel value with mathematical way.
With reference now to Fig. 8,, wherein is illustrated in the actual performer that is pasted with a plurality of EMG and EOG electrode on its face.Utilization is attached to the electrode of performer, the performance that the performing artist can carry out desired.Receive and handle the electric signal of self-electrode by electrode interface 112, and be sent to motion tracking processor 108 to store and to be processed into animation, as shown in Figure 9.
Bundle of electrode conductors can be bundled so that the performing artist moves easily.Electrode interface 112 can with multiple signal multiplexing to the single conductor to be sent to motion tracking processor 108.Can maybe can utilize wireless connections further to strengthen the degree of freedom that the performing artist moves by electric conductor or conductor fiber with the electrode interface 112 motion tracking processor 108 of tying.Perhaps, electrode interface 112 can comprise memory storage, for example hard disk or flash memory module, and this makes the performance data can be stored in this locality, afterwards it is downloaded to motion tracking processor 108 then.In an embodiment of the present invention, electrode interface 112 can be carried on performing artist's behind, and wherein bundle of electrode conductors is directed to the back of performing artist's head, along vertebra down to electrode interface 112.In another alternate ways, electrode interface 112 can be carried in the cap that the performing artist wears, and this makes head and easier the moving of neck and can not be subjected to the interference of bundle of electrode conductors.Should be realized that electrode and conductor can be integrated into the flexible face shield that forms with performer identically.
The motion tracking processor will use technique known to produce animated character's digital model.Digital model can be based on a plurality of digital photographies of taking performers from a plurality of different angles, and they are combined to produce three-dimensional (3D) image file or digital model.As mentioned above, the Software tool that is used to produce the 3D rendering file all is road known in the art.For example, can generate the 3D face structure, the part on the surface of each spline surface definition face structure from a plurality of spline surfaces.Spline surface is formed by each line segment, and these line segments are commonly referred to as " wire frame ".This face structure represents to be positioned at facial (sub-facie) structure of son under the skin surface.As the part of subsequent animation process, the outer surface organizations that will have expectation texture and a color in this sub-face structure to produce the animated character.Should be appreciated that the shape of face structure will be controlled the facial expression that the animated character forms.
In case the establishment digital model, then virtual facial muscles structure covers on the digital model of this face structure.Human face's myoarchitecture is able to good note in medical literature, can be easily with the position of each muscle with interconnect and be mapped to digital model.In digital model by compressing selected muscle vector or muscle set of vectors to realize reorientating and thus the moulding again muscle of realizing in the surface of digital model being activated of a plurality of corresponding vectors.To capture the respective muscle vector that facial and eye movement data is mapped to digital model from the performing artist then.As a result of, the animation digital model will present face and the eyes identical with the performance that captures and move.Should be realized that the animated face that performing artist's action and expression can be mapped to any other type, for example the one-tenth human face of children, animal or different appearance.
As mentioned above, aforesaid motion tracking system can be realized the close coordination of facial muscles and eye motion.For example, with blink detection be the spike of EMG data of the muscle of around eyes.When detecting spike, animation process can be inserted nictation with corresponding with data.This will increase the degree true to nature of the animation that is produced greatly.Above-mentioned motion tracking system has the advantage that is better than the traditional facial motion tracker, because enforcement of the present invention need not video camera or motion-captured in a large number.The camera lens that this means the face that in time will follow the tracks of is blocked, and enforcement will be carried out.And enforcement of the present invention is automatic with the motion-captured muscle activity that is parameterized into, and irrelevant with final geometric figure, helps facial motion capture to reorientating thus.
Though aforementioned content only solves the seizure that facial muscles move, should be realized that identical technology can catch body muscle and move.Can adopt the mode identical that electrode pair is attached to a plurality of different body muscle groups with above-mentioned facial muscles.Similarly, can will be provided with the EMG electrode in the suit that form with health, to improve easiness and the accuracy that electrode paste is attached to performing artist's health identically.The body muscle capture of this form can be to using the conventional motion-captured further enhancing of optical system, because can also catch the muscle suppleness except the rotation of abutment.Obtain the animation that more lively personage moves like this.
Should be appreciated that above-mentioned eye motion process can also be with using optical data or catching the conventional motion capture system of moving and use.Then can be health and/or facial movement that eye motion and optical mode capture is synchronous by above-described same way as.And the EMG data capturing system that uses in conjunction with conventional optical motion capture system provides the enhancing data of relevant performance, and uses can be blocked in the optical interference during optical data is because of performance the time.
Described the preferred embodiment of the system and method for the face that can under the situation of not using video camera, catch the performing artist and eye motion thus, it should be apparent to those skilled in the art that and realized some advantage of the present invention.It should further be appreciated that and in scope and spirit of the present invention, can implement multiple modification, adaptive and alternative.The present invention further is defined by the following claims.
Claims (16)
1. motion tracking animation system comprises:
A plurality of electromyograms (EMG) electrode pair is suitable for being attached at performing artist's skin surface a plurality of positions corresponding with each muscle;
Be coupled to the processor of described a plurality of EMG electrode pairs in operation, described processor comprises the programming instruction that is used to carry out following function:
Obtain the EMG data from described a plurality of EMG electrode pairs, described EMG data comprise with performance during performing artist's muscle move corresponding electric signal;
The digital model of described EMG data to provide muscle to move is provided;
Described digital model is mapped on the animated character, and wherein said animated character will present the muscle identical with the performing artist and move.
2. the system as claimed in claim 1 is characterized in that, described processing capacity comprises that also part generates one group of calibration poses based on handled EMG data.
3. the system as claimed in claim 1 is characterized in that, the described function of obtaining also comprises described electric signal rectification and/or filtering.
4. the system as claimed in claim 1 is characterized in that, described processor also comprises the Computergraphic animation processor.
5. the system as claimed in claim 1 also comprises the electrode interface that is coupled to described a plurality of EMG electrodes, and described electrode interface provides mixed signal to arrive described processor.
6. system as claimed in claim 5 is characterized in that, described electrode interface is suitable for being carried on performing artist's the health.
7. system as claimed in claim 5 is characterized in that, described electrode interface is suitable for being carried on performing artist's the health.
8. system as claimed in claim 5 is characterized in that described electrode interface has the wireless connections with described processor.
9. the system as claimed in claim 1, also comprise a plurality of electroculograms (EOG) electrode pair, be suitable for being attached at performing artist's skin surface and performing artist's eyes position adjacent place, described processor is coupled to described a plurality of EOG electrode pair and comprises the programming instruction of carrying out following function in operation:
Obtain the EOG data from described a plurality of EOG electrode pairs, described EOG data comprise with performance during performing artist's eyes move corresponding electric signal;
Handle described EOG data; And
To the animated character, wherein said animated character will present the eyes identical with the performing artist and move with handled EOG data map.
10. system as claimed in claim 9 is characterized in that, described EOG electrode detection eyes move the electric field change that is produced.
11. one kind is used motion tracking to make cartoon method, comprising:
A plurality of electromyograms (EMG) electrode pair is attached at performing artist's skin surface a plurality of positions corresponding with each muscle;
Obtain the EMG data from described a plurality of EMG electrode pairs, described EMG data comprise with performance during performing artist's muscle move corresponding electric signal;
The digital model of described EMG data to provide muscle to move is provided;
Described digital model is mapped on the animated character, and wherein said animated character will present the muscle identical with the performing artist and move.
12. method as claimed in claim 11 is characterized in that, described treatment step comprises that also part generates one group of calibration poses based on handled EMG data.
13. method as claimed in claim 11 is characterized in that, described obtaining step also comprises described electric signal rectification and/or filtering.
14. method as claimed in claim 11 comprises that also the electrode interface that will be coupled to described a plurality of EMG electrodes in operation is carried on performing artist's the health.
15. method as claimed in claim 11 also comprises the described skin surface and the performing artist's eyes position adjacent place that a plurality of electroculograms (EOG) electrode paste are invested the performing artist.
16. method as claimed in claim 15, also comprise from described a plurality of EOG electrode pairs and obtain the EOG data, described EOG data comprise with performance during performing artist's eyes move corresponding electric signal, handle described EOG data, and with handled EOG data map to the animated character, wherein said animated character will present the eyes identical with the performing artist and move.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US61526804P | 2004-10-01 | 2004-10-01 | |
| US60/615,268 | 2004-10-01 | ||
| US60/616,049 | 2004-10-04 | ||
| US10/984,488 | 2004-11-08 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101069214A true CN101069214A (en) | 2007-11-07 |
Family
ID=38880953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2005800407214A Pending CN101069214A (en) | 2004-10-01 | 2005-09-29 | System and method for tracking facial muscle and eye motion for computer graphics animation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101069214A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901497A (en) * | 2009-05-29 | 2010-12-01 | 影像原动力数码有限公司 | The inverse kinematics of motion-captured characters |
| CN101933804A (en) * | 2009-06-29 | 2011-01-05 | 索尼公司 | Mountable unit for biological signal measurement and biological signal measuring method |
| CN101995947A (en) * | 2009-08-21 | 2011-03-30 | 香港中文大学 | Systems and methods for reproducing body motions via networks |
| CN102349037A (en) * | 2009-03-13 | 2012-02-08 | 微软公司 | Wearable electromyography-based controllers for human-computer interface |
| US8441485B2 (en) | 2008-12-03 | 2013-05-14 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for generating video animation |
| TWI463474B (en) * | 2009-09-18 | 2014-12-01 | Hon Hai Prec Ind Co Ltd | Image adjusting system |
| CN106030661A (en) * | 2014-02-21 | 2016-10-12 | 高通股份有限公司 | View independent 3d scene texturing |
| WO2016165052A1 (en) * | 2015-04-13 | 2016-10-20 | Empire Technology Development Llc | Detecting facial expressions |
| CN108784694A (en) * | 2018-04-23 | 2018-11-13 | 京东方科技集团股份有限公司 | A kind of signal acquisition circuit, signal acquisition method and wearable device |
| WO2021004257A1 (en) | 2019-07-10 | 2021-01-14 | 广州市百果园信息技术有限公司 | Line-of-sight detection method and apparatus, video processing method and apparatus, and device and storage medium |
| CN112657214A (en) * | 2020-12-28 | 2021-04-16 | 苏州源睿尼科技有限公司 | Facial expression capturing mask for performance and installation method thereof |
| CN113274038A (en) * | 2021-04-02 | 2021-08-20 | 上海大学 | Lip sensor device combining myoelectricity and pressure signals |
| CN113855019A (en) * | 2021-08-25 | 2021-12-31 | 杭州回车电子科技有限公司 | Expression recognition method and device based on EOG, EMG and piezoelectric signals |
-
2005
- 2005-09-29 CN CNA2005800407214A patent/CN101069214A/en active Pending
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8441485B2 (en) | 2008-12-03 | 2013-05-14 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for generating video animation |
| CN102349037B (en) * | 2009-03-13 | 2015-08-05 | 微软公司 | Controller is worn based on Electromyographic for man-machine interface |
| CN102349037A (en) * | 2009-03-13 | 2012-02-08 | 微软公司 | Wearable electromyography-based controllers for human-computer interface |
| CN101901497A (en) * | 2009-05-29 | 2010-12-01 | 影像原动力数码有限公司 | The inverse kinematics of motion-captured characters |
| CN101933804B (en) * | 2009-06-29 | 2013-09-04 | 索尼公司 | Mountable unit for biological signal measurement and biological signal measuring method |
| CN101933804A (en) * | 2009-06-29 | 2011-01-05 | 索尼公司 | Mountable unit for biological signal measurement and biological signal measuring method |
| CN101995947B (en) * | 2009-08-21 | 2012-07-04 | 香港中文大学 | Systems and methods for reproducing body motions via networks |
| CN101995947A (en) * | 2009-08-21 | 2011-03-30 | 香港中文大学 | Systems and methods for reproducing body motions via networks |
| TWI463474B (en) * | 2009-09-18 | 2014-12-01 | Hon Hai Prec Ind Co Ltd | Image adjusting system |
| CN106030661A (en) * | 2014-02-21 | 2016-10-12 | 高通股份有限公司 | View independent 3d scene texturing |
| CN106030661B (en) * | 2014-02-21 | 2019-05-21 | 高通股份有限公司 | The independent 3D scene texture background in the visual field |
| WO2016165052A1 (en) * | 2015-04-13 | 2016-10-20 | Empire Technology Development Llc | Detecting facial expressions |
| CN108784694A (en) * | 2018-04-23 | 2018-11-13 | 京东方科技集团股份有限公司 | A kind of signal acquisition circuit, signal acquisition method and wearable device |
| WO2021004257A1 (en) | 2019-07-10 | 2021-01-14 | 广州市百果园信息技术有限公司 | Line-of-sight detection method and apparatus, video processing method and apparatus, and device and storage medium |
| CN112657214A (en) * | 2020-12-28 | 2021-04-16 | 苏州源睿尼科技有限公司 | Facial expression capturing mask for performance and installation method thereof |
| CN113274038A (en) * | 2021-04-02 | 2021-08-20 | 上海大学 | Lip sensor device combining myoelectricity and pressure signals |
| CN113855019A (en) * | 2021-08-25 | 2021-12-31 | 杭州回车电子科技有限公司 | Expression recognition method and device based on EOG, EMG and piezoelectric signals |
| CN113855019B (en) * | 2021-08-25 | 2023-12-29 | 杭州回车电子科技有限公司 | Expression recognition method and device based on EOG (Ethernet over coax), EMG (electro-magnetic resonance imaging) and piezoelectric signals |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101069214A (en) | System and method for tracking facial muscle and eye motion for computer graphics animation | |
| CA2582451C (en) | System and method for tracking facial muscle and eye motion for computer graphics animation | |
| Terzopoulos et al. | Analysis and synthesis of facial image sequences using physical and anatomical models | |
| US10210425B2 (en) | Generating and using a predictive virtual personification | |
| Lou et al. | Realistic facial expression reconstruction for VR HMD users | |
| Terzopoulos et al. | Techniques for realistic facial modeling and animation | |
| CN101310289B (en) | Capturing and processing facial motion data | |
| CN109145788A (en) | Attitude data method for catching and system based on video | |
| de San Roman et al. | Saliency Driven Object recognition in egocentric videos with deep CNN: toward application in assistance to Neuroprostheses | |
| CN110717391A (en) | Height measuring method, system, device and medium based on video image | |
| Waters et al. | The computer synthesis of expressive faces | |
| Jensenius | Methods for studying music-related body motion | |
| US11055892B1 (en) | Systems and methods for generating a skull surface for computer animation | |
| CN106272446A (en) | The method and apparatus of robot motion simulation | |
| Kuruganti et al. | Two-channel enhancement of a multifunction control system | |
| CN111178172A (en) | Laboratory mouse sniffing action recognition method, module and system | |
| Karthik et al. | Automated Home-Based Physiotherapy | |
| US7764283B2 (en) | Eye movement data replacement in motion capture | |
| CN115048954A (en) | Retina-imitating target detection method and device, storage medium and terminal | |
| CN112001236A (en) | Writing behavior identification method and device based on artificial intelligence | |
| Lu et al. | A real-time motion capture framework for synchronized neural decoding | |
| Wibowo et al. | The Use of Motion Capture Technology in 3D Animation | |
| Will et al. | An Optimized Marker Layout for 3D Facial Motion Capture. | |
| CN113552945B (en) | Man-machine interaction glove system | |
| Zhang | Evaluation of Motion Standard Based on Kinect Human Bone and Joint Data Acquisition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20071107 |