WO2006108236A1 - Animation apparatus and method - Google Patents

Animation apparatus and method Download PDF

Info

Publication number
WO2006108236A1
WO2006108236A1 PCT/AU2006/000506 AU2006000506W WO2006108236A1 WO 2006108236 A1 WO2006108236 A1 WO 2006108236A1 AU 2006000506 W AU2006000506 W AU 2006000506W WO 2006108236 A1 WO2006108236 A1 WO 2006108236A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
movement
sounds
display means
language
Prior art date
Application number
PCT/AU2006/000506
Other languages
French (fr)
Inventor
John Noel Bryson
Timothy James Crook
Original Assignee
Bryson Investments Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2005901868A external-priority patent/AU2005901868A0/en
Application filed by Bryson Investments Pty Ltd filed Critical Bryson Investments Pty Ltd
Priority to AU2006235217A priority Critical patent/AU2006235217A1/en
Publication of WO2006108236A1 publication Critical patent/WO2006108236A1/en

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/02Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00Animation
    • G06T13/203D [Three Dimensional] animation
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00Teaching, or communicating with, the blind, deaf or mute
    • G09B21/06Devices for teaching lip-reading
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/06Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
    • G10L21/10Transforming into visible information
    • G10L2021/105Synthesis of the lips movements from speech, e.g. for talking heads

Definitions

  • the present invention relates to animation apparatus and methods. It has particular, although not exclusive, application to apparatus and methods for animating computer-generated images, and images of the kind that might be displayed on electronic displays or in motion picture films.
  • One use to which the invention is particularly suited is in the production of animated images in which movement of the image is associated with the concomitant generation of a pre-determined sound.
  • animated images can be used for - amongst other things - training the hearing or speech impaired to hear or speak more proficiently. Accordingly, the background to the invention will be described in the next section of this specification with specific (although non-limiting) reference to this particular use to which use of the invention is especially suited.
  • controllers such as glove devices that an animator can use in association with a computer, to control the movement of a digital image displayed on a display apparatus.
  • the glove may (for example) be fitted with flexible wires or coils that generate electrical signals when the glove is moved in particular ways, and wherein particular movements of the glove correspond to signals sent to the computer to control the movement of the image on the display in a particular pre-defined manner.
  • Lip synchronization is a process by which movements of the mouth and tongue during speech are determined. This type of process is used by animators in order to replicate in an image to be animated, a sequence of the facial movements required for a given passage of speech in a living person.
  • the known equipment used in lip synchronization are time consuming mechanical and manual processes. They involve attaching a device to the subject in order to record mechanical movement of the mouth during speech. This process is tedious, as it requires each frame to be created and altered individually to ensure that the visual image is correctly synchronized with the sound recording.
  • an animation controller apparatus or method which uses more commonplace computer equipment, so that its use would more readily be accessible to the general public, either for training hearing impaired individuals, or for other uses to which such a system could be put.
  • an apparatus/method could also be used by professional animators, for a variety of purposes.
  • the invention therefore attempts to address these problems, and to provide apparatus and methods for animating an image, where the animation movement of the image can be linked to one or more pre-determined sounds.
  • the invention generally provides an apparatus for controlling the movement of an image displayed on an image display means, where the apparatus comprises a device capable of:
  • the image display means comprises an electrically powered screen, which also comprises means for generating or playing one or more sounds.
  • the screen may be a computer monitor, a television screen or another type of monitor that is capable of displaying analog or digital visual data.
  • the image display means could take the form of a projector which projects visual data onto a surface (such as a canvas screen) capable of receiving and displaying the data to a user of the apparatus and/or one or more other persons.
  • the image is preferably an analog or digital image that is capable of being displayed on the display means, and which is able to be moved by the device.
  • the image could be either: • an actual representation of a human being or an animal (or a part of the body of an actual human being or an animal);
  • the image is a life-like character, such as the face or mouth of a human being.
  • the image could alternatively be a cartoon character, in which case, it could take the form of a cartoon likeness of a human being or an animal or other object.
  • the image would be likely to appeal to users such as children, who might wish or need to use the apparatus to learn sign language or lip reading
  • the device for generating movement of the image comprises a computerised means for controlling movement or animation of the image.
  • the computerised means preferably take the form of a computer system comprising:
  • a computer system would take the form of a personal computer system, so that it could be used in a home or small enterprise or business or educational environment.
  • the computer system could however, take any of a number of alternative forms, depending on the particular use or uses to which it would be put in any given situation.
  • One option is that the computer system would be operable in a networked environment.
  • a computer system of the kind apprehended by the invention could be operated on line or remotely, either via a dedicated connection, or via the Internet.
  • the operating system used in such a computer system structure could take any of a number of forms. When designed for use in the home or in relatively smaller business or educational settings, the operating system would preferably comprise at least one of the following:
  • a network server might use a Linux system, and one or more individual workstations could use Microsoft Windows XP.
  • the application software would comprise animation controllers and/or editors written to operate on at least one of the operating systems mentioned in the preceding paragraphs.
  • animation controllers and/or editors written to operate on at least one of the operating systems mentioned in the preceding paragraphs.
  • such software would enable the user to effect or control:
  • the application software would enable the control of both these functions concomitantly.
  • the application software would preferably be written so as to operate in (or in association or co-operation with) one or more of the operating environments discussed earlier.
  • the application software would comprise one or more computer programs operable in or compatible with a version of the Microsoft Windows operating system.
  • the application software would co-operate with the controller device (and with the remainder of the computer system) so as to enable the generation of a sound co- ordinated with a movement of the image.
  • the sound might be co-ordinated to play at the same time as the movement of the image.
  • the sound might (for example) be played at a given time delay after the movement.
  • the application software would play one or more specified sounds when the image is controlled (via the controller device) to undergo a particular movement.
  • a controller device for use in the invention would take the form of a computer keyboard, mouse, tablet controller, joystick or pen.
  • the controller device could be a voice activated means, operated by using voice recognition software, which would co-operate with a suitable input microphone or headset.
  • voice recognition software would have the advantage that the user of the system would be able to so other things with their hands, such as operating a keyboard (or, for a person learning sign language, to practise sign formations which controlling the displayed image using the voice recognition software).
  • the controller device (of whatever form) would co-operate with the other components of the computer system, so as to enable the user to control movement of the image and the generation of one or more associated sounds.
  • the controller in use, would be capable of generating specified movements of the image, which when actuated, would be accompanied by the generation of one or more specific pre-determined sounds.
  • pressing a particular key on the keyboard would instruct the application software to make the image move in one or more particular ways, and to make one or more specific sound(s) to accompany the particular movement(s) (such as for example, to make the sound of the letter of the keyboard that was pressed by the user).
  • the image could be controlled so as to provide visual and audio reactions, in response to specific instructions from the control device.
  • the image comprises a digital image capable of being displayed on a display in a computer system, where the image takes the form of either an actual representation or a character or cartoon representation of the mouth of a human or an animal subject.
  • the image is able to move and make one or more sounds, in response to signals sent to the computer system (including application software) via a control device.
  • the control device is a computer keyboard.
  • the sound or sounds that are associated with the movement of the image comprise any one of or combinations of two or more of the following:
  • Tonal languages such as Chinese and Japanese are also specifically embraced by the invention.
  • the sounds capable of being generated by particular movements include whole words in a language.
  • the invention can be used to simulate speech, which potentially could assist not only in teaching the hearing impaired to learn sign language or lip reading, but which could also find utility in, for example, the animation of films or other recorded subject matter where images move and make sounds.
  • the device could be used to control a display means which is located physically distant to the device. This could be achieved via a networked computer environment, or via an on-line environment (including the Internet) in which the display means or a computer associated with the display means are located physically distant to the device, but can be controlled remotely via the use of the device.
  • the invention also generally provides a method of controlling the movement of an image displayed on an image display means, the method including the steps of using an apparatus to:
  • the apparatus used to perform the method steps recites above preferably has the attributes recited earlier in the discussion of the preferred features of the apparatus aspect of the invention.
  • F ⁇ g f represents a schematic representation of a computer system suitable for use in accordance with the invention
  • Fig 2 (a) Depicts a representation of a computer generated image of the kind that could be displayed on a computer screen or monitor, in a first ('stationary") position;
  • Fig 2(b) Depicts the image of Fig 2 (a), shown to have moved so that the image is making the facial movements associated with reciting the letter "A" of the English alphabet;
  • Fig 3 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "O" of the English alphabet;
  • Fig 4 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "M" of the English alphabet;
  • Fig 5 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "C" of the English alphabet;
  • Fig 6 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "E" of the English alphabet;
  • Fig 8 Depicts a flow chart which represents schematically, an embodiment of the method aspect of the invention.
  • Fig 1 schematically depicts an apparatus for controlling the movement of an image displayed on an image display means, in accordance with the invention.
  • the apparatus takes the form of a computer system.
  • the computer system schematically depicted in Fig 1 includes: • a central processing unit, which as shown, operates the
  • a display means in the form of a video card associated with the central processing unit, and a computer monitor;
  • application software capable of: o generating an image to be displayed on the monitor; o generating one or more pre-determined movements by the image on the display means; and o generating one or more sounds associated with, or concomitantly with the movement;
  • a sound system in the form of a sound card associated with the central processing unit, and a pair of speakers to play sounds generated by the system, when the application software is operated via a control device in order to move the image and play any sounds associated with particular movements
  • a control device in the form of a computer keyboard, the keyboard being capable of controlling or instructing the application software to generate pre-determined movements of the image on the monitor and to play any sounds associated with particular movements.
  • the method aspect of the invention commences with the creation of a graphic image capable of being displayed on a computer monitor, the image being one which is capable of being altered so as to simulate movement. Persons of ordinary skill in the art would readily understand how this might be achieved.
  • the exemplary image takes the form of a cartoon-like caricature of a mouth.
  • the next step is to record animations of the lips of the mouth caricature, so as to simulate the formation of gestures that would correspond to the formation of certain sounds by a human being. This would be achieved typically by studying actual mouth movements and then manually creating animation sequences using a three- dimensional computer animation software package.
  • Fig 2(a) the mouth caricature is shown to be in a "resting" or “stationary” position.
  • Figs 2(b) to (7) the caricature is shown to have moved in such a way as to form the facial positions associated with a human speaker pronouncing the following letters of the English language alphabet:
  • phoneme that is, a basic unit of sound in a given language.
  • the caricature After the image has formed the facial feature(s) associated with making a particular sound, the caricature would return, as desired, to the default (ie, moth closed) position.
  • the animation process would desirably be as realistic as possible, and would preferably take the form of a sequence of life-like continuous movements between the "starting" position, to the position where the caricature pronounces the desired sound, and then (as desired) to where it returns to the starting position. This would be achieved in methods that would be known to persons of ordinary skill in the art.
  • an animator would record the sequence of facial movements involved with the pronunciation of at least 26 phonemes in the English language (each one corresponding to a given character of the English alphabet).
  • additional movement sequences corresponding to other phonemes, such as diphthongs, for example
  • an animator could create a "library" of motion picture sequences of the movements associated with pronouncing the sounds used in the English language.
  • the invention is not limited to the English language. In other languages, phonemes other than those used in the English language are known. Accordingly, the number of facial movement sequences recorded in any given application will depend on the particular application desired, and the specific language in which the animation sequences are to be used.
  • the caricature is shown as pronouncing single letters of the English alphabet.
  • the invention is not limited to that use. Indeed, in accordance with the invention, visual images of the facial sequences associated with pronouncing whole words or sentences could also be recorded. In this way, true speech could be simulated by the use of the invention.
  • the next step is to create sound files, so that in addition to the image simulating the pronunciation of a sound, the image can also be observed to "say” or "make” the sound. This involves two aspects:
  • the first of these sub-steps could be achieved in any of a number of ways, but in one embodiment, where the sound is intended to be a letter, word or sentence of human speech, then a sound recording of a human being saying the letter, word or sentence could be made. Desirably, this would be recorded in digital form, or in analogue form and then converted to a digital format (where the sound is to be used - as in the present embodiment under discussion - in a computerised environment). Preferably, where a recording of speech is taken from a real human being, the recording is as clear as possible. Alternatively, computer-generated sounds could be used, as desired.
  • the combined video/audio files were saved as stand alone movies that could be stored as computer files.
  • the movies were changed into a format that can be understood by Macromedia Director.
  • a shell for an application software program was then created, to provide a user interface for playing the pre-recorded video/audio animation sequences ("movies") of the various phonemes. As part of this process, the following steps were undertaken:
  • the software program so designed would be distributed in the form of a Microsoft Windows executable file.
  • the applicant's current version of the executable file is named 'k2sync.exe". This program is invoked simply by running the executable program.
  • a user is presented with a graphical display titled "Macromedia Flash Player 6".
  • the animation controller is operating, and so the user can enter key strokes on the keyboard so as to control movement of the animatable image and so as to play the associated sounds.
  • each key stroke is interpreted as a signal to play a specific animated sequence.
  • the screen (or display) is then updated with the letter of the alphabet that has been selected and the animated sequence corresponding to the key pressed is played, so that the character is seen to "speak" the letter.
  • the sequence of steps that the software performs in this process is depicted via the flow chart that appears as Fig 8 of the accompanying drawings.
  • the apparatus and method of the invention By using the apparatus and method of the invention, a user is able more readily than has hitherto been possible, to create animated sequences combining visual images with synchronized, associated sounds.
  • professional animators now have an additional tool to create animations where co-ordination of movements of a visual image with sounds are required.
  • those who are hearing impaired are provided with an additional and potentially powerful tool for learning lip reading and sign language.
  • the use of the invention could also entail the provision of on- screen visual displays of the various hand movements that correspond to any particular sign language sound.
  • the apparatus and method of the invention could be used in association with a sign language text book, and/or as a teaching tool used by sign language instructors to teach hearing impaired persons and/or people with normal hearing to learn sign language and then impart their learning to others.
  • An additional advantage of the invention is - as explained earlier - that it can be operated by using relatively commonplace computer equipment, and in the embodiments described, the equipment used to control the operation of the application software is a computer keyboard, such as would form part of a home computer system. In more sophisticated applications (such as in professional animation) other forms of controllers (eg, voice activation software) could be used in place of, or in addition to a keyboard. As explained earlier, the invention can also be operated on line (such as via the Internet), which demonstrates the many settings in which it could potentially be used.

Abstract

A method and apparatus for controlling the movement of an image displayed on a display means, by generating one or more pre-determined movements by the image on the display means; and generating one or more sounds associated with, or concomitantly with the movement. One use of the method and apparatus is the training of the hearing or speech impaired by relating mouth shapes or sign language gestures to sounds. Another use is in the animation of films.

Description

ANIMATION APPARATUS AND METHOD Field of the Invention The present invention relates to animation apparatus and methods. It has particular, although not exclusive, application to apparatus and methods for animating computer-generated images, and images of the kind that might be displayed on electronic displays or in motion picture films.
One use to which the invention is particularly suited is in the production of animated images in which movement of the image is associated with the concomitant generation of a pre-determined sound. In this application of the invention, such animated images can be used for - amongst other things - training the hearing or speech impaired to hear or speak more proficiently. Accordingly, the background to the invention will be described in the next section of this specification with specific (although non-limiting) reference to this particular use to which use of the invention is especially suited.
Background to the Invention
Human beings communicate in many ways, but arguably the most important communication mechanism is through speech. For communication through speech to be effective, sounds must adequately be heard by the person(s) to whom spoken communications are directed. Sadly, there are many members of the population however, who suffer from hearing impairment. Due to their hearing impairment, such people often also have difficulty in learning to speak. Those people face obvious challenges in communicating with others who are hearing able.
People who suffer from hearing impairment are thus at an obvious disadvantage to the rest of the community, in the conduct of their everyday lives. While devices (such as the so called Cochlear 'Bionic Ear") have been developed over the course of the last few decades for restoring hearing to some hearing impaired persons, such devices are relatively expensive (and thus, on economic grounds alone, not available to all hearing impaired persons), and do not correct all known forms of hearing impairment. Accordingly, other techniques have to be used in order to assist hearing impaired persons to be able to communicate with others.
Traditionally, the approach has been to teach the hearing impaired sign language. While sign language, when practiced proficiently, can be an effective means of addressing hearing impairment, the process of teaching a hearing impaired person to become proficient in its use can be very labour intensive, requiring many hours of personal training by a professional skilled in teaching hearing impaired individuals to learn the technique.
Another approach used is to teach hearing impaired persons to "lip read", that is, to attempt to recognise characteristic facial gestures and/mouth movements that correspond to particular sounds or words. Again however, in order to teach a hearing impaired person to practice this type of technique proficiently, frequently, many hours of personal training are required from a suitably qualified professional. Apart from the significant time required in order to impart the technique so that the user can practise it with at least reasonable competence, there is clearly a cost associated with such training, and the provision of the training itself is often limited to normal work hours. This means that a hearing impaired person often does not have the benefit of being able to practise learning either sign language or lip reading at times when another user of those techniques is not present, thus often extending the time and convenience with which particular levels of competence in their use could be acquired.
It would therefore be desirable to have means by which hearing impaired people could learn sign language and/or lip reading techniques more proficiently.
With the increased use of, and advances in computer and information technologies in the last decade or so, technological options have become available for attempting to address these issues. One possible approach for addressing problems such as those described previously, might be to use computer animated images that would simulate facial or other movements of the kind that would characteristically be associated with a human being making particular sounds as part of everyday speech. If suitable animation techniques could be established as part of a computerised training module, they could - at least to some extent - reduce the dependence of a hearing impaired person on the need for personal tuition by a professional in techniques such as sign language and/or lip reading. To date however, the techniques that have been developed have not provided practically effective apparatus or methods for assisting such persons. For example, known computer-mediated animation mechanisms have utilised controllers such as glove devices that an animator can use in association with a computer, to control the movement of a digital image displayed on a display apparatus. In such apparatus, the glove may (for example) be fitted with flexible wires or coils that generate electrical signals when the glove is moved in particular ways, and wherein particular movements of the glove correspond to signals sent to the computer to control the movement of the image on the display in a particular pre-defined manner.
While such forms of apparatus are known, their use in animating images is at present, of limited practical utility to ordinary members of the public. Presently, the known forms of such apparatus are generally expensive, and somewhat difficult for a lay person to set up and use. Often, an experienced or expert technician is required in order to set up a glove-type controller of this kind. Even when a technician sets up such a glove, using it to control animation typically requires practice, as well as practice in using the software which it is intended to control.
Further, even for professional animators, the currently known equipment is relatively difficult and time consuming to use. Lip synchronization is a process by which movements of the mouth and tongue during speech are determined. This type of process is used by animators in order to replicate in an image to be animated, a sequence of the facial movements required for a given passage of speech in a living person. However, the known equipment used in lip synchronization are time consuming mechanical and manual processes. They involve attaching a device to the subject in order to record mechanical movement of the mouth during speech. This process is tedious, as it requires each frame to be created and altered individually to ensure that the visual image is correctly synchronized with the sound recording.
More recently, computer software has been developed which performs mathematical calculations on the audio waveform recorded from the recital of an animation transcript. While such techniques do increase the speed with which the lip synchronization process can be performed, they are not particularly accurate, but they are expensive.
These limitations mean that for most members of the lay community, and even for some professional animators, presently, the use of known controllers is not a practical proposition.
In contrast, it would be desirable to have an animation controller apparatus or method which uses more commonplace computer equipment, so that its use would more readily be accessible to the general public, either for training hearing impaired individuals, or for other uses to which such a system could be put. Desirably such an apparatus/method could also be used by professional animators, for a variety of purposes.
It is problems such as these which have led to the development of the present invention. The invention therefore attempts to address these problems, and to provide apparatus and methods for animating an image, where the animation movement of the image can be linked to one or more pre-determined sounds.
Summary of the Invention
The invention generally provides an apparatus for controlling the movement of an image displayed on an image display means, where the apparatus comprises a device capable of:
• generating one or more pre-determined movements by the image on the display means; and • generating one or more sounds associated with, or concomitantly with the movement.
Preferably, the image display means comprises an electrically powered screen, which also comprises means for generating or playing one or more sounds. In that case, the screen may be a computer monitor, a television screen or another type of monitor that is capable of displaying analog or digital visual data. Alternatively, the image display means could take the form of a projector which projects visual data onto a surface (such as a canvas screen) capable of receiving and displaying the data to a user of the apparatus and/or one or more other persons.
The image is preferably an analog or digital image that is capable of being displayed on the display means, and which is able to be moved by the device. The image could be either: • an actual representation of a human being or an animal (or a part of the body of an actual human being or an animal);
• a cartoon or caricature representation of a human being or an animal (or a part of the body of an actual human being or an animal); or
• an abstract image.
Preferably, the image is a life-like character, such as the face or mouth of a human being. The image could alternatively be a cartoon character, in which case, it could take the form of a cartoon likeness of a human being or an animal or other object. In this latter embodiment, the image would be likely to appeal to users such as children, who might wish or need to use the apparatus to learn sign language or lip reading
Preferably, the device for generating movement of the image comprises a computerised means for controlling movement or animation of the image. The computerised means preferably take the form of a computer system comprising:
• at least one central processing system;
• an operating system; • a display means for displaying the image;
• a sound system, for playing sounds associated with movement or animation of the image;
• application software to effect movement or animation of the image and the playing of one or more sounds associated with such movement(s); and
• at least one hardware or software means to act as the device for controlling the movement of the image and any associated sounds.
Preferably, a computer system according to the invention would take the form of a personal computer system, so that it could be used in a home or small enterprise or business or educational environment. As persons of ordinary skill in the art would readily appreciate, the computer system could however, take any of a number of alternative forms, depending on the particular use or uses to which it would be put in any given situation. One option is that the computer system would be operable in a networked environment. Alternatively again, a computer system of the kind apprehended by the invention could be operated on line or remotely, either via a dedicated connection, or via the Internet.
The operating system used in such a computer system structure could take any of a number of forms. When designed for use in the home or in relatively smaller business or educational settings, the operating system would preferably comprise at least one of the following:
• MD-DOS;
• Microsoft Windows 3.1 , 95, 98, 2000, Me or XP;
• The Macintosh operating system (Version 5.0 or higher);
• Linux; or • Unix.
In a networked environment, more than one operating system might be used. For example, in a networked environment, a network server might use a Linux system, and one or more individual workstations could use Microsoft Windows XP.
The application software would comprise animation controllers and/or editors written to operate on at least one of the operating systems mentioned in the preceding paragraphs. Preferably, such software would enable the user to effect or control:
• one or more movements of the image; and
• the playing of one or more sounds associated with such movements.
Preferably, the application software would enable the control of both these functions concomitantly. The application software would preferably be written so as to operate in (or in association or co-operation with) one or more of the operating environments discussed earlier. In a preferred embodiment however, the application software would comprise one or more computer programs operable in or compatible with a version of the Microsoft Windows operating system. In a particularly preferred embodiment, the application software would co-operate with the controller device (and with the remainder of the computer system) so as to enable the generation of a sound co- ordinated with a movement of the image. In some embodiments of the invention, the sound might be co-ordinated to play at the same time as the movement of the image. In yet other embodiments, the sound might (for example) be played at a given time delay after the movement.
In an especially preferred embodiment, the application software would play one or more specified sounds when the image is controlled (via the controller device) to undergo a particular movement.
Persons of ordinary skill in the art would readily understand that the device for controlling the movement of the image and any associated sounds could take any of a number of forms. In most applications however, those means would take the form of a peripheral either forming part of, or capable of forming a part of an overall computer system in accordance with the invention. Conveniently, a controller device for use in the invention would take the form of a computer keyboard, mouse, tablet controller, joystick or pen. Alternatively, the controller device could be a voice activated means, operated by using voice recognition software, which would co-operate with a suitable input microphone or headset. The use of a voice recognition software would have the advantage that the user of the system would be able to so other things with their hands, such as operating a keyboard (or, for a person learning sign language, to practise sign formations which controlling the displayed image using the voice recognition software).
The controller device (of whatever form) would co-operate with the other components of the computer system, so as to enable the user to control movement of the image and the generation of one or more associated sounds. In an particularly preferred embodiment of the invention, in use, the controller would be capable of generating specified movements of the image, which when actuated, would be accompanied by the generation of one or more specific pre-determined sounds. Thus, for example, where the controller is a keyboard, pressing a particular key on the keyboard would instruct the application software to make the image move in one or more particular ways, and to make one or more specific sound(s) to accompany the particular movement(s) (such as for example, to make the sound of the letter of the keyboard that was pressed by the user). In this way, the image could be controlled so as to provide visual and audio reactions, in response to specific instructions from the control device.
In one preferred embodiment of the invention, the image comprises a digital image capable of being displayed on a display in a computer system, where the image takes the form of either an actual representation or a character or cartoon representation of the mouth of a human or an animal subject. Preferably further, in this embodiment, the image is able to move and make one or more sounds, in response to signals sent to the computer system (including application software) via a control device. Preferably, in this embodiment, the control device is a computer keyboard. Preferably, the sound or sounds that are associated with the movement of the image comprise any one of or combinations of two or more of the following:
• single letters of the alphabet in a language (including vowels and consonants); • more complex sounds, such as diphthongs and combinations of two or more letters of the alphabet; and
• other sounds, including sounds made by non-human life forms, sounds made by inanimate objects and abstract sounds.
Any language may be used in association with the invention. Tonal languages, such as Chinese and Japanese are also specifically embraced by the invention.
In an especially preferred form of the invention, the sounds capable of being generated by particular movements include whole words in a language. Thus, desirably, the invention can be used to simulate speech, which potentially could assist not only in teaching the hearing impaired to learn sign language or lip reading, but which could also find utility in, for example, the animation of films or other recorded subject matter where images move and make sounds.
In another preferred form of the invention, the device could be used to control a display means which is located physically distant to the device. This could be achieved via a networked computer environment, or via an on-line environment (including the Internet) in which the display means or a computer associated with the display means are located physically distant to the device, but can be controlled remotely via the use of the device.
The invention also generally provides a method of controlling the movement of an image displayed on an image display means, the method including the steps of using an apparatus to:
• generate one or more pre-determined movements by the image on the display means; and
• generate one or more sounds associated with, or concomitantly with the movement. In the method aspect of the invention, the apparatus used to perform the method steps recites above preferably has the attributes recited earlier in the discussion of the preferred features of the apparatus aspect of the invention.
Brief description of the Drawings
A preferred embodiment of the Invention will now be described by way of example only, with reference to the accompanying drawings, in which:
Drawing Number Description
FΪg f Represents a schematic representation of a computer system suitable for use in accordance with the invention;
Fig 2 (a) Depicts a representation of a computer generated image of the kind that could be displayed on a computer screen or monitor, in a first ('stationary") position;
Fig 2(b) Depicts the image of Fig 2 (a), shown to have moved so that the image is making the facial movements associated with reciting the letter "A" of the English alphabet;
Fig 3 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "O" of the English alphabet;
Fig 4 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "M" of the English alphabet;
Fig 5 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "C" of the English alphabet;
"Fig 6 Depicts the image shown to have moved to a position where the image is making the facial movements associated with reciting the letter "E" of the English alphabet;
Fig T JDepicts the imajge_shown_to have moved tqjyDosjtion where the image is making the facial movements associated with reciting the letter "U" of the English alphabet; and
Fig 8 Depicts a flow chart which represents schematically, an embodiment of the method aspect of the invention.
Detailed description of preferred embodiments of the Invention
Referring now to the drawings, Fig 1 schematically depicts an apparatus for controlling the movement of an image displayed on an image display means, in accordance with the invention.
As will be seen in Fig 1 , the apparatus takes the form of a computer system. The computer system schematically depicted in Fig 1 includes: • a central processing unit, which as shown, operates the
Microsoft Windows operating system;
• a display means, in the form of a video card associated with the central processing unit, and a computer monitor;
• a library containing one or more video files, representing the image to be animated, stored either in a memory device associated with the central processing system of the computer or associated with the application software;;
• application software capable of: o generating an image to be displayed on the monitor; o generating one or more pre-determined movements by the image on the display means; and o generating one or more sounds associated with, or concomitantly with the movement;
• a library containing one or more sound files, stored either in a memory device associated with the central processing system of the computer or associated with the application software;
• a sound system, in the form of a sound card associated with the central processing unit, and a pair of speakers to play sounds generated by the system, when the application software is operated via a control device in order to move the image and play any sounds associated with particular movements; and • a control device, in the form of a computer keyboard, the keyboard being capable of controlling or instructing the application software to generate pre-determined movements of the image on the monitor and to play any sounds associated with particular movements.
By way of example, the applicant has found that a computer system having the following specifications would be suitable for performing the invention in the exemplary embodiment presently under discussion:
Processor: Intel Pentium 3, running at 866 MHz
Operating system: Microsoft Windows XP Professional RAM: 786 Mb Video card: ASUS AGP-V7700 Pure/Deluxe V56.55
Sound card: Creative Sound Blaster Live
Software: Alias Maya
Macromedia Director
Macromedia Director Adobe Photoshop
Proprietary software for controlling the movement/ playing of sounds.
In this exemplary embodiment of the invention, the method aspect of the invention commences with the creation of a graphic image capable of being displayed on a computer monitor, the image being one which is capable of being altered so as to simulate movement. Persons of ordinary skill in the art would readily understand how this might be achieved.
In Figs 2(a) to 7 of the accompanying drawings, the exemplary image takes the form of a cartoon-like caricature of a mouth. Conceptually, the next step is to record animations of the lips of the mouth caricature, so as to simulate the formation of gestures that would correspond to the formation of certain sounds by a human being. This would be achieved typically by studying actual mouth movements and then manually creating animation sequences using a three- dimensional computer animation software package.
In Fig 2(a), the mouth caricature is shown to be in a "resting" or "stationary" position. In Figs 2(b) to (7) however, the caricature is shown to have moved in such a way as to form the facial positions associated with a human speaker pronouncing the following letters of the English language alphabet:
• "A"; • "O";
• "M"
• "C";
• "E"; and
• "U".
Each of these letters can be termed a "phoneme", that is, a basic unit of sound in a given language.
Thus, the pronunciation of both vowels and consonants is possible, in accordance with the invention.
After the image has formed the facial feature(s) associated with making a particular sound, the caricature would return, as desired, to the default (ie, moth closed) position. The animation process would desirably be as realistic as possible, and would preferably take the form of a sequence of life-like continuous movements between the "starting" position, to the position where the caricature pronounces the desired sound, and then (as desired) to where it returns to the starting position. This would be achieved in methods that would be known to persons of ordinary skill in the art.
Essentially, as desired, an animator would record the sequence of facial movements involved with the pronunciation of at least 26 phonemes in the English language (each one corresponding to a given character of the English alphabet). Optionally, additional movement sequences (corresponding to other phonemes, such as diphthongs, for example) could also be recorded. In this manner, an animator could create a "library" of motion picture sequences of the movements associated with pronouncing the sounds used in the English language. As persons of ordinary skill in the art would apprehend, the invention is not limited to the English language. In other languages, phonemes other than those used in the English language are known. Accordingly, the number of facial movement sequences recorded in any given application will depend on the particular application desired, and the specific language in which the animation sequences are to be used.
In the accompanying drawings, the caricature is shown as pronouncing single letters of the English alphabet. However, the invention is not limited to that use. Indeed, in accordance with the invention, visual images of the facial sequences associated with pronouncing whole words or sentences could also be recorded. In this way, true speech could be simulated by the use of the invention.
In concept, the next step is to create sound files, so that in addition to the image simulating the pronunciation of a sound, the image can also be observed to "say" or "make" the sound. This involves two aspects:
• Creating a sound file; and
• associating (or "attaching") the sound file to the visual movement sequence, so that the image is seen to be speaking or making the sound.
The first of these sub-steps could be achieved in any of a number of ways, but in one embodiment, where the sound is intended to be a letter, word or sentence of human speech, then a sound recording of a human being saying the letter, word or sentence could be made. Desirably, this would be recorded in digital form, or in analogue form and then converted to a digital format (where the sound is to be used - as in the present embodiment under discussion - in a computerised environment). Preferably, where a recording of speech is taken from a real human being, the recording is as clear as possible. Alternatively, computer-generated sounds could be used, as desired.
In the embodiment presently under discussion, actual human speech phonemes were recorded, and then imported into the Maya software. In the next sub-step, the sound files were attached to the corresponding video files, so that the commencement of the video and audio files was more or less simultaneous.
The preceding sequence of steps might be performed in other sequences.
From here, the combined video/audio files were saved as stand alone movies that could be stored as computer files. The movies were changed into a format that can be understood by Macromedia Director.
A shell for an application software program was then created, to provide a user interface for playing the pre-recorded video/audio animation sequences ("movies") of the various phonemes. As part of this process, the following steps were undertaken:
• creating a work space in Macromedia Director, and defining the size of the program, the basic operating parameters for the program and where the files associated with the program would be stored on the computer system;
• importing the movie files into the program;
• programming the inputs to control the various movies (ie, assigning key strokes to go to a particular frame of the movie, play the scene, then stop the movie and return to start of the movie;
• creating a help file to provide users with basic instructions as to how to operate the program; • programming keys on the keyboard to control the playing of the corresponding phoneme (so that, for example, pressing the letter "A" on the computer keyboard would play the movie for the letter "A");
• creation of a program interface (to establish the layout of the look and feel of the program's user interface);
• assembling the program elements (linking all the different elements of the program so as to give rise to the final program version);
• testing and then running the finalised version of the program.
The software program so designed would be distributed in the form of a Microsoft Windows executable file. The applicant's current version of the executable file is named 'k2sync.exe". This program is invoked simply by running the executable program.
Once the executable program has been executed, a user is presented with a graphical display titled "Macromedia Flash Player 6". At this point, the animation controller is operating, and so the user can enter key strokes on the keyboard so as to control movement of the animatable image and so as to play the associated sounds. In accordance with the preceding description, each key stroke is interpreted as a signal to play a specific animated sequence. The screen (or display) is then updated with the letter of the alphabet that has been selected and the animated sequence corresponding to the key pressed is played, so that the character is seen to "speak" the letter. The sequence of steps that the software performs in this process is depicted via the flow chart that appears as Fig 8 of the accompanying drawings.
By using the apparatus and method of the invention, a user is able more readily than has hitherto been possible, to create animated sequences combining visual images with synchronized, associated sounds. Thus, professional animators now have an additional tool to create animations where co-ordination of movements of a visual image with sounds are required. Similarly, those who are hearing impaired are provided with an additional and potentially powerful tool for learning lip reading and sign language. In the latter application, the use of the invention could also entail the provision of on- screen visual displays of the various hand movements that correspond to any particular sign language sound. Alternatively, the apparatus and method of the invention could be used in association with a sign language text book, and/or as a teaching tool used by sign language instructors to teach hearing impaired persons and/or people with normal hearing to learn sign language and then impart their learning to others.
An additional advantage of the invention is - as explained earlier - that it can be operated by using relatively commonplace computer equipment, and in the embodiments described, the equipment used to control the operation of the application software is a computer keyboard, such as would form part of a home computer system. In more sophisticated applications (such as in professional animation) other forms of controllers (eg, voice activation software) could be used in place of, or in addition to a keyboard. As explained earlier, the invention can also be operated on line (such as via the Internet), which demonstrates the many settings in which it could potentially be used.
Interpretation of this specification
It will therefore be understood that the invention could take many forms and be put to many different uses. All such forms and uses are embodied within the spirit and scope of the invention, which is to be understood as not being limited to the particular constructional details of the embodiments discussed above, but which extends to each novel feature and combination of features disclosed in or evident from this specification and the accompanying claims and drawings. All of these different combinations constitute various alternative aspects of the invention.
It will also be understood that the term "comprises" (or its grammatical variants), as used in this specification, is equivalent in meaning to the term "includes" and should not be taken as excluding the presence of other elements or features. Further, wherever used in this specification, the term "includes" is not a term of limitation, and is not be taken as excluding the presence of other elements or features. It is also to be understood that any discussion in this specification of background or prior art documents, devices, acts, information, knowledge or use ('Background Information') is included solely to explain the context of the invention. Any discussions of such Background Information is not be taken as an admission that any such Background Information constitutes part of the prior art base or the common general knowledge in the field of the invention on or before the priority date of the appended claims or any amended claims later introduced into this specification.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:
1. An apparatus for controlling the movement of an image displayed on a display means comprising a sound playing or generation means, the apparatus comprises a device for: • generating one or more pre-determined movements by the image on the display means; and
• generating one or more sounds associated with, or concomitantly with the movement.
2. An apparatus as claimed in claim 1 in which the display means is an electrically powered screen
3.
• an analogue; or
• a digital screen that is capable of displaying visual data.
4. An apparatus as claimed in claim 3, in which the screen is a:
• a television screen; or
• a computer monitor.
5. An apparatus as claimed in any of the preceding claims, in which the screen is a material which is capable of displaying images projected onto the screen by a projector means.
6. An apparatus as claimed in any of the preceding claims, in which the image comprises:
• analog or
• digital data that is capable of being displayed on the display means, and which is able to be moved on the display means by the use of the device.
7. An apparatus as claimed in any of the preceding claims, in which the image comprises:
• an actual representation of a human being or an animal (or a part of the body of an actual human being or an animal);
• a cartoon or caricature representation of a human being or an animal (or a part of the body of an actual human being or an animal); or • an abstract image.
8. An apparatus as claimed in claim 7, in which the image simulates:
• a real human being or animal; or
• a part of a real human being or animal.
9. An apparatus as claimed in claim 8, in which the image simulates the face or mouth of a human being or an animal.
10. An apparatus as claimed in any of the preceding claims, in which the device for generating movement of the image comprises a computerised means for controlling movement or animation of the image.
11. An apparatus as claimed in claim 10, in which the computerised means comprise a computer system comprising:
• at least one central processing system;
• an operating system;
• a display means for displaying the image; • a sound system, for playing sounds associated with movement or animation of the image; • application software to effect movement or animation of the image and the playing of one or more sounds associated with such movement(s); and
• at least one hardware or software means to act as the device for controlling the movement of the image and any associated sounds.
12. An apparatus as claimed in claim 11 , in which the operating system for the computer system comprises one or more of the following:
• MD-DOS;
• Microsoft Windows 3.1 , 95, 98, 2000, Me or XP; • The Macintosh operating system (Version 5.0 or higher);
• Linux; and/ or
• Unix.
13. An apparatus as claimed in 12, in which the application software comprises animation controllers and/or editors that operate on at least one of the operating systems.
14. An apparatus as claimed in 13, in which the application software enables a user of the software to effect or control:
• one or more movements of the image; and/or
• the playing of one or more sounds associated with such movements.
15. An apparatus as claimed in claim 14 in which the application software plays one or more specified or pre-determined sounds when the image is controlled via the device to undergo a particular movement.
16. An apparatus as claimed in claim 15 in which the device comprises one or more of the following: • computer keyboard;
• computer mouse;
• tablet controller; • joystick;
• pen controller for a computer;
• voice activated control means.
17. An apparatus as claimed in any of the preceding claims, in which the sound or sounds that are associated with the movement of the image comprise any one of, or combinations of two or more of the following:
• single letters of the alphabet in a language;
• diphthongs;
• combinations of two or more letters of the alphabet; • words in a language;
• sentences in a language;
• fluent or quasi-fluent speech in a language;
• sounds made by non-human life forms;
• sounds made by inanimate objects; and/or • abstract sounds.
18. An apparatus as claimed in claim 17, in which the sounds are taken from one or more languages selected from the group comprising:
• tonal languages; and
• non-tonal languages.
19. An apparatus as claimed in claim 18, in which the language is English.
20. An apparatus as claimed in claim 18, in which the language is a tonal language.
21. An apparatus as claimed in claim 20, in which the language comprises one or more of the following: • Chinese; and/or
• Japanese.
22. An apparatus as claimed in any of the preceding claims, in which the apparatus is capable of being used in a networked computer environment.
23. An apparatus as claimed in any of the preceding claims, in which the apparatus is capable of being used in or via an on-line facility.
24. An apparatus as claimed in claim 23, in which the on-line facility comprises the Internet.
25. An apparatus as claimed in any one of claims 22, 23 or 24 in which, in use of the apparatus, the device generates: • movements of the image; and/or
• one or more sounds associated with, or concomitantly with the movement of the image on the display means, where the display means is physically located distant to the device.
26. A method of controlling the movement of an image displayed on an image display means comprising a sound playing or generation means, the method including the steps of using an apparatus as claimed in any of the preceding claims, to:
• generate one or more pre-determined movements by the image on the display means; and
• generate one or more sounds associated with, or concomitantly with the movement.
27. An apparatus as claimed in any of claims 1 to 25, substantially as described in this specification and with reference to the examples given and the accompanying drawings.
28. A method as claimed in claim 26, substantially as described in this specification and with reference to the examples given and the accompanying drawings.
PCT/AU2006/000506 2005-04-14 2006-04-13 Animation apparatus and method WO2006108236A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2006235217A AU2006235217A1 (en) 2005-04-14 2006-04-13 Animation apparatus and method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2005901868A AU2005901868A0 (en) 2005-04-14 Animation apparatus and method
AU2005901868 2005-04-14

Publications (1)

Publication Number Publication Date
WO2006108236A1 true WO2006108236A1 (en) 2006-10-19

Family

ID=37086533

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2006/000506 WO2006108236A1 (en) 2005-04-14 2006-04-13 Animation apparatus and method

Country Status (1)

Country Link
WO (1) WO2006108236A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3022388A1 (en) * 2014-06-16 2015-12-18 Antoine Huet CUSTOM FILM AND VIDEO MOVIE

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4913539A (en) * 1988-04-04 1990-04-03 New York Institute Of Technology Apparatus and method for lip-synching animation
US5286205A (en) * 1992-09-08 1994-02-15 Inouye Ken K Method for teaching spoken English using mouth position characters
WO1996016388A1 (en) * 1994-11-22 1996-05-30 Softrade International, Inc. Foreign language teaching aid method and apparatus
WO1996017323A1 (en) * 1994-11-30 1996-06-06 California Institute Of Technology Method and apparatus for synthesizing realistic animations of a human speaking using a computer
US5630017A (en) * 1991-02-19 1997-05-13 Bright Star Technology, Inc. Advanced tools for speech synchronized animation
WO2000045288A2 (en) * 1999-01-31 2000-08-03 Jacob Fromer Computerized translating apparatus
WO2001045088A1 (en) * 1999-12-16 2001-06-21 Interactive Solutions, Inc. Electronic translator for assisting communications
WO2002058010A2 (en) * 2001-01-22 2002-07-25 Digital Animations Group Plc. Character animation system
US20020140718A1 (en) * 2001-03-29 2002-10-03 Philips Electronics North America Corporation Method of providing sign language animation to a monitor and process therefor
JP2003058908A (en) * 2001-08-10 2003-02-28 Minolta Co Ltd Method and device for controlling face image, computer program and recording medium
JP2003296753A (en) * 2002-04-04 2003-10-17 Computer Image Laboratory Co Ltd Interactive system for hearing-impaired person
US6662161B1 (en) * 1997-11-07 2003-12-09 At&T Corp. Coarticulation method for audio-visual text-to-speech synthesis
US20040120554A1 (en) * 2002-12-21 2004-06-24 Lin Stephen Ssu-Te System and method for real time lip synchronization
US6796798B1 (en) * 2002-03-27 2004-09-28 University Of South Florida Dynamic reading instruction
JP2004354620A (en) * 2003-05-28 2004-12-16 Nakajima Tsushinki Kogyo Kk English and sign language training software program, and recording medium with the software program recorded thereon

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4913539A (en) * 1988-04-04 1990-04-03 New York Institute Of Technology Apparatus and method for lip-synching animation
US5630017A (en) * 1991-02-19 1997-05-13 Bright Star Technology, Inc. Advanced tools for speech synchronized animation
US5286205A (en) * 1992-09-08 1994-02-15 Inouye Ken K Method for teaching spoken English using mouth position characters
WO1996016388A1 (en) * 1994-11-22 1996-05-30 Softrade International, Inc. Foreign language teaching aid method and apparatus
WO1996017323A1 (en) * 1994-11-30 1996-06-06 California Institute Of Technology Method and apparatus for synthesizing realistic animations of a human speaking using a computer
US6662161B1 (en) * 1997-11-07 2003-12-09 At&T Corp. Coarticulation method for audio-visual text-to-speech synthesis
WO2000045288A2 (en) * 1999-01-31 2000-08-03 Jacob Fromer Computerized translating apparatus
WO2001045088A1 (en) * 1999-12-16 2001-06-21 Interactive Solutions, Inc. Electronic translator for assisting communications
WO2002058010A2 (en) * 2001-01-22 2002-07-25 Digital Animations Group Plc. Character animation system
US20020140718A1 (en) * 2001-03-29 2002-10-03 Philips Electronics North America Corporation Method of providing sign language animation to a monitor and process therefor
JP2003058908A (en) * 2001-08-10 2003-02-28 Minolta Co Ltd Method and device for controlling face image, computer program and recording medium
US6796798B1 (en) * 2002-03-27 2004-09-28 University Of South Florida Dynamic reading instruction
JP2003296753A (en) * 2002-04-04 2003-10-17 Computer Image Laboratory Co Ltd Interactive system for hearing-impaired person
US20040120554A1 (en) * 2002-12-21 2004-06-24 Lin Stephen Ssu-Te System and method for real time lip synchronization
JP2004354620A (en) * 2003-05-28 2004-12-16 Nakajima Tsushinki Kogyo Kk English and sign language training software program, and recording medium with the software program recorded thereon

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DATABASE WPI Week 200328, Derwent World Patents Index; Class P86, AN 2003-283628 *
DATABASE WPI Week 200374, Derwent World Patents Index; Class P85, AN 2003-784523 *
DATABASE WPI Week 200504, Derwent World Patents Index; Class P85, AN 2005-035204 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3022388A1 (en) * 2014-06-16 2015-12-18 Antoine Huet CUSTOM FILM AND VIDEO MOVIE
WO2015193605A1 (en) * 2014-06-16 2015-12-23 Huet Antoine Tutorial model comprising an assistance template
KR20170018907A (en) * 2014-06-16 2017-02-20 앙투안 후에트 Tutorial model comprising an assistance template
CN106663381A (en) * 2014-06-16 2017-05-10 安托万·休伊特 Tutorial model comprising an assistance template
JP2017521703A (en) * 2014-06-16 2017-08-03 ユエ,アントワーズ Tutorial model with assistance template
US10424220B2 (en) 2014-06-16 2019-09-24 Antoine HUET Tutorial model comprising an assistance template
RU2761316C2 (en) * 2014-06-16 2021-12-07 Антуан Ют Mobile platform for creating personalized movie or series of images
KR102430890B1 (en) 2014-06-16 2022-08-09 앙투안 후에트 Tutorial model comprising an assistance template

Similar Documents

Publication Publication Date Title
US11145100B2 (en) Method and system for implementing three-dimensional facial modeling and visual speech synthesis
JP3016864B2 (en) Audio-visual work with writing, method of meaningfully combining oral pronunciation and writing in audio-visual work, and apparatus for linear and conversational applications
US5613056A (en) Advanced tools for speech synchronized animation
US6062863A (en) Method of associating oral utterances meaningfully with word symbols seriatim in an audio-visual work and apparatus for linear and interactive application
CA2287315A1 (en) Method and system for making an audio-visual work with a series of visual word symbols coordinated with oral word utterances and such audio-visual work
CN112673400A (en) Avatar animation
CN113253836A (en) Teaching method and system based on artificial intelligence and virtual reality
US20050255430A1 (en) Speech instruction method and apparatus
Luna et al. Words worth learning-augmented literacy content for ADHD students
US10825357B2 (en) Systems and methods for variably paced real time translation between the written and spoken forms of a word
Cui et al. Animation stimuli system for research on instructor gestures in education
WO2006108236A1 (en) Animation apparatus and method
Govindasamy Animated Pedagogical Agent: A Review of Agent Technology Software in Electronic Learning Environment
JP6892478B2 (en) Content control systems, content control methods, and content control programs
AU2006235217A1 (en) Animation apparatus and method
Wolfe et al. A survey of facial nonmanual signals portrayed by avatar
JP2021009351A (en) Content control system, content control method, and content control program
Luerssen et al. Head x: Customizable audiovisual synthesis for a multi-purpose virtual head
Granström et al. Speech and gestures for talking faces in conversational dialogue systems
Kunc et al. ECAF: Authoring language for embodied conversational agents
Xiao et al. Computer Animation for EFL Learning Environments.
JP7347725B1 (en) Display program, display method and display system
US11581006B2 (en) Systems and methods for variably paced real-time translation between the written and spoken forms of a word
WO2024029135A1 (en) Display rpogram, display method, and display system
Shaw et al. Evaluation of the prototype mobile phone app Pugh: a 3D cartoon character designed to help deaf children to speech read

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

NENP Non-entry into the national phase

Ref country code: RU

WWE Wipo information: entry into national phase

Ref document number: 2006235217

Country of ref document: AU

WWW Wipo information: withdrawn in national office

Country of ref document: RU

WWP Wipo information: published in national office

Ref document number: 2006235217

Country of ref document: AU

122 Ep: pct application non-entry in european phase

Ref document number: 06752599

Country of ref document: EP

Kind code of ref document: A1