WO2001039180A1

WO2001039180A1 - System and method of templating specific human voices

Info

Publication number: WO2001039180A1
Application number: PCT/US2000/032328
Authority: WO
Inventors: Steven J. Keough; Katherine Axia Keough
Original assignee: Keough Steven J; Katherine Axia Keough
Priority date: 1999-11-23
Filing date: 2000-11-23
Publication date: 2001-05-31
Also published as: IL149813A0; ZA200204036B; BR0015773A; KR20020060975A; EA200200587A1; AU2048001A; AP2002002524A0; EP1252620A1; NO20022406L; NO20022406D0; JP2003515768A; CA2392436A1; CN1391690A; EA004079B1

Abstract

Systems and methods are disclosed to capture (103) an enabling portion of a voice and then create a voice template (127) or profile signal which may be combined at a later time with noise of another origin to reconstitute the original voice. Such reconstituted voice may then be used to speak any form or content provided via digital input thereto, and to say content which was not spoken in an original form by the original voice. Products and processes for online use are disclosed, as are certain business methods and industry applications.

Description

SYSTEM AND METHOD OF TEMPLATING SPECIFIC HUMAN VOICES

Field of the Invention Systems, methods, and products for preserving and adapting sound, and more specifically human voices.

Background of the Invention

Since the beginning of time mammals and other creatures have communicated in some form by voice or similar noises. Indeed, such noises are normally quite distinct m view of the differences in morphology of creatures- even within species. The distmctiveness of creatures mcludes the very distinct elements of speech patterns and tones. Unfortunately, the joy of listening to the speech of others with a voice of particular interest is lost when that person dies or ceases contact with the listener. Only the very basic forms of media capture exist today by which voices may be preserved.

For example, a tape or digital recording device is used to record someone's voice and thereby retain it for future listening and replay as it was recorded originally, or portions of the original recording may be played as desired. These devices and methods of voice recording also include a range of artificial voices, created by computers, which may be used for many different functions, including for example telephone automatic assistance and verification, very basic speech between toys or equipment and users, synthesized voices for the film and entertainment industry, and the like. In some applications, these artificial voices are preprogrammed to a narrow set of responses according to a specific input. Although more responsive, m some instances, than a mere recording of an actual voice, these artificial voice sounds are nevertheless simple compared to the robust voice capabilities of the present invention. Indeed, in certain embodiments of the invention there are elements that are either quite different from such systems or which take the previous technology far beyond that ever contemplated or even suggested by such pπor discoveπes or innovations.

Many publications worldwide disclose aspects of artificial vocalization. In similar fashion, some references disclose systems and techniques of using and creating artificial voice sounds. However, none of these references disclose the concepts of the present invention.

Brief Description of the Drawings

Figure 1 is a flow diagram of one embodiment of the system operation of the invention. Figure 2 is a schematic diagram of one embodiment of a voice capture subsystem. Figure 3 is a schematic diagram of one embodiment of a voice analysis subsystem. Figure 4 is a schematic diagram of one embodiment of a voice characterization subsystem.

Figure 5 is a schematic diagram of one embodiment of a voice template subsystem.

Figure 6 is a schematic diagram of one embodiment of a voice template signal bundler subsystem. Figure 7 is one embodiment of a schematic diagram of the system of the invention used with remote information download and upload options.

Figure 8 is one embodiment of an exemplary plan view of an embodiment of the invention embodied m a mobile, compact component.

Figure 9 is an exemplary plan view of an embodiment of the invention used with a visual media source.

Summary of the Invention

Systems and methods are provided for recording or otherwise capturing an enabling amount of a specific person's voice to form a voice pattern template. That template is then useful as a tool for building new speech sounding like that precise voice, using the template, with the new speech probably never having been actually said or never having been said in the precise context or sentences by the specific human but actually sounding identical in all aspects to that specific human's actual speech. The enabling portion is designed to capture the elements of the actual voice necessary to reconstruct the actual voice, however a confidence rating is available to predict the limits of the re- constructed or re-created speech in the event there is not enough enabling speech to start with. A new voice or voices may be used with a database of subject matter, histoπcal data, and adaptive or artificial intelligence modules to enable new discussions with the user just as if the templated voice's originator were present. This system and method may be combined with other media, such as a software file, a chip embedded tool, or other forms. Interactive use of this system and method may occur in various manners. A unit module itself may comprise the entirety of an embodiment this invention, e.g. a chip or electronic board which is configured to capture and enable use of a voice in the manner disclosed herein.

The template is useful, for example, as a tool for capturing and creating new dialogs with people whom are no longer immediately available, who may be deceased, or even those who consent to having the voices templated and used in this manner. Another example is the application to media, such as film or photos or other depictions of the actual voιce(s) originator to create on-demand virtual dialog with the originator. Various other uses and applications are contemplated withm the scope of the invention.

Detailed Description of the Invention Voice is a sound of extraordinary power among mammals. The sound of a mother's voice is recognized by and soothes a child even before birth, and the sound of a grandfather's voice calms the fears of even a grown person. Other voices may inspire complete strangers or may elicit memoπes from loved ones of long past events and moments. These are but a few examples of the great gift of distinctiveness that the human and other species have; and their ability to influence others (and themselves) by the very unique sound of each creatures' voice. In humans, for example, this particularity of one's voice derives from the genetic contribution of the parents resulting in the shape, size and position of the vaπous human body components that influence the way one sounds when speaking or otherwise communicating with voice or through the mouth and nasal passages. Other influences exist as well It is understandable therefor that there is a range of differences among people, often even withm the same family. Indeed, even the same person may sound slightly different accordmg to temporal influences such as the health, stress level, emotional state, fatigue, the ambient temperature around the person, or other factors.

There is general agreement worldwide, however, that a person's voice qualities present a very unique combination, that is discernible to those who have heard the voice before. The ability of humans to associate through their senses is remarkable, particularly as such sensing relates to identification and association with the human voice. Life's grand and small events are often recalled many years or decades later by the nature of comments made or tones remembered. Thus is the enduring strength and emotive power of voice.

It is of course well known to capture and play back human voice on vaπous media and machines. Basic manipulation of recorded human voice has been done for many decades, both intentionally and unintentionally, m tape and digital media. However, this manipulation has been generally limited by the bounds of what has actually been stated by the human rather than what could be stated by that human. For example, segments of actual statements by the human have been played, edited, mixed and re-played, sometimes even at different speeds. Other examples of human voice use include playback of intentionally distorted voice segments, such as may be used in cartoons or other audio related to animation or certain music. Of course, the animation medium also has used artificial voice not necessaπly created using actual voice. One example of this is a computer generated "voice" operator used by some telephone and communication systems. One method of synthesizing voices and sounds is referred to as concatenative, and refers to the recordings of wave form data samples or real human speech. The method then breaks down the pre-recorded oπgmal human speech into segments and generates speech utterances by linking these human speech segments to build syllables, words, or phrases. The size of these segments vanes. Another method of human speech synthesis is known as parametric. In this method, mathematical models are used to recreate a desired speech sound. For each desired sound, a mathematical model or function is used to generate that sound. As such, the parametric method is generally without human sound as an element. Finally, there are generally a few well-known types of parametric speech synthesizers. One is known as an articulatory synthesizer, which mathemetically models the physical aspects of the human lungs, larynx, and vocal and nasal tracts. The other type of parametπc speech synthesizer is known as a formant synthesizer, which mathematically models the acoustic aspects of the human vocal tract. Other systems include means for recognizing a specific voice, once the using system has been trained m that voice. Examples of this include the vaπous speech recognition systems useful in the field of capturing spoken language and then translating those sounds into text, such as with systems for dictation and the like. Other speech related systems concern the field of biometncs. and use of certain spoken words as secuπty codes or ciphers. None of these systems, methods, means or other forms of disclosure recognize the vaπous inventions disclosed herein, nor do any such disclosures even recognize a need for such technical innovations What has long been needed is a system and method for preserving the voices of other beings in a dynamic and adaptive manner for future use and benefit by the originator or by others. What has been further needed are systems and methods for accomplishing and utilizing such voice capture or profiling in manners which present a seamless, articulate, or otherwise genuine vocalization or voice in the voice of the oπginal person in ways possibly never contemplated by that person. Certain additional advantages accrue to systems and methods for accomplishing this which are easily used by all people of virtually any skill, culture or language. What has been further needed is a new business method, technique and model, along with implementing apparatus and other means, to create and to facilitate access to specific voice templates and then facilitate use of those voice templates for personal needs or desires, whether related to business or pleasure. Once again, although much has been accomplished in the field of voice technology, none of these past efforts contemplate the instant inventions and merely highlight the novel and heretofore unrecognized need for these inventions. Figure 1 is a schematic diagram of one embodiment of a system 10 for capturing an enabling portion of a specific voice sufficient for using that portion as a template in further use of the voice characteπstics. System 10 may be part of a handheld device, such as an electronic handheld device, or it may be part of a computing device of the size of a laptop, a notebook, or a desktop, or system 10 may be pan of merely a circuit board within another device, or an electronics component or element designed for temporary or permanent placement m or use with another electronic element, circuit, or system, or system 10 may, in whole or in part, comprise computer readable code or merely a logic or functional circuit in a neural system, or system 10 may be formed as some other device or product such as a distributed network-style system. In one embodiment, system 10 compπses input or capture means 15 for captuπng or receiving a portion of a voice for processing and construction of a voice algoπthm or template means 19, which may be formed as a stream of data, a data package, a telecommunications signal, software code means for defining and re-generatmg a specific voice, or a plurality of voice characteπstics organized for application to or template on another organization of sound or noise suitable to arrange the sound or noise as an apparent voice of an oπgmator's voice. Other means of formatting computer readable program code means, or other means, for causing use of certain identified voice characteπstics data to artificially generate a voice is also contemplated withm this invention. The logic or rules of the algoπthm or template means 19 are preferably formed with a minimum of voice input, however vaπous amounts of voice and other data may be desired to form an acceptable data set for a particular voice

In one embodiment of the invention, it is desired to capture an enabling portion of a human voice, for example, with a small amount of analog or digital recording, or real-time live input, of the person's voice that is to be templated. Indeed, a prescπbed grouping of words may be formed to optimize data capture of the most relevant voice characteπstics of the person to enable accurate replication of the voice. Analysis means are contemplated for most efficiently determining what form of enabling portion is best for a particular person Whether by a single data input or a seπes of inputs, the voice data is captured and stored in at least one portion of storage means 22. Analysis of the voice data is performed at processor means 25, to identify characteπstics useful in creating a template of that specific user's voice. It is recognized that the voice data may be routed directly to the processor means and need not necessaπly go initially to the storage means 22. Further exemplary discussion of the interaction among the processor means, storage means, and the template means is found below, and m relation to Figures 2-8 After adequate voice data has been analyzed, then a template of the voice is, m one embodiment, stored until called for by the processor means 25. For example, after voice AA has had an enabling portion captured, analyzed and templated (now referred to as AA_t) it is stored in a storage means 22 (which may be either resident near the other components or located in a remote or distributed mode at one or more locations) until a demand request occurs. One example of a demand request is a user of system 10 submitting a request via representative input means 29 to utilize the voice AA template AA_t m a newly created conversation with voice AA participating as a generated voice rather than an actual, live use of voice AA. This may occur in conjunction with or utilization of one or more vaπous databases, a few of which are represented by situational database 33 or personal database 36. In turn, then voice AA template AA, is called and provided as a forming mechanism with certain other noise to create a new conversational voice AA¹ that sounds precisely like the oπginal voice AA of the oπgmally inputted data, once formed. Although the new voice AA¹ sounds like oπgmal voice AA in all respects, it is actually an artificially created voice with the template AA_t providing the matching key, such as a genetic code, to voice AA. In this way an enabling portion of an actual voice may encode the system 10 using a template to allow regeneration and unlimited utilization of the captured voice m virtually any way desired by the user. This is not simply a synthesis of pπor utterances of bits of voice AA which are electronically fused together, by either concatenation or formant techniques, but rather an entirely new voice that is designed, manufactured and assembled or constructed using the voice data characteπstics of voice AA (i.e., the voice template or profile), and possibly other characteπstics relevant to the ongmator of voice AA. It is recognized, of course, that the implications of this technology are vast, and safeguards will be necessary to maintain the proper use of this templated voice technology. Indeed, this technology may require further use of authoπzation means to only allow authoπzed users to access and use the voice template technology and data. An additional necessity may be to have means for veπfying that voices heard are either real or templated, m order to ensure against fraudulent or unauthoπzed use of such created voices. Legal mechanisms may need to be created to recognize this realm of technology, in addition to the licensing, contract, and other mechanisms already m existence in most countries.

In Figure 1, connection means 41 represents pathways for energy or data flow which may be actual leads, light channels, or other electronic, biologic or other activatable paths among system components. In one embodiment power means 44 is shown withm system 10, but may also be remote if desired. In another embodiment of system 10, the algoπthm, signal, code means or template which is created in whole or m part may be returned for storage or refinement withm either storage means 22, template means 19, or other system component or architecture. This capability permits and facilitates improvement or adaptation of the specific voice template according to the instructions of the creator or another user. This could be accomplished, for example, if multiple data sets of the same person's voice could be inputted over time, or if different ages, development, or other changes to physiology or temperament of the oπgmator of the voice occur. Indeed, it is possible to tram the templated voice to recall the context of previous engagements and to include such knowledge m future operations. In these instances it may be useful to select a refinement mode to retrieve voice AA¹ template (AA¹,) and refine the voice or template with a compaπson and update using the analysis means 22 or input means 29. Yet another example includes location of a person with a voice BB that compπses one or more voice characteπstics that are similar to voice AA which was the originator for voice template AA¹,. In this case it may be useful to input the one or more similar characteπstics from voice BB as either limited or general refinement inputs to voice AA¹ or voice template AA¹,. It is then possible to also retain voice BB and create a voice BB¹ and voice template BB¹, , either of which may be useful at a future date. Another example includes creation of a database of variously refined voices for a single oπginator of the voice, useful on demand or as appropriate by system or user, according to the situation that is presented. In yet another example, a service may be offered to voice match and provide suitable refinement tools, such as natural or artificially generated waveforms or other acoustic or signal elements, to refine voice templates according to the user's desires. Prior to descπbing further embodiments of system 10 or related systems and methods, it is useful to examine possible applications of this technology. In general, there are applications so numerous as to be difficult to list them all. However, it is contemplated that any use of a voice-like noise, which is generated by data provided to and data resulting from a template or coding tool for creation of that voice-like noise, is captured withm the scope of this invention, particularly when such coding tool is used with other noise or sound generating means, if needed, to re-create a voice sound that is virtually identical to the oπgmator's actual voice. The use of the generated voice in completely new sentences, or other language structures, is also within the scope of this invention. The ability to provide machine, component, or computer readable code means as part of the signal forming or transmitting of the voice template process or product further facilitates use of this technology. Means to tie or activate use of this voice templatmg and voice generating technology to streaming or other forms of data allows for virtual dialog, which may be adaptive and intelligent, as well as merely informational or reactive, and with such dialog or conversations being with voices selected by the user. It is also recognized that the technology herein disclosed may be utilized with visual images as well as aural sounds.

Moreover, it is believed that a voice template as descπbed herein may be created using data that does not include an actual enabling portion of an oπginator's voice, but that the enabling portion of the oπginator's voice may be used, possibly with other data, to validate the replication accuracy of the originator's voice. In this manner, it is possible to either use an enabling portion of a voice m either the templatmg of the voice or merely in the validation of the accuracy of an otherwise templated voice. A templated or replicated voice may be used to interact with or prompt users of computers or other machines and systems. The user may select such templated voice from either her own library of templated voices, another source of templated voices, or she may simply create a new voice. For example, templated voice AA¹ may be selected by the user for voicemail prompts or reading of texts, or other communication interface, whereas templated voice CC may be selected for use in relation to an interactive entertainment use. Troubleshooting or problems lurking in the user's machine, or alerting signals to a user of a device, may be identified or resolved by the user while working with templated voice DD. These are simply examples of how this technology will enable improved user interface and association by the user with functions, tasks, modes or other features by use of templated voice technology. Template selection and use, and generated voice creation and use may be accomplished either within the user's machine or device, partially withm the user's machine or device, or external of the user's machine or device There may be instances of only temporal use of one or more devices, such as in a hotel room, a visiting office, or other transient scenaπo or with a temporary device use, but which nevertheless provides the above features m the above-vaπed manner. For example, a traveler may wish to carry or access certain voices for accompaniment of the traveler on aircraft, or m hotel rooms. The invention may be useful in hospital or hospice rooms, or other locations. These uses are possible with one or more of the embodiments herein. Interestingly, this system may also be used by some individuals on their own voice and given as a legacy to others. Many other uses are withm the scope of the teachings herein.

Other uses of the inventions disclosed herein include education, such as teaching children and others about historical events using a templated voice of choice. For example, if a parent desired her child to learn about race relations in the United States in the decade of the 1960s using one of the child's deceased grandparent's voices, then the templated voice of the selected grandparent (if available) would be designed, manufactured and designated for use. System 10 would access one or more databases to harvest information and knowledge about the designated topic and provide that information to one or more databases withm system 10. such as situational database 33 for use as needed. The grandparents' templated voice EE¹ would be used, following access to the desired information, and the demand request would be met by the templated voice EE¹ commencing a discussion on the designated topic when desired. Such discussion can be saved for later use within system 10 or at a remote location as desired, or the discussion may be interactive between the "grandparent" i.e. the templated voice, and the child. This feature is possible by use of a voice recognition module to know in advance of the discussion the identity of the child's voice and to include adequate vocabulary and neural cognition of the vaπous question combinations likely from the child. In addition, a bπdge would be provided from the input and voice recognition module to the templated voice portion of the system, to enable responsiveness by the templated voice. Vaπous speech recognition tools are conceivable for use this manner, when so configured according to the novel uses descπbed herein. Of course this configuration also requires means to rapidly search for the answer to the question and to formulate a response appropπate to the listening child. Clearly this example illustrates the extraordinary potential of this technology, particularly when combined with suitable data, system power, and system speed.

Alternatively, using the optional voice recognition module, it is possible to utilize only limited features to enable a listener of a templated voice to direct the generated voice to cease or continue, or to enable certain other features with certain commands. This would be a form of limited interactive mode appropπate for some but not all types of use. Even if the user chose not to use the optional features and instead merely arranged for a story or a discussion in the absent grandparents' voice, the effect and utility of this is enormous to this or other types of uses.

In the event the user wishes to only use a templated voice consistent with the education and life expeπences of the originator of that voice, then such is possible through input of vaπous filters or modifiers. For example, the templated voice may again be that of the grandparent selected above (templated voice EE¹), and the filter of DATA DATES is used with a selected date of "BEFORE DECEMBER 1963" for a discussion of race relations in the United States in the decade of the 1960s. The result would be a discussion that would not include any information that occuπed after the designated date. In this example, the "grandparent" could not discuss the Voting Rights Act of 1965 or the urban riots of the late 1960s m that country. In similar fashion it is possible to adjust the numerous different aspects of the data or the templated voice itself, for example using the characteπstics type of data shown m Figure 4. It is recognized, however, that other adjustments are possible and contemplated withm the scope of the inventions herein, and that the above examples are merely representative of the capabilities of the invented technology.

In another embodiment of the system and methods disclosed herein, a user may direct a templated voice of a loved one or someone else to read to the user. In this example it is possible for people of all ages to have books read to them in the voice of an absent or deceased family member or other person known to the user. When combined with a vast array of properly configured media and computer readable code means to implement the data links, this innovation alone will provide enormous benefit to users. This type of use has wide applications beyond the specific example just provided. Indeed, an even broader use of this technology in this manner is to have available a database of authoπzed and templated voices which may be accessible and useable by others for a fee or other form of compensation. When used for music, this technology has similar profound implications, particularly if one can access templated voices of past and present singers of renown- many of whose voices are still available for templatmg. Clearly, this technology enables a new industry of manufactuπng, leasing, purchasing, or otherwise using voice templates and associated means, techniques and methods of conducting business therewith. The invention may also have utility in medical treatments for certain minor or major psychological ailments, for which proper use of templated voice therapy may be quite palliative or even therapeutic. Yet another possible use of this technology is to create a newly designed voice for use, but one which has a basis or precursor in one or more templated voices from actual mammalian oπgm Ownership and further use of the newly created voice may be controllable under vaπous means or legal enforcement, such as licensing or royalties and the like. Of course, such voices may be retained as pπvate possessions for limited use by the creator as well. One can imagine the nature of such libraries which may be created. Such voices will represent the creative aspirations of the creator, but each voice will actually have a component or strain of actual mammalian voice as a basis through use of the templatmg tool or code, similar to a strand of tissue DNA but applicable to a specific voice. This type of combination presents powerful new communication capabilities and relationships based on voice and other sounds created by mammals.

Systems according to the invention may be handheld or of other size. Systems may be embedded in other systems or may be stand alone in operation. The systems and methods herein may have part or all of the elements in a distπbuted, network or other remote system of relationship. Systems and methods herein may utilize downloadable or remotely accessible data, and may be used for control of vaπous other systems or methods or processes. Embodiments of the invention include exposed interface routines for requesting and implementing the methods and operations disclosed herein but which may be earned out m whole or in part by other operating or application systems. The templatmg process and the use of templated voices may be accomplished and used by either mammals or artificial machines or processes. For example, a bot or other intelligent aide may create or use one or more templated voices of this type. Such an aide may also be utilized to search for voices automatically according to certain general or limited cntena, and may then generate templated voices m voice factoπes, either virtual or physical. In this manner, large databases of templated voices may be efficiently created. In this or similar systemic use, it may be desirable to create and apply data or other types of tagging and identification technology to one or more portions of the actual voice utilized to create a templated voice. The following are examples of applications using the technology disclosed herein. These are not meant to be limiting, but rather are provided as representative possible uses in addition to those enabled and otherwise suggested elsewhere m this disclosure.

Example I

A templatmg process using elements of the embodiments herein yields a voice coding signal, comprising the logic structure of characteristics of a specific voice essential for accurately replicating the sound of that voice.

Example 2

A personal computer prompter and updater, status reporter, or mate using one or more selected voices using the technology herein.

Example 3 A home energy monitor, reporter, or mate, using one or more selected voices using the technology herein.

Example 4

A hotel room assistant, or automobile assistant to prompt the user according to desired prompting, such as for example a wake-up call in a hotel in the voice selected by the user. In similar manner, an operator of a vehicle might receive information m the voice or voices selected by the user.

Example 5

Using one or more selected voices using the technology herein in a personal digital assistant, a handheld personal computing device, or other electronic device or component at any time for voice capture, mate, alerter, etc.

Example 6

Creating or managing one or more selected voices or voice templates in computer/electronic chip logic, instructions, or code means for implementing the business and technology methods and manufactures disclosed herein.

Example 7

Using the voice template technology in combination with other visual media, such as with a photograph, digital video or a holographic image. Example 8

Using the technology disclosed herein with a flash-memory based profile card for plug-m with any device that can record, play, or reconstitute a voice.

Example 9

Using the technology disclosed herein with a personal device that scans and updates downloadable information for a user as desired in voice or voices of one's choosing. For example, this may be useful for organizing actions capable of being done by a bot, such as an mfo-bot for background searching and interface while the user is not available and then reporting status to the user in one or more designated voices using the technology herein.

Example 10

Using the technology disclosed herein in combination with one or more components of a vehicle or other transportation system

Example 11

Using the technology disclosed herein with one or more components of an airplane for an in-flight companion.

Example 12

Using the technology disclosed herein as a safety reminder when used with one or more components of gear or equipment in the workplace, such as a personal computer posture monitor, electπcal equipment, dangerous equipment, etc.

Example 13

Using the technology disclosed herein as an add-on to other voice activated systems, such as dictation devices, as prompts, companions, or text readers.

Example 14 Using the technology disclosed herein use as social mediation or control mechanisms, such as a tool against road rage or other forms of anger and frustration, activatable by driver or automatically, or by other means.

Example 15 Using the technology disclosed herein as a teaching tool m home, school or the workplace. Example 16

Using the technology disclosed herein for inspirational readings.

Example 17 Using the technology disclosed herein as a tool to act as a family history machine.

Example 18

Using the technology disclosed herein as a MusicMatch™ brand of voice sourcing and matching technology for singers with best or desired voice.

Example 19

Using the technology disclosed herein use as a VoiceSelect™ brand of movie or video match technology to utilize preferred voices for templatmg of entertainment scnpt already used by the oπgmal performer or subsequently created for voice template technology combination uses

Example 20

Using the technology disclosed herein use as an "alter ego" device such as a handheld unit which engages on "SelectVoice™" brand or "VoiceX™" brand mode(s) of operation and has a database of images of those who match the voice as well as anonymous models which can be selected, similar to that referred to m Example 7.

Example 21

Using the technology disclosed herein to create a profile of a profiled or templated voice.

Example 22

Using the technology disclosed herein use as a bedtime reader or a night mate m a dwelling for momtonng and interactive secunty.

Figure 2 is a flow diagram of one embodiment of a voice capture subsystem which may compπse computer readable code means or method for accomplishing the capture, analysis and use of a voice AA designated for templatmg. Figure 3 is one embodiment of a voice analysis subsystem which may compπse logic or method means for efficiently determining voice data charactenzation routing. In these embodiments, voice AA is captured in acquisition module or step 103 and then routed by logic steps and data conductive pathways, such as pathway 106, through the templatmg process. Capture may be accomplished by either digital or analog methods and components. The signal which then represents captured voice AA is routed through analysis means 111 or method to determine whether an existing voice profile or template matches voice AA. This may be accomplished, for example, by compaπng one or a plurality of characteπstics (such as those shown in voice characteπzation subsystem 113 of Figure 4) as determined by either acquisition module 103 or analysis means 111, and then compaπng those one or more charactenstics with known voice profiles or templates available for access, such as at analysis step 111. Representative feedback and initial analysis loop 114 facilitates these steps, as does pathway 116. Such comparison may include querying of a voice profile database or other storage medium, either locally or remotely. The analysis step at analysis module 11 1 and voice characteπzation subsystem 113 may be repeated accordmg to algorithmic, statistical or other techniques to affirm whether the voice being analyzed does or does not relate or match an existing voice profile or data file. Figure 4 provides further detail of voice characteπzation subsystem 113.

Referring again to Figure 2, if the signal corresponding to voice AA does not have a match or is not identified with an existing voice profile set then the signal is routed to the voice charactenzation subsystem for comprehensive characteπzation. However, if an existing voice profile data file matches the profile signal of voice AA, then creation of a template may not be required at module/step 127. In that situation, the signal might be analyzed and or charactenzed for possible generation of a revised profile or template- which itself may then be stored or applied. This situation might occur, for example, when additional characteπzation data is available (such as size of enabling portion, existence or lack of stress, or other factors) which had not been previously available. Accordingly, a specific voice data file might compπse a plurality of templates. This is a validation process, having logic steps and system components shown generally at validation subsystem 133 m Figures 2 and 3. It is emphasized that, as to relational location to subsystems and components, these Figures are generally schematic. Also, as shown in Figure 3, after determination that a voice profile data file exists (step 137), then the validation logic at step 139 will, optionally, occur. If a revision of an existing template is merited, then it is generated at step 142. Alternatively, logic step 145 notes that no revision to an existing template is to be made. Following either steps 143 or 145, then the new, revised, or previous voice profile or template is stored or used at step 155.

The template creation module/step 127 of Figure 2 comprises utilizing the voice characteπzation subsystem to create a unique identifier, preferably a digital identifier, for that specific voice being templated or profiled. This data is similar, m the abstract, to genetic codes, gene sequence codes, or bar codes, and like identifiers of singularly unique objects, entities or phenomena. Accordingly, applicants refer to this voice profile or template as "Voice Template Technology™" as well as "Voice DNA™ or VDNA™" and "Voice Sequence Codes™ or Voice Sequence Coding™" The terms "Profile, Profiles or Profiling" and derivative terms may be substituted in the above trademark or other reference terms for this new technology. Following completion of template creation, the voice template may be stored (shown at storage module or step 161 or applied in use at module or step 164). Figure 4 is a schematic representation of a voice characteπzation subsystem. This disclosure comprises at least one embodiment of charactenzation data and means for determining and charactenzmg salient data to define a voice using voice templatmg or profiling, as disclosed herein. As shown, various types of data is available for comparison in formulating the characteπzation data. This characteπzation data will then be used to create the voice template or profile according to coding criteria. Although the data in Figure 4 appears to be arranged in discreet modules, an open comparator process may be preferred m which any data may be accessed for companson in any of vaπous sequences or weighted pπonties. Regardless, as shown m this figure, data may compπse the categoπes of language, gender, dialect, region, or accent (shown as "Voice Characteπstics" output signal VC₀ at module or step 201); frequency, pitch, tone, duration, or amplitude (shown as output signal VCi at module or step 203); age, health, pronunciation, vocabulary, or physiology (shown as output signal VC₂ at module or step 205); patterns, syntax, volume, transition, or voice type (shown as output signal VC₃ at module or step 207); education, expeπence, phase, repetition, or grammar (shown as output signal VC₄ at module or step 209); occupation, nationality, ethnicity, custom or setting (shown as output signal VC₅ at module or step 211); context, vanances, rules/models, enabling portion type, size or number (shown as output signal VC₆ at module or step 213); speed, emotion, cluster, similaπties, or acoustic model (shown as output signal VC₇ at module or step 215); math model, processing model, signal model, sounds-like model, or shared model (shown as output signal V at module or step 217); vector model, adaptive data, classifications, phonetic, or articulation (shown as output signal VC₉ at module or step 219); segments, syllables, combinations, self-learned, or silence (shown as output signal VC₎₀ at module or step 221); packets, breathing rate, timbre, resonance, or recurrence model (shown as VCπ at module or step 223); harmonics, synthesis models, resolution, fidelity, or other characteπstics (shown as output signal VCj₂ at module or step 225); or various other techniques for uniquely identifying a portion (whether fractional or in its entirety) of a voice. For example, this may further include a digital or analog voice signature, modulation, synthesizer input data, or other data formed or useful for this purpose, all of which is shown as output signal VC_% at module or step 227.

It is recognized that one or more data types from any one or more modules or steps may provide value to a voice template. Also, for purposes of this invention, VC_X encompasses any known categoπzation technique at the time of interpretation, regardless of mention herein, provided it is useful m then defining a unique voice profile or template for a specific voice- and is used according to the novel teachings disclosed herein. Again, it is recognized that data combined in voice characteπstic files and output signals VC₀, VC,, VC₂, VC₃, VC₄, VC₅, VC₆, VC₇, VC₈, VC₉, VC,₀, VC„, VC_I2, and VC_X may be pπoπtized and combined in vaπous ways in order to accurately and efficiently analyze and charactenze a voice, with VC_X representing still further techniques incorporated herein by reference. Figures 5 and 6 illustrate an exemplary signal bundler suitable for receiving the vanous voice charactenstic data, such as digital or coded data representative of the information deemed relevant and formative of the voice being templated. The signal bundler 316 then combines the output of signal content module or step 332 and values/sconng from one or more signals VC₀ - VC and formats the signal or code at module or step 343 as appropπate for proper transfer and use by vaπous potential user interfaces, devices or transmission means to create an output voice template, code, or signal VT_X. It is recognized that various methods are possible to create a unique identifier to delineate the vaπous voice characteπstics- and that such vaπous possibilities are enabled herein in view of the broader context and scope of this invention- to a certain degree independent of some component methodology. Figure 7 is a representative organization and method of an electronic query and transfer between a voice template generation or storage facility 404 and a remote user. In this representation, enabling portions may be sent to a remote voice template generation or storage facility 404 by any number of various users 410, 413, 416. The facility 404 then generates or retπeves a voice template data file and creates or retπeves a voice template signal. The template signal is then transmitted or downloaded to the user or its designee, shown at step 437. At the time of download, or later, following a user request 441, the template signal is formatted for appropπate use by a destination device, including activation instructions and protocols, shown at step/module 457.

Figure 8 is a schematic representation of a mobile medium, such as a card, disk, or chip on which are essential components, depending on the user mode and need, for utilizing voice template technology. For example, using Figures 7 and 8, a hotel door card 477 may be provided at check-in to a hotel by a traveler. However, in addition to the normal onsite security code programming and circuitry 479 applied to the card, additional features incorporating aspects of this invention may be made available. A schematic representation of optional features withm such a card include means 481 for receiving and using a voice template for a voice or voices selected by the traveler for vaπous purposes during the traveler's stay at the hotel. As shown, such features may include a template receiving and storage element 501, a noise generator or generator circuitry 506, a central processmg unit 511, input/output circuitry 515, digital to analog/analog to digital elements 518, and clock means 521. Again, vanous other elements may be utilized, such as voice compression or expansion means- such as those known in the cellular phone industry, or other components to enable the card to function as desired. The user may then enjoy dialog or interface with inanimate devices within the hotel m the voιce(s) selected by the traveler. Indeed, a traveler profile may even retain such voice preference information, as appropπate, and certain added billings or benefits may accrue through use of this invention. It is recognized that the invention may be employed in a wide vaπety of applications and articles, and the example of Figures 8 and 9 should not be considered limiting. Figure 9 is a depiction of a photograph 602 which is configured for interactive use of voice template technology with voice JJ attributable to figure F_JJ and voice KK attributable to figure F . Means are combined with the frame 610 or other structure, whether computer readable code means or simple three dimensional material, for interfacing the subjects or objects of the photo (or other media) with the appropnate voice templates to recreate a dialogue that either likely occurred or could have occurred, as desired by the user.

It is recognized that vaπous means and methods exist to capture, analyze, and synthesize real and artificial voice components. For example, the following United States patents, and their cited or listed references, illustrate a few of the means for captuπng, synthesizing, translating, recognizing, characteπzmg or otherwise analyzing voices, and are incorporated herein m their entirety by reference for such teachings: 4,493,050; 4,710,959; 5,930,755; 5,307,444; 5,890,117; 5,030,101; 4,257,304; 5,794,193; 5,774,837; 5,634,085; 5,704,007; 5,280,527; 5,465,290; 5,428,707; 5,231,670; 4,914,703 4,803,729; 5,850,627; 5,765,132; 5,715,367; 4,829,578; 4,903,305; 4,805,218; 5,915,236; 5,920,836 5,909,666; 5,920,837, 4,907,279; 5,859,913; 5,978,765; 5,475,796; 5,483,579; 4,122,742; 5,278,943 4,833,718; 4,757,737; 4,754,485; 4,975,957; 4,912,768; 4,907,279; 4,888,806; 4,682,292; 4,415,767 4,181,821; 3,982,070; and 4,884,972. None of these references illustrates the inventive contnbutions claimed or elsewhere disclosed herein. Rather, the above patents illustrate tools that may be useful rather than necessary m practicing one or more embodiments of this invention. Thus, it is recognized that various systems, products, means, methods, processes, data formats, data related storage and transfer media, data contents and other aspects are contemplated withm this invention to achieve the novel and nonobvious innovations, advantages, products and applications of the technology disclosed herein. Therefore the above disclosures shall be considered exemplary rather than limiting, where appropπate, so that the claims are afforded the breadth of scope to which this pioneeπng technology should be entitled without limitation by the pace of development and availability of implementing technologies.

Claims

What is claimed:

1. A system for captuπng an enabling portion of a specific voice sufficient for using that portion as a template in further use of the voice, comprising: a. means for captuπng an enabling portion of a voice in a form useful for analysis as to voice characteristics; b. analysis means for receiving and analyzing the captured voice and for charactenzmg elements of the captured voice as characteπzation data; c. storage means for receiving characterization data from the analysis means for a specific voice; and d. retneval means for retrieving the analysis and characteπzation data for further use.

2. The system of claim 1 m which the means for captuπng the voice compπses digital recording means.

3. The system of claim 1 m which the means for capturing the voice compπses a flash memory card.

4. The system of claim 1 m which the means for captuπng the voice comprises analog recording means.

5. The system of claim 1 m which the means for capturing the voice compπses input means for receiving a live voice and for transmitting that live voice to the analysis means.

6. The system of claim 1 in which the analysis means comprises digital data storage means.

7. The system of claim 1 in which the analysis means comprises means for identifying specific patterns, syntax, frequency, pitch and tones of speech in the captured voice data.

8. The system of claim 1 m which the analysis means compπses means for identifying specific vocabulary, pronunciation, or accent unique to the captured voice.

9. The system of claim 1 m which the analysis means compπses means for identifying specific features unique to the captured voice deπvmg pπncipally from specific anatomic structures of the oπgmator of the voice.

10. The system of claim 1 in which the analysis means compπses means for determining the vocabulary of the oπgmator of the captured voice.

11. The system of claim 10 m which the analysis means compπses means for setting the vocabulary as characteπzation data for use in forming a future templated voice.

12. The system of claim 1 in which the analysis means compπses digital processing apparatus for digitally processing input data in the form of a voice or digital representation of a recorded voice.

13. The system of claim 1 in which the analysis means compnses second input means for receiving additional data regarding the physiology of the voice ongmator.

14. The system of claim 13 in which the analysis means second input means compπses digital signal processor means suitable for selectively receiving audio or other data compnsmg visualization information on the morphology of the voice originator.

15. The system of claim 1 in which the analysis means compπses comparison means for companng an input voice data set with stored data compnsmg age data, language data, educational data, gender data, occupation data, accent data, nationality data, ethnic data, voice type data, custom data and setting data.

16. The system of claim 1 in which the analysis means compπses third input means for receiving data regarding the voice ongmator compπsmg age data, educational data, gender data, occupation data, accent data, nationality data, ethnic data, voice type data, custom data, language data and setting data.

17. A method of creating a voice-like noise which is identical m sound to an actual specific human's voice, comprising the steps of: a. capturing an enabling portion of a specific human's voice for storage and use: b. stonng the enabling portion of the specific human's voice; c. analyzing the enabling portion to identify essential components or characteπstics of the captured voice; and d. utilizing the identified essential components or characteπstics to create a new voice which, when assigned data from one or more database means and when heard, sounds identical in all respects to the voice of the specific human's voice to a listener having normal aural discretion abilities.

18. The method of claim 17 m which the analyzing step compπses the steps of identifying the components m the captured enabling portion of the specific human's voice relating to at least one of the components including frequency, tone, pitch, volume, accent, gender, harmonic structure, acoustic power, phonetic or timing accent, power and periodicity.

19. The method of claim 18 in which the step of captuπng an enabling portion of a specific human's voice for storage and use includes captuπng either larynx generated noise or turbulence generated noise of the specific human's voice.

20. A method of accurately replicating a human voice compnsmg the steps of: a. identifying a minimum size data set compnsmg a combination of words, sounds or phrases which must be emitted by the oπgmator of a voice to be replicated; b. capturing the emission of the combination of words, sounds or phrases by the ongmator of the voice to be replicated m a medium; c. analyzing the captured emission to identify voice charactenstics of the ongmator of the voice sufficient to allow artificial generation of the voice, using the identified charactenstics, so that the artificially generated voice is substantially identical in all respects to a listener having normal aural discretion abilities when the listener hears the generated voice utilizing some language components not contained in the captured emission of the ongmator' s actual voice.

21. An article of manufacture comprising: a. a computer usable medium having computer readable program code means embodied therein for causing replication of a human voice, the computer readable program code means m said article of manufacture comprising: b. computer readable program code means for causing a computer to effect an analysis of a captured enabling portion of an oπginator's voice to identify voice characteπstics data sufficient to allow artificial generation of the voice; and c. computer readable program code means for causing use of the identified voice characteπstics data to artificially generate a voice, so that the artificially generated voice is substantially identical in sound and usage to a listener when the listener hears the generated voice utilizing some language components not contained m the captured emission of the oπginator's actual voice.

22. The article of manufacture of claim 21 further comprising computer readable program code means for stonng the generated voice for later use.

23. The article of manufacture of claim 21 further compnsmg computer readable program code means for using the voice charactenstics data to create a voice profile of the ongmator of the voice.

24. The article of manufacture of claim 21 further comprising computer readable program code means for accessing data base means for stonng data compnsmg age data, educational data, gender data, occupation data, accent data, language, nationality data, ethnic data, voice type data, custom data, general data and setting data.

25. A computer program product for use with an aural output device, said computer program product comprising. a. a computer usable medium having computer readable program code means embodied therein for causing replication of a human voice via an output aural device, the computer program product compnsmg: b computer readable program code means for causing a computer to effect an analysis of a captured enabling portion of an ongmator' s voice to identify voice characteπstics data sufficient to allow artificial generation of the voice; and c computer readable program code means for causing use of the identified voice characteπstics data to artificially generate and output a voice via an aural output device, so that the artificially generated voice is substantially identical in sound and usage to a listener when the listener hears the generated voice utilizing some language components not contained in the captured emission of the ongmator's actual voice.

26. A computer program product for use with a display device, said computer program product compnsmg: a. a computer usable medium having computer readable program code means embodied therein for causing replication of a human voice and venfication of the accuracy of the replicated voice displayed on the display device, the computer program product compnsmg: d. computer readable program code means for causing a computer to effect an analysis of a captured enabling portion of an oπginator's voice to identify voice characteπstics data sufficient to allow artificial generation of the voice; and e computer readable program code means for causing use of the identified voice charactenstics data to artificially generate a voice and to compare the charactenstics of the generated voice to the oπginator's voice on a display device, so that the artificially generated voice is substantially identical m sound to a listener when the display device so indicates and when a listener actually hears the generated voice utilizing some language components not contained in the captured emission of the oπginator's actual voice.

27. A computer program product for use with an aural output device, said computer program product comprising: a. a computer usable medium having computer readable program code means embodied therein for initiating replication of a human voice via an output aural device, the computer program product compnsmg: b. computer readable program code means for causing a computer to receive and activate a voice charactenstics data file unique to a specific voice sufficient to allow artificial generation of the voice; and c computer readable program code means for causing use of the identified voice charactenstics data to artificially generate and output a voice via an aural output device, so that the artificially generated voice is substantially identical in sound to a listener when the listener hears the generated voice and a captured emission of the oπginator's actual voice.

28. A computer program product for use with an electronic device, said computer program product comprising: a. a computer usable medium having computer readable program code means embodied therein for initiating replication of a human voice, the computer program product compnsmg: b. computer readable program code means for causing receipt and activation of a voice characteπstics data file unique to a specific voice sufficient to allow artificial generation of the voice; and c. computer readable program code means for causing use of the identified voice characteπstics data file and a noise generation means sound output to artificially generate a voice, so that the artificially generated voice is substantially identical m sound to the originator's actual voice.

29. A memory for stonng data for access by an application program being executed on a data processing sub-system, compnsmg: a a data structure stored in said memory, said data structure including information resident in a database used by said application program and including: b. at least one voice enabling portion data file stored in said memory, each of said voice enabling portion data file set containing information substantially different from any other voice enabling portion data file set; c a plurality of voice characteπstics data files containing different reference information for a plurality of voice charactenstics; and d. a plurality of voice profile sets each having at least one voice profile data file havmg data unique to that data file only; wherein the data structure allows access to the voice characteπstics data files and the voice profile data files to conduct companson operations with at least one voice enabling portion data file.

30. A data processing system executing an application program and containing a database used by said application program, said data processing system compnsmg: a. CPU means for processing said application program; and b. memory means for holding a data structure for access by said application program, said data structure being composed of information resident in a database used by said application program and including: at least one voice enabling portion data file stored m said memory, each of said voice enabling portion data file set containing information substantially different from any other voice enabling portion data file set; a plurality of voice characteπstics data files containing different reference information for a plurality of voice characteπstics; a plurality of voice profile sets each having at least one voice profile data file having data unique to that data file only; and c. wherein the data processing system allows access to the voice characteπstics data files and the voice profile data files to conduct companson operations with at least one voice enabling portion data file.

31. A computer data signal embodied in a transmission medium compnsmg: a. an encryption source code for a unique voice profile template useful for keying additional electronic noise to create a specific generated voice; and b. a earner medium suitable for carrying the encryption source code to a location and configured so that the encryption source code is removable from the earner medium to be applied as a key to create a generated voice.

32. A method for using a selected voice as a personal voice assistant with an electronic device, comprising the steps of: a. activating electronic means for accessing a remote database; b. transmitting a signal portion to a remote database having a voice database containing a plurality of voice profile sets each having at least one voice profile data file having data unique to that data file only and identifiable by a unique identifier; c. transmitting a signal portion to the remote database to uniquely identify a desired data file and then to effect transfer of the data file content to the user's designated electronic device location; and d. implementing use of the selected and transferred data file as a voice template, in combination with appropriate noise generated either by the electronic device or other means for generating such noise, so that as desired the user may receive noise from the electronic device in the sound of the selected voice as determined by the identified voice.

33. The method of claim 32 m which the data file includes data characteπstics of the selected voice arranged as computer readable program code means for causing use of the identified voice characteπstics data to artificially generate a voice template.

34. The method of claim 32 in which the implementing step comprises application of authorization means to only allow authoπzed users to access and use the voice template technology and data.

35. The method of claim 32 m which the implementing step compπses application of selectively accessible verification means for veπfying that voices heard are either real or template generated.

36. A method of doing business in which a system is used for capturing an enabling portion of a specific voice sufficient for using that portion as a template in further use of the voice, compnsmg the steps of: a. capturing an enabling portion of a voice in a form useful for analysis as to voice characteπstics; b. inputting the enabling portion into an analysis module for charactenzmg elements of the captured voice as charactenzation data; c. receiving the characteπzation data from the analysis module for a specific voice; and d. stonng the characteπzation data for further use.

37. The method of claim 36 in which the means for captuπng the voice compnses digital input means

38. The method of claim 36 in which the enabling portion of the voice is received electronically.

39. The method of claim 36 m which the characterization data is bundled to form a voice template signal useful for combining with generated noise to create a templated voice which sounds like the oπgmal specific voice.

40. The method of claim 36 in which the templated voice is controlled so that the templated voice may receive speech input commands to elicit new words in the templated voice but which were not inputted by the specific voice.

41. An automated machine for captunng an enabling portion of a specific voice and for using that portion as a template useful for further use of the templated voice, comprising: a. an acquisition module for acquiring an enabling portion of a voice in a form useful for analysis as to voice characteπstics; b an analysis module for receiving and analyzing the captured voice and for charactenzmg elements of the captured voice as charactenzation data; and c. a template generator module for automatically generating a voice template signal as a unique identifier of the acquired specific voice.

42. The machine of claim 41 further comprising communication means for communicating with storage means for receiving charactenzation data from a database.

43. The machine of claim 41 further compnsmg communication means for communicating with storage means for storing the generated template until requested.

44. An online method for creating voice templates and generating revenue for such generation, comprising: a. captuπng an enabling portion of a specific voice; b. analyzing the enabling portion of the specific voice to generate a data profile which defines the characteπstics of the captured voice m a way that can be reconstituted for later use; c. generating a voice template signal as a unique identifier of the acquired specific voice; and d. providing at least one generated data profile for commercial use by another.

45. A machine operated method for creating a voice template and generating revenue for such generation, comprising: a. captuπng an enabling portion of a specific voice; b. analyzing the enabling portion of the specific voice to generate a data profile which defines the characteristics of the captured voice in a way that can be reconstituted for later use; c. using the data profile, generating a voice template signal as a unique identifier of the captured specific voice; and d. providing at least one voice template signal for commercial use.

46. A business method for creating a voice template, compnsmg: a. capturing an enabling portion of a specific voice or templated voice; b. using computer means, analyzing the enabling portion of the voice to generate a data profile which defines the charactenstics of the captured voice in a way that can be reconstituted for later use; c. electronically generating or retπevmg a voice template signal as a unique identifier of the captured voice; and d. providing at least one voice template signal for commercial use.

47. The method of doing business of claim 46 in which the step of providing is accomplished on an electronic data exchange.

48. A method for creating a voice template from a plurality of voices, comprising: a. capturing an enabling portion of a plurality of voices or templated voices; b. using computer means, analyzing the enabling portions of the voices to generate a data profile which defines the characteristics of the captured voices in a way that can be bundled as a single voice signal suitable for reconstitution for later use; and c. electronically generating a voice template signal as a unique identifier of the newly generated voice.