JPWO2007069512A1 - Information processing apparatus and program - Google Patents

Abstract

It is possible to configure an information processing apparatus that is used for a distribution apparatus that distributes and stores information in which changes are made with respect to the markup language notation used for content information and stores the changed information. . More specifically, speech recognition technology and phoneme recognition can be achieved by changing the phoneme dictionary information written in the markup language to add tags, adding variables and attributes, and saving, changing, and delivering. Operate information processing equipment using technology. As a result, when speech recognition is performed on words or character strings included in the content information, even if the word model, acoustic model, grammar model, and part-of-speech information are not registered in the recognition dictionary, the content information It is possible to provide an information processing apparatus and the like capable of realizing appropriate speech recognition by dynamically constructing and using recognition dictionary information by phonetic symbols used for phonetic symbol recognition made up of phonemes and phonemes. it can.

Description

  The present invention relates to an information processing apparatus that uses phoneme recognition and / or phoneme piece recognition in speech recognition.

  2. Description of the Related Art Conventionally, technologies relating to speech recognition are known as information processing apparatuses and operation methods that generally use speech. As a method of performing speech recognition, generally, a phoneme or phoneme segment is obtained by phoneme recognition or phoneme recognition using an acoustic model or a standard parameter or template based on a phoneme or phoneme statistically generated from speech accompanying user utterance. There is known a method for extracting phonemes and phoneme symbol strings by extracting chronologically.

  Then, using a speech recognition dictionary in which words consisting of phoneme strings and phoneme string strings are recorded, evaluation is made on the match between the recognized phoneme string or phoneme string string and the phoneme string or phoneme string string registered in the speech recognition dictionary. Then, a speech associated with a phoneme sequence or a phoneme segment sequence with a high degree of coincidence as a result of the evaluation is obtained, or a process associated with speech recognition or recognition is realized by executing a device control command.

  Here, as a user interface for controlling the device, as in Non-Patent Document 1, a device specified by the phoneme recognition dictionary and a device control method registered in the dictionary in association with the word are selected and implemented by phoneme recognition processing. As a technique for recognizing phonemes and phoneme pieces, as shown in Patent Document 1, it has been used as a known technique for a long time.

  In speech recognition, spoken words in human dialogue often change abbreviations and exclamation words and coined words such as “Oo” and “Uon”. Especially in content information, product names and actor names are registered in the dictionary. There were many difficult proper nouns, and not all words could be registered in the dictionary. Therefore, Patent Document 2, Non-Patent Document 2, Non-Patent Document 3, and the like have proposed techniques for searching for content using phoneme recognition.

  Here, Patent Document 3 proposes that a user can easily perform a voice operation by changing the display expression of a recognizable word in use for speech recognition in HTML, which is one of markup languages. Has been made.

  Patent Document 4 proposes a method for dynamically acquiring recognition dictionary data based on an acoustic model with a minimum vocabulary.

Further, according to Patent Document 5, in the use for speech recognition in HTML, which is one of markup languages, a range can be designated by a specific symbol to identify a recognizable word, and speech recognition can be performed. Is proposed to the user, and the words that are difficult to pronounce are described with a recognizable reading method for convenience.
Japanese Patent Laid-Open No. 62-220998 JP 2005-70312 A JP-A-11-25098 JP 2002-91858 A JP 2005-18241 A “Research and development on life support interface for aging society”, Key Project Research Report by Aomori Prefectural Industrial Research Center Vol.5, Apr.1998-Mar.2001 031 Masayuki Nakazawa, Takashi Endo, Kiyoshi Furukawa, Jun Toyoura, Ryuichi Oka (New Information Processing Development Corporation), "Study of speech summaries and topic summaries using phoneme symbol sequences from speech waveforms," IEICE Tech. 28, pp. 61--68, June 1996. Ryuichi Oka, Hironobu Takahashi, Takuichi Nishimura, Nobuhiro Sekimoto, Hidehide Mori, Masanori Ihara, Hiroaki Yabe, Hiroki Hashiguchi, Hiroshi Matsumura. Pattern Search Algorithm Map-What Supports "CrossMediator" Study Group of Society, volume 1, pages 1-6. Japanese Society for Artificial Intelligence, 2001.

  Also, the phoneme symbol string markup language can be used in the MPEG7 description used in a video stream such as MPEG2, using the structure of Segment and Media Locator, and the <Media Locator> can be used to specify the Segment or Frame in the video content. You can specify it directly, or use <Media Locator> to point to the content, <Media Time> to specify the time position inside the content and combine it with a tag that specifies an appropriate proper noun, When assigning, there is a method of using <Series> description method to attach the same kind of metadata at fixed intervals as low level metadata of Visual and Audio. At this time, there is a method of specifying as <Scalable Series> for audio, and MPEG7 audio has <Spoken Content DS> which describes a word lattice and a phone lattice as a result of automatic speech recognition. .

  In addition, VoiceXML, a standardized method for speech recognition, proposes a method for expressing user interface and user interface descriptions that have been divided between products so far in order to implement grammar-dependent recognition according to the context. However, a method for dynamically constructing dictionary information by assigning an attribute to a target range of an arbitrary tag using a phonetic symbol identifier such as a phoneme or a phoneme without depending on context or grammar has not been proposed.

  According to conventional applications and documents, many phonemes and syllables are confused, but the phoneme in the present invention is the case where the pronunciation of “Akasana” in Japanese is used as an example, and the syllable is used. If there is a phonetic notation, “a / ka / sa / ta / na” or “a / ka / sa / ta / na” is displayed in a single voice, and “a / k / a / s / a / t / a / n / a '' or `` a / cl / k / a / s / a / cl / t / a / n / a '', or `` a / ak / k / ka / a '' / as / s / sa / a / at / t / ta / a / an / n / na / a '' or `` a / a-cl / cl / cl-k / k / ka / a / as / s / sa An example such as `` / a / a-cl / cl / cl-t / t / ta / a / an / n / na / a '' can be considered as bigram, `` aaa / a-cl-cl / cl-cl- cl / cl-cl-k / cl-kk / kka / aaa / aas / sss / saa /… taa / aan / nnn / naa / aaa Arbitrary parts such as the first half, middle board, and second half / Cl / points to the silent or unvoiced part before sounding, and both these phonemes and phonemes are phonemes that represent any sound by any improvement. You may change into a notation symbol piece, a phonetic symbol piece, and a phonetic symbol piece like a notation symbol, a phonetic symbol, a phonetic symbol, and the phoneme piece which decomposed them time-sequentially.

  In addition, the difference between phonetic symbol recognition using phonemes and phonemes and normal speech recognition is explained. Phone recognition and phoneme recognition do not perform vocabulary recognition that interprets meaning and content unlike general speech recognition. And acoustic models are not dynamically constructed according to changes in language models such as words, grammar, and parts of speech. More specifically, phoneme recognition and phoneme recognition do not use grammar-related language models. Does not capture meaning, or does not convert to symbols that contain meaning such as kanji, or does not distinguish homonyms or homonyms, and differentiates parts of speech such as nouns and verbs according to context In this case, phonemes, phonemes, phonetic symbols, phonetic symbols, phonetic symbols, It is denoted as phonetic symbol recognition together recognition using phonetic symbols based on time series of the divided phonetic symbol) phonetic symbols pieces and their symbol string.

  In this way, recognition by phonemes and phonemes is performed by analyzing a speaker's utterance using a static acoustic model for each phonetic symbol and matching the phonetic symbol string accompanying the utterance with the phonetic symbol string in the recognition dictionary. The recognition process and the structure of the recognition dictionary are simplified due to the feature of evaluating only the phoneme, and the phonetic symbols and phonetic symbols such as phonemes and phonemes are used even for unregistered words and exclamation words to evaluate only the sound match. It is possible to recognize the identifier string.

  At this time, as in the past, a dynamic acoustic model that learns according to the speaker's utterance characteristics and improves performance may be used, but depending on words and grammar as in general speech recognition, There is a feature that the process of dynamically switching the model is not performed in phoneme recognition or phoneme recognition.

  For this reason, comparing phoneme strings and phoneme string sequences with registered dictionary contents makes it possible to easily detect unregistered phoneme strings and phoneme string sequences, resulting in recognition results by phonetic symbol recognition using phonemes and phoneme segments. It is also conceivable to implement efficient speech recognition by limiting speech based on words and implementing speech recognition that takes into account general grammar again.

  And even if there is a word that is not registered in the dictionary, the recognition method using phonemes and phoneme fragments converts unregistered words in the sentence to be recognized into hiragana character notation, and changes to the transition state of the converted hiragana character string. In addition, a symbol string converted to a phoneme string or phoneme string string based on the prosody obtained from witness information is temporarily registered in the recognition dictionary, and acquired after the user's utterance is recognized as a phoneme string or phoneme string string. By comparing the phoneme sequence or phoneme sequence with the phoneme sequence or phoneme sequence of the recognition dictionary, the degree of coincidence between the symbol sequences is measured, and the recognition result is acquired. In addition, it is possible to perform speech recognition having a dictionary configuration with dynamic phonemes and phonemes having a higher degree of freedom than conventional speech recognition.

  At this time, the user can improve the recognition accuracy with a dynamic acoustic model dictionary that does not depend on grammar or words by re-learning the phoneme or phoneme unit acoustic model according to the user's utterance. Re-learning for recognition may be performed by reusing the acoustic information obtained from the utterances as teacher information.

  In conventional speech recognition technology, abbreviations of spoken words in human dialogue, exclamation words (exclamation words) such as “Oo” and “Uon” and coined words have many differences depending on the times and environments, especially in content information Although dynamic proper nouns that depend on trends such as product names and actor names have been inefficiently registered in the recognition dictionary, they have existed for a long time as a challenge when putting huge and varied speech recognition into practical use. Since it is relatively difficult to repeatedly distribute recognition dictionaries including acoustic models and grammatical models, recognition based on vocabularies not registered in the recognition dictionary is virtually impossible.

  Also, in conventional speech recognition, learning of prosodic models and grammatical models is generally indispensable, and such processing procedures have become issues for recognition of unregistered words based on the aforementioned coined words, buzzwords and proper nouns. There is a problem that it is difficult to learn a grammar model based on the prosody related to such unregistered words and the co-occurrence relationship between words.

  Conventionally, audio information in a markup language cannot be searched or operated except for audio information synchronized with video and audio as content information. In order to provide information including a phoneme symbol string to a user, it is necessary to recognize speech information in advance and store the phoneme and a word ID (word identifier) in association with each other.・ There was a problem that there was no method of providing or operating a phoneme.

  Further, in the technique disclosed in Patent Document 3 described above, a method for recognizing a word that is not registered in the dictionary is not presented. Furthermore, in the technique disclosed in Patent Document 4, speech recognition that does not depend on vocabulary cannot be performed, and it is necessary to take measures such as learning prosodic models for unknown words each time. High voice recognition could not be realized. Furthermore, the technique disclosed in Patent Document 5 has a problem that the speech recognition method is not different from the conventional speech recognition method, and recognition using phonemes or phonemes cannot be performed.

  Based on such problems, the object of the present invention is to use a word model, an acoustic model, a grammar model, and a part-of-speech information in a speech recognition dictionary when speech recognition is performed on a word or character string included in content information. An information processing device that can realize more appropriate speech recognition by using recognition dictionary information based on phonetic symbols using phonetic symbol recognition consisting of phonemes and phoneme pieces even if is not registered The purpose is to provide.

  In order to solve the above problems, an information processing apparatus according to a first aspect of the present invention includes a content information acquisition unit that acquires content information including character information and / or meta information, and content information acquired by the content information acquisition unit. And a recognition phonetic symbol string detecting means for detecting a recognized phonetic symbol string consisting of phonetic symbols, and a recognition dictionary information generating means for generating recognition dictionary information using the recognized phonetic symbol string. Features.

  An information processing apparatus according to a second aspect of the present invention includes a content information acquisition unit that acquires content information including character information and / or meta information, and character information and / or meta information from the content information acquired by the content information acquisition unit. By referring to the expansion target character string detection means for detecting the expansion target character string based on the character string, the phonetic symbol storage means for storing the character string and the phonetic symbol in association with each other, and the phonetic symbol storage means, A phonetic symbol conversion unit that converts the expansion target character string into a recognized phonetic symbol string; and a recognition dictionary information generation unit that generates recognition dictionary information using the recognition phonetic symbol string. .

  In addition, in the information processing apparatus according to the second aspect, in the information processing apparatus according to the second aspect, the content information is stored by adding the phonetic symbols converted by the phonetic symbol conversion means to the content information. The apparatus further comprises means.

  According to a fourth aspect of the present invention, in the information processing apparatus according to any one of the first to third aspects, content information stored by the content information storage unit and a recognition dictionary generated based on the content information It is further characterized by further comprising transmission means for transmitting information to another information processing terminal.

  According to a fifth invention, in the information processing apparatus according to any one of the first to fourth inventions, a voice input means for inputting voice, and a feature amount of the voice input by the voice input means are extracted. A feature amount extracting unit; a feature amount phonetic symbol converting unit that converts the feature amount extracted by the feature amount extracting unit into a phonetic symbol; and a phonetic symbol converted by the feature amount phonetic symbol converting unit. And a process executing means for evaluating a phonetic symbol constituting a recognized phonetic symbol string included in the recognition dictionary information and executing a predetermined process corresponding to the most similar phonetic symbol. Features.

  According to a sixth aspect of the present invention, in the information processing apparatus according to the fifth aspect of the invention, the content information includes phoneme information and / or phoneme piece information, and the processing execution means includes the feature quantity phonetic symbol. The phonetic symbols converted by the conversion means and the phonetic symbols constituting the recognized phonetic symbol string included in the recognition dictionary information are evaluated, and the user is notified of the voice corresponding to the most similar phonetic symbol. It is characterized by presenting information by utterance.

  According to a seventh aspect, in the information processing apparatus according to any one of the first to sixth aspects, the phonetic symbol is a phoneme or a phoneme piece.

  The eighth invention is characterized in that in the information processing apparatus according to any one of the first to sixth inventions, the executed process is an authentication process accompanying phoneme recognition.

  According to a ninth aspect of the invention, there is provided a program for interpreting information described using a markup language in a computer, an attribute acquiring step for acquiring an attribute designated by the interpretation, an attribute acquiring step, A phonetic symbol extraction step for extracting a phonetic symbol sequence and / or a phoneme sequence and / or a phoneme fragment sequence associated with the attribute acquired by the phoneme sequence, and a phoneme sequence dictionary used in the phoneme recognition unit by the phonetic symbol extraction step And a dictionary changing step for changing.

  According to a tenth aspect of the present invention, there is provided a program for interpreting information described using a markup language in a computer, an attribute acquiring step for acquiring an attribute designated by the interpretation, an attribute acquiring step, A phonetic symbol extraction step of extracting a phonetic symbol string and / or a phoneme sequence and / or a phoneme fragment sequence associated with the attribute acquired by the user, and a user inputs based on the attribute acquired by the attribute acquisition step An information type evaluation step for evaluating the type of information and a dictionary changing step for changing a phoneme string dictionary used in the phoneme recognition unit are realized by the information evaluation step.

  According to the present invention, in order to use an information processing device using phoneme recognition, a phoneme dictionary necessary for recognition of provided content information is acquired from a markup language associated with the content information or included in the content information. By doing so, it is possible to deal with unspecified words related to display contents. Therefore, in order to perform processing with a high possibility of unspecified words such as product sales in a single device or server / client environment, the name of a phoneme string, phoneme string string, or various identifiers is a tag attribute in the markup language. It is a problem to be able to specify the utterance phoneme dictionary in scene units that span multiple frames of moving images or frames as frames of moving images or frames in content images, sentences in page units or sentence structures It is intended to solve this problem.

  Also, by expanding the phonemes used for these operations, as variables, attributes, and specific tags depending on the file format for distribution and markup language such as HTML, XML, RSS, EPG, BML, MPEG7, CSV sent to the user Indexing using voices easily by incorporating identifiers using phoneme strings and phoneme strings related to voice operations in association with arbitrary markup languages and scripts, and users can acquire, view, and manipulate information using voice Therefore, it is possible to distribute and share voice control information for the purpose of acquisition and to acquire voice control information on the terminal side, and to solve the problem.

  And, when the present invention implements the conventional technique of recognizing unspecified words using phonemes or phonemes, it restricts the words appearing in one scene of content information to various contents changing in the Internet environment. By providing a dynamic dictionary construction method using phoneme sequences and phoneme segment sequences, it is possible to realize speech recognition processing for unspecified words without using prosodic models or grammatical models. We are trying to realize improvements.

  In the case of MPEG7, a scene name, an actor name, and a cast name in a tag representing a scene of video information are specified by attributes, variables, and range specification by a tag using a phoneme symbol string or a phoneme symbol string. Using the information described in the markup language other than the recognition location, and performing a search with any actor name or cast name using the phoneme search technology, any phoneme string corresponding to the scene can be obtained from the markup language information. A device capable of performing instructions and searching can be realized, and the problem can be solved.

  In HTML, a variable or attribute including a phoneme symbol string is provided in the target link or CGI notation, or a range surrounded by a specific tag is converted into a phoneme string and embedded as a tag variable or attribute. , Provide variables and attributes for each table element of the table tag surrounding the product you want to select and give each element tag a name as a variable or attribute with a phoneme string symbol, or give a phoneme string as a variable or attribute for a form tag or input tag, Based on a given phoneme string, the problem can be solved by a method of transmitting information or transitioning to the next page.

  Also, a phoneme string or phoneme string string may be distributed by RSS, ID is associated with a keyword using a tag, and a CSV file is provided as recognition dictionary information that associates the ID with the phoneme string / phoneme string string. You can use a method that identifies the keywords that are to be recognized in, an image recognition dictionary such as a face or fingerprint, a recognition dictionary with proper nouns using phonetic or phonetic symbol strings, and for each speaker Individual authentication using secret words may be performed by associating with an acoustic model based on phonemes or phoneme pieces.

  In this way, the contents of the recognition dictionary based on phonemes and phonemes are acquired from the outside as markup language attributes, arbitrary tags, and dictionary files, thereby enabling operation of the information processing apparatus and solving problems. Can do.

  That is, since words that are not clearly included in the content information are not included in the recognition dictionary unless specified, the probability of misrecognition is reduced, voice operation is performed, and device control by phoneme strings or phoneme string strings The existing markup language can be used for personal authentication with high versatility, and for information exchange between information processing devices. A highly convenient user interface can be realized by adding notations with phonemes and phonemes, and using dictionary information with phonemes and phonemes associated with or associated with markup languages and contents.

1 is a block diagram of an information processing apparatus using the present invention. The figure which showed an example of the data structure of recognition dictionary information. The figure which showed the operation | movement flow of the phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure which showed the operation | movement flow of recognition dictionary update processing. The figure which showed the data structure from which recognition dictionary information differs. The figure which showed the operation | movement flow of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure which showed the operation | movement flow at the time of applying to a server client model. The figure which showed the operation | movement flow at the time of applying to a server client model. The figure for demonstrating the modification in this embodiment. The figure for demonstrating the modification in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 10 Control part 20 Storage part 202 Content information 204 Phonetic symbol conversion table 206 Recognition dictionary information 208 Phonetic symbol addition program 210 Recognition dictionary information update program 212 Voice operation program 30 Communication part 40 Input / output part 50 Operation part 60 Display section

  In the present invention, a markup language notation used for content information is changed and saved, or saved and used, or the changed information is used as it is, and a change has been added. It is possible to configure an information processing apparatus that is used for a distribution apparatus that distributes information and a reception terminal that is received and used for recognition and operations and responses associated with recognition. More specifically, as shown in the example of XML or HTML, a method of saving, changing, and distributing information by changing information written in an existing markup language and adding a tag or adding a variable or attribute And receiving the information and operating the information processing apparatus.

<Example of content information>
First, the content and content information to be searched and indexed using the present invention will be described. The content is exclusively a movie, a drama, a photo, a news report, an animation, an illustration, a painting, music, a promotional video, It is generally well known that novels, magazines, games, papers, textbooks, dictionaries, books, comics, catalogs, posters, broadcast program information, etc., but in the present invention, public information, map information, products Information, sales information, advertising information, reservation status, viewing status, viewing status, road status information and questionnaires, surveillance camera images, satellite photos, blogs, models, dolls, robots, and cameras equipped in these devices・ Information obtained by microphone / sensor input, etc., names of such information, status and situation, abstract concepts, superordinate concepts and subordinate concepts It may include information that expand the designation about the symbol string by phonemes or phoneme.

  In addition, there are video time-series changes, audio time-series changes, texts that expect readers' reading positions in time-series, electronic information in markup language notation in HTML, search index information generated by them, and the like. Alternatively, the reading position may be interpreted as a time axis, and a punctuation mark, sentence, chapter, or sentence may be captured as a frame.

  Also, meta information attached to content, EPG and BML as documents and text information by character information, musical scale as musical score information, general still images and moving images, polygon data, vector data and texture data as three-dimensional information And motion data (motion data), still images and moving images based on visualization numerical data, content information for the purpose of advertisement and advertisement, etc., and natural information including visual information, auditory information, text information, and sensor information It is comprised by.

  A method of recognizing the content of a content using a <Spoken Content DS> tag describing a phoneme (phone) lattice such as MPEG7 used conventionally and adding a phoneme sequence has been proposed. Is indexed by a symbol string based on the recognition of audio information generated in the content, so that the user can search for the title and performer of the content by voice operation, phoneme notation and phoneme notation Information is not provided by notation using phonetic symbols and phonetic symbols.

  For this reason, unspecified words and proper nouns related to the names and expressions of content titles and performers cannot always be used for speech recognition, and therefore are related to content information such as scene descriptions, titles, and performer names as in the present invention. Character information is written together by expanding the phoneme, and as a variable or attribute of the tag, an identifier based on a phonetic symbol string, phoneme symbol string, syllable symbol string, or any phonetic symbol or phonetic symbol is embedded in the MPEG information. Use of phoneme recognition technology for speech recognition can be achieved.

  In other words, if the part enclosed by the tag that is the target of speech processing in markup language is an arbitrary character string, the character string is expanded into a phoneme symbol string or phoneme symbol string, and a phoneme symbol or phoneme symbol is used. It can be used for recognition, and it is evaluated by matching the phoneme symbol string and phoneme symbol string recognized from the user's utterance, or by converting the uttered phoneme into arbitrary phonetic characters. The phonetic characters may be matched, or the phoneme symbol string based on the user's utterance recognition result may be evaluated to be the user's operation target or search target. In addition, any character or symbol such as an at sign or a key bracket described in ideograms may be converted to a phoneme symbol or phoneme symbol by an appropriate phonetic symbol string, or a character string from which multiple utterances can be estimated If so, a plurality of phoneme strings, phoneme string strings, and syllable symbol strings may be given as in conventional speech recognition.

  Then, the recognized phoneme sequence or phoneme sequence is given to the database as a query, searched by a symbol sequence matching method such as DP or HMM, and added to the search result so that it can be viewed by the user by adding the phoneme sequence or phoneme sequence. The search result is presented, the product is selected based on the phoneme sequence included in the search result, and the phoneme sequence and phoneme sequence for performing the charge and purchase procedure are detected from the recognition dictionary with the recognition. In this way, a series of processes associated with sales can be performed, and passwords can be combined with phoneme recognition dictionaries composed of user utterance voice features and fingerprint dictionaries such as fingerprints, irises, faces and palm prints. By performing the authentication and charging with, it is possible to realize a search / browse / sales / authentication / billing procedure for goods and content information such as goods and rights.

  In this way, an identifier or identifier string recognition dictionary based on phonetic symbols necessary for evaluating an arbitrary word to be acquired as a recognition result in accordance with a position in information such as a content reproduction location, page, or display location. Can be used to create a highly versatile dictionary structure for unspecified words in a variety of usage environments. Words recognized based on a phonetic symbol recognition dictionary using dynamically constructed phonemes and phonemes By presenting, performing arbitrary processing, acquiring the URL of the advertisement, presenting the advertisement, and operating the device, information that is highly convenient for content and advertisement distribution to the user Specifying search conditions and sending them by using phoneme strings and phoneme string sequences as variables used for CGI processing posts and get in the Internet environment such as presentation and Web Or, it is possible to and go the Web page switching and operation.

  The procedure for expanding Japanese into phonemes is well known, and after converting it to “kana notation” using a “split writing” program that converts kanji-kana mixed notation obtained from ideograms into phonograms, There is a method of constructing a symbol string to be used for recognition by using phonetic symbol conversion and phoneme symbol conversion using phonetic symbols such as `` Roman characters '' accompanying `` notation '', and there is also a method of describing with syllable symbols in the same procedure .

  If it is English, it can be converted into a phoneme symbol string using an English phoneme symbol or a phonetic symbol, or converted into a phoneme symbol string using an international phoneme symbol. Appropriate phonetic symbols and phoneme symbols may be used, and there are pronunciation dictionaries in various languages, so phonemes and phonetic symbols as phonogram identifiers are time-sequentially based on phonetic symbols according to the language. Distributing information using markup languages based on arbitrary phonetic symbols by dissociating phonemes as disassembled identifiers and expressing them as appropriate character codes in association with numbers It becomes possible.

  At this time, if necessary, the phoneme symbol string may be converted into a phoneme symbol string to improve the convenience of the search. The environmental sound identifier, the scale identifier, the image identifier, and the motion identifier are assigned to each environment. It may be a sound lattice or a scale lattice, a section of image identifiers or motion identifiers may be provided in the MPEG stream, and a phoneme sequence or a phoneme segment sequence may be given based on pronunciation related to the names of these identifiers.

Next, a more specific procedure will be described with reference to the drawings.
〔Device configuration〕
First, the apparatus configuration of the information processing apparatus 1 when the present invention is applied will be described with reference to FIG. Here, the information processing apparatus 1 is an apparatus that is realized by each information processing device such as a general-purpose computer, a dedicated terminal, and a portable mobile terminal.

  As illustrated in FIG. 1, the information processing apparatus 1 includes a control unit 10, a storage unit 20, a communication unit 30, an input / output unit 40, an operation unit 50, and a display unit 60. . Here, each functional unit is connected to the control unit 10 via a bus. Note that the operation unit 50 and the display unit 60 may be arbitrarily removable devices.

  First, the communication unit 30 is a functional unit for exchanging information with other devices via a LAN (Local Area Network) or a communication network such as the Internet. Here, the communication unit 30 is generally configured by a device capable of transmitting and / or receiving content information such as Ethernet (registered trademark), a modem, a wireless LAN, and a cable television device.

  Next, the input / output unit 40 is a functional unit for inputting / outputting information from / to another device or from the outside, such as an input device such as a microphone, a scanner, a capture board, a camera, or sensors, a speaker, a printer, It consists of output devices such as devices and display devices.

  The storage unit 20 is a functional unit that acquires and stores information in the information processing apparatus 1 and stores a program executed by the control unit 10. The storage unit 20 includes a ROM or RAM as a semiconductor storage element, a hard disk or magnetic tape as a magnetic storage medium, a CD (Compact Disk) or a DVD (Digital Versatile Disk) as an optical storage medium, or the like.

  Specifically, the storage unit 20 stores content information 202, a phonetic symbol conversion table 204, and recognition dictionary information 206, a phonetic symbol addition program 208, a recognition dictionary information update program 210, and the like. The voice operation program 212 is stored.

  The content information 202 stores content acquired from the outside via the communication unit 30 and content input via the input / output unit 40. The phonetic symbol conversion table 204 is a table that is referred to when the content information is converted into phonetic symbols. For example, a character string and a phonetic symbol such as a phoneme are associated with each other and stored. It is a table.

  The recognition dictionary information 206 is information that stores a relationship between a word and a phoneme sequence, a phoneme segment sequence, and the like (hereinafter, these phoneme sequences are indicated as phonetic symbols). For example, as shown in FIG. 9, an item “Title”, a “profit campaign” as a target word, and “o / t / o / k / u /” expanded from the target word as a phoneme string (phonetic symbol). “ky / a…” is stored in association with each other. The recognition dictionary information 206 may register proper nouns such as “product name”, “product nickname”, and “phoneme string based on product nickname” in addition to items, etc., and may be registered in a general language dictionary. A recognition dictionary that realizes a variety of recognitions is constructed by dynamically exchanging words including exclamation words and misunderstandings that are not used by phoneme strings or phoneme string strings.

  The operation unit 50 is a functional unit that receives an operation input from a user, and includes an input device that inputs information associated with an operation such as a keyboard, a mouse, a camera, and a remote controller (including wireless). The display unit 60 is a functional unit that outputs information output by the information processing apparatus 1 so that the user can visually recognize the information. The display unit 60 is configured using a display device that performs display related to operations including a display and a projector. ing.

  The control unit 10 calls various programs stored in the storage unit 20 to execute a process for realizing a function corresponding to the program or to control each functional unit of the information processing apparatus 1. It is carried out.

  The control unit 10 reads out the phonetic symbol addition program 208 from the storage unit 20 and executes it, thereby realizing a phonetic symbol addition process to be described later. Moreover, the recognition dictionary information update program 210 mentioned later is implement | achieved by reading and executing the recognition dictionary information update program 210 from the memory | storage part 20. FIG. Further, the voice operation processing is realized by reading and executing the voice operation program 212.

  In addition, the control unit 10 executes a program to acquire phoneme / phoneme recognition processing, tag information acquisition, tag identifier acquisition, phoneme string / phoneme string array, and phoneme / phoneme recognition of user utterance speech. Can select words based on similarity evaluation between phoneme sequence / phoneme segment sequence and phoneme sequence / phoneme segment sequence associated with dictionary registration information, and acquire speech waveform from input / output unit using microphone It may be used for speech recognition, or information may be provided to the user by speech synthesis using a phoneme sequence or phoneme segment sequence obtained by the present invention using a speaker.

  Note that the control unit 10 is usually configured using a CPU (Central Processor Unit), DSP, ASIC, or the like, and can be realized by arbitrarily combining them.

<Operation>
Next, each operation process executed by the information processing apparatus 1 will be described.

<Phonetic symbol addition processing>
First, the phonetic symbol addition processing will be described with reference to FIG. FIG. 3 is an operation flow for explaining the phonetic symbol addition processing, which is realized by the control unit 10 reading and executing the phonetic symbol addition program 208 in the storage unit 20.

  First, the control unit 10 acquires content information 202 stored by being received by the communication unit 30 or input by the input / output unit 40 (step S301).

  Next, a development target character string is detected from the read content information 202 (step S302). Here, the expansion target character string is a character string (information) for identifying a change in display control method. For example, in the case of a markup language, a tag <A> indicating a link or a title It is a tag <TITLE> indicating A character string to be expanded that is expanded into phonetic symbols such as phonemes and phoneme pieces is detected for the range between the tags.

  Next, the expansion target character string is expanded into a phoneme string or a phoneme string string (phonetic symbol) composed of phonetic symbols accompanying the utterance (step S303). Thereby, for example, the title and the name of the link destination are converted into phonetic symbols. When this expanded character string is converted into phonetic symbols, the character string is acquired by referring to other attributes such as the ALT attribute and ID attribute that are information contained in the tag, and converted into a phoneme string or phoneme string string. To compose recognition dictionary information using phonetic symbol strings for dictionary registration, compose phoneme and phoneme string sequences from image file names, music file names, video file names and document file names, A phoneme or phoneme string sequence is constructed using the tag attributes and the text string sandwiched between tags, which are described in files, music files, video files, and document files, and phoneme strings are generated from the text strings sandwiched between tags. Or a phoneme segment string, or by using link information associated with the tag as an attribute, the name of the file at the link destination or the character information contained in the file is sandwiched between the tag or tag attribute or tag String Or configure the phoneme sequence and phoneme sequence by use, how to configure the phonetic symbol string registered in the dictionary by using an arbitrary method such as is conceivable.

  Specifically, the phonetic symbols are converted into phonetic symbols using the phonetic symbol conversion table 204. For example, a character string “main” surrounded by title tags is converted to “m / e / i / n /” as a phonetic symbol by referring to the phonetic symbol conversion table 204.

  In addition, there is a case where the attribute of the phonetic symbol expansion is already given as a tag of the content information itself without performing such development to the phonetic symbol, and the recognition table by the phonetic symbol string to be used for recognition is used. The phonetic symbol string may be configured. For example, after performing step S401 for acquiring the content information, the “pronounce attribute” is detected from the accompanying meta information from the markup language information as shown in FIGS. Step S402 is performed, and a phonetic symbol string composed of phonemes and phoneme pieces described as a variable of the detected “pronounce attribute” is extracted, and the extracted phonetic symbol string and “pronounce attribute” are detected. By executing step S403 of associating with meta information and registering it as dictionary information, a table that uses a voice operation program or expresses an utterance sound that can be recognized by the information processing apparatus A dynamic phonetic symbol string that specifies any processing, procedure, or operation by specifying a processing content or transition destination page using a tag or CGI as meta information associated with the phonetic symbol string, Realize recognition using.

  As a result, the phonetic symbol storage process is executed, and the recognized phonetic symbol string used for phonetic symbol recognition is stored (step S304). The phonetic symbol storage processing is processing for storing the phonetic symbols used for recognition converted in step S303. For example, the phonetic symbols that are already recorded as attributes in the respective tags are extracted, A process (step S304a) of adding a phonetic symbol (phoneme string or phoneme fragment string) as a new attribute by expanding from the sandwiched character string, and a tag or attribute indicating that each tag is a speech recognition target Processing to add to content information (step S304b), processing to configure and update recognition dictionary information 206 by separating proper nouns to be recognized, converting them into phonetic symbols, and forming recognition phonetic symbol strings ( Step S304c) is executed. Thus, processing for clarifying the content information and the recognized phonetic symbol string as a phonetic symbol string including a phoneme string or a phoneme string string of a word to be used for recognition is performed.

  Then, the control unit 10 updates and saves the changed content information 202, and updates and saves recognition dictionary information for phonetic symbol recognition using phonemes and phonemes composed of associated recognition phonetic symbol strings. (Step S305). This makes it possible to use the content information changed to the recognition of the user's utterance and the distribution via the communication unit.

  Although the above-described processing has been described as being executed by the information processing device 1, the processing load of information accompanying conversion to a phoneme string on the receiving side is executed by the distribution device (server) side that distributes content information. You may make it reduce. When the distribution apparatus executes, the distribution apparatus distributes content information accompanied by voice control information in response to a call of content information from a user. Therefore, the information processing device 1 (terminal device) can acquire phoneme information classified according to the page and frame of the content by the information processing device, and can use voice of arbitrary words with less restrictions.

  Here, an operation example when the phonetic symbol addition processing is executed will be described with reference to the drawings. First, FIG. 4 is a diagram showing the state of the content information 202 acquired by the information processing apparatus 1. By executing step S301, the content information 202 is acquired from the communication unit 30 or the input / output unit 40 and stored in the storage unit 20.

  And the information relevant to the tag used as the evaluation object by the phonetic symbol (phoneme sequence / phoneme segment sequence) is detected (step S302). Note that the information in FIG. 4 is an example of content information using an RSS item section when the extraction process is executed in step S302. FIG. 5 shows an example in which a target character string is extracted from the item section and converted. Or it is described as FIG.

  When a tag that is the target of the expansion target character string is detected from the tags included in the acquired content information, the character string in the range specified by the tag is detected. For example, in FIG. 4, a “profit campaign” sandwiched between tags “<title>” and “</ title>” meaning a title is detected as a character string to be expanded into a phonetic symbol string. At this time, unnecessary parenthesis symbols may be deleted, and an arbitrary title character string designated by the distribution side can be acquired by extracting the character string.

  Then, the acquired character string is confirmed and converted into a phonetic symbol string according to the pronunciation of the character string using the phonetic symbol conversion table 204. Then, as shown in FIG. 5, for example, a new pronouncation attribute is added as an attribute or variable to the tag described in the original content information 202 to add a phonetic symbol string (step S304a), and FIG. As shown in the figure, processing for newly setting <pronounce> to </ pronounce> tags, a word and a command recognized as a phonetic symbol are associated and stored as recognition dictionary information 206, and the recognition dictionary information 206 is stored as content information 202. A process of describing and associating a URL with a <META> tag or the like as the acquisition destination of the recognition dictionary information 206 (step S304c) or the like is executed, and phonogram symbol string information used for phonogram recognition is added to or associated with content information It becomes possible.

  Then, the operation based on phonetic symbols (phonemes, phonemes, phonetic symbols, phonetic symbols) by distributing the content information 202 with the above-mentioned changes directly to other terminals or using it in the device. Will be able to.

  In addition, for example, in MPEG7, as shown in FIG. 7, a “<Pronounse DS>” tag is added to describe a phoneme string of content type, or “Rainsound” is generated as an environmental sound behind. The phonetic symbol string “pronounce =” t / o / m / u ”” is added as an attribute to the notation about the performer that can be placed in the phrase tag. In addition, HTML presents an example of a configuration associated with a button or link, and a range between arbitrary tags is searched as a keyword, which is useful for browsing and searching for content, and pronunciation for operation is a phoneme symbol. It may be provided as a sequence, or a phonetic symbol sequence such as acquired phonemes or phonemes may be used in a word dictionary of speech phonemes or speech phonemes for speech synthesis speech in the information processing apparatus 1.

  Thus, according to the phonetic symbol addition processing, a table for incorporation into the phonetic symbol dictionary used for phonetic symbol recognition by adding a phonetic symbol for performing voice operation based on the acquired content information. Content information including phonetic symbol string information can be configured.

<Recognition dictionary information update process>
Next, a recognition dictionary information update process in the case where a phonetic symbol has already been added to the content information 202 will be described with reference to FIG. FIG. 8 is a diagram illustrating an operation flow related to the recognition dictionary information update process, which is a process realized when the control unit 110 executes the recognition dictionary information update program 210 of the storage unit 120.

  First, the control unit 10 acquires content information 202 (step S401). Next, the control unit 10 extracts a phonetic symbol string from the read content information 202 (step S402). In the present embodiment, by extracting a tag (a portion sandwiched between “<” and “>”) included in the content information 202, a tag including a phonetic symbol string is specified and extracted. Become.

  For example, the control unit 10 extracts the “pronounce attribute” of the title tag “<TITLE>” to extract the phoneme symbol string “o / t / o / k / u... To do. Then, the extracted phoneme string is stored as a page title and registered in the recognition dictionary information 206 (step S403).

  In addition, by switching these dictionaries according to the display contents that change according to the page switching, it is possible to avoid erroneous recognition due to words that are not in the display contents, thereby improving the speech recognition rate and improving operability. Then, a necessary phoneme string may be acquired from a URL or the like of dictionary information associated with content information using an arbitrary tag or character string, and dictionary information for associating with a recognized word or control method may be updated.

  In addition, when the phoneme sequence or phoneme sequence is not described in the content information to be distributed or the related phoneme dictionary is not associated, the phoneme or phoneme sequence is performed according to the procedure for embedding the phoneme sequence / phoneme sequence. May be constructed from the content to construct dictionary information, and the constructed dictionary information may be reused if it can be used by detecting whether or not the same word is used. good.

  If the control command does not change the phoneme symbol string when the control recognition dictionary is configured, the control command using the dictionary in which the ID related to the control command, the command word, and the phoneme sequence are associated as shown in FIG. After the distribution of the content information in which the ID for specifying the word is described as the command discrimination ID and the recording to the storage medium, the phoneme is used in the information associated with the content information received from the communication unit or the content information acquired from the storage medium. The command word is specified from the command discrimination ID described in the location where the information or phoneme information is described, and the phoneme string or phoneme string is converted by performing the conversion function from the specified command word to the phoneme or phoneme. Sounds at the time of transmission that tend to be redundant by configuring and using for recognition, or by using a hash value based on a phoneme sequence or phoneme segment sequence associated with the control command as an instruction discrimination ID To shorten the string representation may be working to improve the communication efficiency.

  In addition, regarding the content information 202 acquired via the storage medium or the communication means and stored in the storage unit, when the conversion to the phonetic symbol or addition is not performed, the content is interpreted by the above-described method and the information processing is performed. Conversion to a phonetic symbol by an identifier sequence corresponding to the device 1, and if the content of the content information 202 has already been described, converted or updated, the content of the content information 202 is converted or updated. You don't have to.

  In addition, these conversions are information obtained from an external storage medium by a single device, whether it is converted and distributed on the server side depending on the situation of the content information distributor or user, even if it is properly converted by the client May be converted so that it can be used by the device itself, or may be converted using relay means such as a gateway or a router.

<Voice operation processing>
Next, the voice operation process will be described with reference to FIG. First, the control unit 10 acquires content information acquired in the communication unit 30 or the input / output unit 40 and content information 202 stored in the storage unit 20 (step S501).

  Next, phonetic symbols composed of phonemes and phoneme pieces are extracted from the acquired content information (step S502). Then, the recognition dictionary information 206 is updated and registered based on the extracted phonetic symbols (step S503).

  Next, it waits until there is a voice input based on the utterance from the user from the input / output unit 40 (step S504; No). Here, when a voice input is made by the user (step S504; Yes), the control unit 10 extracts a feature amount of the input voice of the user (step S505). Then, phonetic symbols such as phonemes and phoneme segments are recognized from the extracted feature quantities and converted into phonetic symbols (step S506).

  Then, a coincidence evaluation is performed to determine how much the phonetic symbols converted in step S506 match the phonetic symbols previously registered in the recognition dictionary (step S507). In this coincidence evaluation, the degree of coincidence with a standard model, standard parameter, or standard template of sound or speech stored in the storage unit of the present apparatus is evaluated by an evaluation function, and a phonetic symbol as an evaluation result is specified. Then, the phonetic symbol string is specified by obtaining a plurality of phonetic symbols specified based on the coincidence evaluation in time series. Then, the phonetic symbol string having the highest similarity to the identified phonetic symbol string is set as the phonetic symbol recognition result, and device operation and search processing are executed in accordance with the information associated with the recognition result (step S508). ).

  Here, the process associated with the recognition result is, for example, generation of a character string with proper nouns realized by recognition of a phonetic symbol string using the present invention, execution of a search for each operation command, information or product, and phonetic For example, information is presented to a series of users identified in accordance with the recognition of the symbol string, and an operation is instructed by the user. Specifically, web browser page switching, TV and video operations, robots, navigation devices, computers, audiovisual equipment, cooking appliances, washing machines, air conditioners, responses by voice, text, images, and video, search conditions Is a series of processes and operations such as storing, changing, registering and deleting information presented by the information processing apparatus, specifying and browsing advertisements and program contents associated with recognition results, and personal authentication based on keywords and utterance characteristics. In addition, by using an image recognition dictionary such as face or fingerprint, a recognition dictionary with proper nouns using phonetic symbol strings based on phonemes or phonemes, and an acoustic model based on phonemes or phonemes for each speaker, Authentication may be performed, and charging and service selection can be performed along with the authentication.

  Specifically, by recognizing or recognizing a recognizable word by uttering a word with a phoneme sequence or phoneme sequence registered in the recognition dictionary for speech synthesis to answer a user's question. Performing any operation according to the result, presenting the recognized character string or word string according to the recognition result, or performing an advertisement associated with the phoneme string or phoneme string string, It becomes possible by combining with voice recognition technology.

  Then, it is determined whether or not the next voice input is performed (step S509). If the voice is input again (step S509; Yes), the process returns to step S504 as a process for waiting for the voice to be input. If no voice is input (step S509; No), it is determined whether or not to acquire the next content information (step S510). Here, when acquiring the next content information (step S510; Yes), in order to acquire a new content, a process is repeatedly performed from step S501. If new content information is not acquired (step S510; No), a series of processes such as ending the process and waiting for the user's utterance are performed.

  That is, the apparatus using the present invention marks a place where a user can perform a voice operation by using an identifier by a phonetic symbol such as a phoneme and a phoneme from the information in the acquired markup language and a feature amount for specifying the identifier. Acquired from up-language information, and if necessary, it can be used for personal authentication by acquiring and combining images and actions such as fingerprints, facial expressions, palm prints, etc. It can also be used for operation.

  A selection process conventionally performed by mouse operation is performed based on identifiers and feature values obtained by user's utterances and inputs, and focus is given to any row or column, link or operation button of the table tag, or the cursor is moved. Overlapping, issuing events associated with these operations from the operating system to the browser, controlling other devices using communication means such as infrared, LAN and telephone lines, and agents according to recognized words By changing the robot-corresponding operation, a series of processes associated with recognition of phonetic symbol strings such as phonemes and phoneme pieces can be performed.

  The content information acquired in step S501 detects the “pronounce” attribute information in the tag (step S502) and registers it in the recognition dictionary information 206 (step S503). At this time, by registering what kind of tag is associated with the phonetic symbol string for recognition, each tag in the browser is processed by combining the display position and display item of the screen configuration information with the preceding and following tags. Space for latitude / longitude by specifying the display position at the time of mapping, specifying the scene position by associating it with the tag indicating the scene, title and time-series position in the content information in MPEG7, etc. The physical position can be specified by associating it with the position information, the place name, the area information, or the store information.

  Next, an example of the voice operation process will be described with reference to FIGS. When the user pronounces “first line, drafter” i / ch / i / gy / o / u / m / using the table tag's “pronounce” attribute in the top row in table tag column selection A match between the phoneme sequence such as e and k / i / a / n / sh / a and the user's utterance phoneme is determined from the recognition dictionary. As a result, the “drafter” column in the phoneme column of the utterance is selected, and the “first row” in the tag for designating the row is selected, so that the “first drafter” is selected (step) S506).

  If a phoneme string provided as an attribute for each HTML transmission button shown in FIG. 11 is detected, transmission is performed according to the designation of the form tag, or the phoneme string of the user utterance is recognized and “ts / u If the degree of coincidence with “/ g / i / e” is high, the web browsing process can be performed by moving to the link destination. Then, when moving between pages, an agent such as an agent or a robot that responds to the user such as providing a question such as “Do you want to move?” To the user by voice, character string, image, or video (step S506). Dialogue processing may be performed, image recognition dictionaries such as faces and fingerprints, recognition dictionaries with proper nouns using phonetic symbols and phonemes and acoustic models based on phonemes and phonemes for each speaker It is also possible to perform personal authentication using secret words by associating.

  Next, when the browser image viewed from the user is presented and explained, when FIG. 11 is displayed with an HTML browser, it becomes as shown in FIG. 12. Here, when the utterance “Ichiyoume” is spoken, the attribute “i / ch” is displayed. The focus B300 is set in the first line as shown in FIG. 13 in accordance with the phoneme string of the “pronounced attribute” / i / g / y / o / u / m / e ”, and the phoneme string of the“ pronounced ”attribute is set in accordance with the pronunciation“ shosai ” Is selected as shown in FIG. 14 for the button B302 in which “sh / o / u / s / a / i” is written, a click process is performed and a form is transmitted (step S506).

  At this time, if a lot of detailed buttons are displayed, it will not be understood which button, so it will be announced from the contents of the announcement “How many lines are there from the device side” or “How many draft numbers?” Interactive processing by voice or display may be performed by asking questions so that target phoneme sequences or phoneme segment sequences that can be easily analogized can be obtained. It may be provided in XML.

  A browser that has received such an event performs processing set in advance according to those events. For example, in the case of an <a> tag in HTML, a specified link destination is accessed and an arbitrary web page, image, or video is displayed. , Music and product information, "<INPUT TYPE =" button ">", "<INPUT TYPE =" submit ">", "<INPUT TYPE =" image ">", "<BUTTON TYPE =" ~ If it is an operation input tag such as “>”, the HTML process is shifted to a state where the corresponding button or image is pressed, and if it is a “<FRAME>” tag, a frame is selected according to the name of the frame, or “ If it is a <SELECT> tag, the focus shifts to the select tag with phonetic symbols and phonetic symbol strings generated by the user, configures selection candidates from the option tags, selects any item, If it ’s a “<HR>” or “<A NAME="”>” tag, Scroll to the target line with the related tag using the phonetic symbol string associated with the variable or attribute of the tag as a variable or attribute, or if the tag is a <TITLE> tag, It is possible to execute a process of expanding the phonetic symbol string and storing it in the bookmark in association with its own URL (step S506).

  Of course, recognition by these phonetic symbols may be performed in cooperation with a script or the like, or "<EMBED SRC =" ">", "<OBJECT>", "<APPLET CODE =" " Add arbitrary extension functions with tags such as ``> '', give them as variables and attributes to those programs, and identifiers by phonemes, phonemes and phonetic symbols that are instructions and phonetic symbols for operating them externally It may be used for columns and feature quantities, for information used in conjunction with scripts, or for script control conditions.

  For example, in the case of RSS or MPEG7 using XML or RDF, in order to select an item section as shown in FIG. 4, addition of variables and attributes as shown in FIG. 5 or addition of tags as shown in FIG. You can use a method of adding changes, adding a “pronounce” element to the element type based on the “Dublin Core” of RDF, and naming the scene names, actors, and cast names with phonemes and phonemes. "type" and "img-position" elements are added to describe the display position and feature amount of the image, "motion" elements are added to describe the operation in the screen, and "env-sound" elements are added Then, environmental sound identifiers and feature quantities may be described.

  Further, FIG. 11 shows a syllable notation tag for phonetic pronunciation that is a phonetic character and a tag for phoneme, but these tags may be phoneme segments, image identifiers by input video, etc. For example, when a user's voice or facial expression is recognized as being angry, or when a specific identifier is detected from within an image, processing such as presentation of a script or content, or movement to a link destination is performed. May be.

  These tags, attributes, and elements are generally evaluated by matching character strings in a device that interprets them, and functions, processes, programs, etc. that perform processing recorded in the information processing device according to those character strings. Providing services with information based on their tags and attributes.

  For phoneme and phoneme recognition functions, a phoneme sequence or phoneme sequence is provided and registered in the recognition target dictionary, or if it is another identifier or feature, the evaluation coefficient as a detection result is changed. To output instruction information to peripheral devices, to prepare allophone synonyms with a dictionary structure, and to register a phonetic symbol string to register the allophone synonyms in the dictionary. A column may be described so that it can be discriminated by a boundary symbol, or a process for correcting a result obtained by recognition may be performed.

  Also, simply specify a character string to be displayed by placing a character string to be displayed between pronunciation tags, and use a phoneme or phoneme using a phonetic symbol for pronunciation of a character string in another language such as a kanji mixed sentence, an English sentence, or a Chinese sentence. It can be converted into a symbol string and used for recognition, command control, detection, and search. Not only the phonetic symbols are written in alphabets, they are also replaced with numerical values such as ASCII codes and evidic codes. It may be described in the language.

  Also, it can be used as a tool for creating and producing movies and programs by CG etc. by controlling video and dialogue related to features and identifiers associated with words in this way, screen features and display objects. It is also possible to recognize the utterance status while browsing the content, use the content evaluation based on the vote by the voice operation of the user and the number of browsing, etc. You may evaluate a movie and a program based on the correlation with an identifier.

<Server / client model>
In the above-described mechanism, a search procedure using a markup language may be implemented by a server / client model. Specifically, FIG. 15 shows a state transition in processing in the server / client model.

  First, a terminal device serving as a client generates a query. The query generation method may be a general method using character string input, a method using voice input, or a method of displaying an image and using the feature amount as a query.

  Then, based on the generated query, the distribution device serving as a server searches for an appropriate one, and the distribution base station distributes the search result list information using the present invention to the terminal device according to the search result. Then, the terminal device interprets the markup language of the acquired information, converts the character string in the range sandwiched between specific tags into the above-mentioned identifiers such as phonemes and phonemes, and converts them into voice input information spoken by the user. Therefore, phonemes and phoneme pieces are acquired to form phonetic symbol strings such as phoneme strings and phoneme string strings, and matching processing based on the phoneme strings and phoneme string strings is performed.

  And after performing each process according to operations with high degree of coincidence, keywords, and identifier strings, construct a query with the specified identifier string, and send those queries to the distribution base station to perform a search Then, a search with voice control using a markup language is performed. At this time, the constructed query may be used for a search by a single device.

  In FIG. 16, the insertion and setting of identifier symbol strings such as phonemes and phonemes for speech processing are performed by the markup language interpretation on the terminal side. Operation and processing using the present invention by manually inserting in advance, constructing and inserting identifiers on distribution base station side devices and devices linked with them, or using a single information processing device It is also possible to add variables and attributes, add tags, change the contents of markup language information, or change, add, or delete dictionaries related to identifiers and feature quantities.

  In addition, regarding a new identifier combination generated by the present invention, a search is performed by giving a word based on an arbitrary name, a symbol string using a phoneme or phoneme is given to an arbitrary name, and voice control is performed, For example, a symbol string using phonemes or phonemes may be given as a keyword for the operation, and the phonetic symbol string may be operated by associating with the advertisement, or an advertisement attribute is added to utter the URL of the related advertisement. An advertisement and an utterance symbol string may be associated with each other by describing the symbol attribute in the same tag.

  In addition, identifiers related to images displayed in the browser, and phonemes and phonemes of keywords to be controlled are converted to be easily usable on terminals by using symbol strings and IDs obtained by compressing identifier strings and identifier strings. Sending information and using the voice in the device without interpreting the markup language can be easily considered, such dictionary information can be acquired via a communication line with a remote control or mobile phone, acquired by e-mail, etc. A highly convenient operating environment may be configured by downloading from the above device.

  In addition, a recognition dictionary based on the file name may be configured by describing the file name as a phoneme string, or the file name is set by the phoneme string / phoneme string sequence and the phoneme / phoneme segment recognition is used in the markup language. It is possible to select various information, and in addition to the stock price search by coupon number or company name or the product search by JAN code, the search by product name, performer name, company name or region name is also performed. You can change the phoneme dictionary according to the location and device, change the phoneme dictionary according to the page, or change the content image, the text in the page unit, and the frame or moving image 1 The phoneme dictionary may be changed according to scene units that span frames as frames or multiple frames of moving images.

  In addition, when indexing according to the present invention is performed for an information format having a chunk header such as the RIFF format as shown in FIG. 17, a tag such as “PRON” is arbitrarily provided as a chunk header, and a phoneme string or phoneme segment is provided. Columns may be written, and if the contents are regular files, general metadata such as the file name, production date and time, producers, etc., or 2D / 3D images, the names of display objects and people, You can list the phonemes and phonemes that accompany the name of the parts, if they are audio files, you can list the phonemes and phonemes of the appearing speech, or you can write the lyrics and titles of music files in phonemes and phonemes, Phonemes / phonemes may be written in the free description area, or may be used for searching.

<Modification>
In this embodiment, the phoneme is mainly used as a phonetic symbol. However, the phoneme portion is changed to a phoneme segment, or the phoneme sequence is a phoneme string of different languages such as an international phoneme symbol, English, or Chinese. Or a selection range or branch content in markup language processing may be configured depending on whether a round image is presented to a computer or a triangular image by an identifier based on the recognized image. , Search based on the presented photos, or perform voice operations by expanding the names associated with the feature quantities of the photos into phoneme strings and phoneme strings and sending and receiving them in markup language and dedicated symbol strings Alternatively, Unicode, JIS code, or ISO code, which is a character encoding method other than ASCII code, may be used, and an arbitrary numerical ID based on phonemes or phonemes is given. And it may be used a unique character code system.

  Further, the identifier string and identifier used in the present invention are scale type, instrument type, mechanical sound type, environmental sound type, image type, face type, facial expression type, person type, action type, landscape type, display position type, character symbol An attribute, a variable name, or an identifier may be designated based on a designation used for identification by one or a plurality of identifiers such as a classification, a label classification, a shape classification, a graphic symbol classification, and a broadcast program classification. The identifiers may be captured as continuously described according to time series transitions, converted into phonemes or phoneme strings based on their names, or used in GET methods or POST in CGI. A search result may be obtained by transmitting those identifiers and identifier strings using a method.

  In this way, the feature names related to speech and their identifiers and identification functions, the feature names and identifiers and identification functions related to still images and moving images, and the mark and language that can be operated by speech by giving attributes and variables to the markup language In addition to searching for content, public information, map information, and the like, the user can realize device control by voice using a phonetic symbol string provided by an information processing device that processes such a markup language. It can be applied to merchandise sales, reservation status, viewing status, questionnaires, surveillance camera images, satellite photos, blogs, robots and equipment control. Note that the search and processing results may be returned from the server to the client using any markup language in response to these requests.

<Procedure example of information processing apparatus used for terminal and base station>
The present invention is also applicable to a processing system using a server and a client related to a base station and a terminal. This device and the terminal are configured as shown in FIG. 18, and are connected via a communication line to acquire information from other devices and distribute information to other devices, so that information related to voice operations can be obtained. It can be exchanged to improve user convenience. The shared line used here is not limited to the Internet but may be used regardless of wired wireless as long as it is a wide area communication network such as a LAN or a telephone line or an indoor communication network, and the target devices are home appliances, remote controllers, robots, It may be a mobile phone or a communication base station, or may be a service such as a web service, a telephone service, or an EPG distribution, and can be implemented for any device or service.

  Further, it is configured by a user terminal, a distribution base station, a device such as a robot controlled by the terminal or the base station, and a remote controller to be controlled, and the remote control and the robot may be used as one form of the terminal or one form of the base station. The user speaks voice to the terminal, and performs an arbitrary processing procedure as described below for recognition processing at the terminal or the base station.

  In the first method, a feature amount is extracted from voice obtained by speech or a captured image, the feature amount is transmitted to a target relay location or base station device, and the base station device that has received the feature amount A phoneme symbol string / phoneme symbol string and other image identifiers are generated according to the feature amount. Then, based on the generated symbol string, a matching control means is selected and executed.

  In the second method, feature amounts are extracted from speech obtained by utterance or captured video, and identifiers associated with recognition such as phoneme symbol strings / phoneme symbol strings and other image identifiers are generated in the terminal. The received symbol string is transmitted to the target relay location or base station apparatus. Then, the base station apparatus to be controlled selects and executes a matching control means based on the received symbol string.

  The third method performs feature amount extraction from voice obtained by speech or captured video, recognizes phoneme strings / phoneme symbol strings, and other image identifiers based on the feature amounts generated in the terminal, The control content is selected based on the recognized symbol string, and transmitted to the base station device that controls the control method and the device that relays information distribution.

  And the 4th method transmits to the base station apparatus which controls the voice waveform and image of the audio | voice obtained by utterance using the terminal, and the imaged image | video as it is, and phoneme symbol string and phoneme symbol in the apparatus to control The system recognizes the sequence and other image identifiers, selects the control means based on the recognized symbol sequence, and the selected relay station or base station apparatus performs the selected control. The same applies to sound and video features and identifiers such as environmental sounds.

  At this time, the terminal may simply transmit only a waveform, transmit a feature amount, transmit a recognized identifier string, or transmit a processing procedure such as a command or a message associated with the identifier string. The client server model may be implemented by changing the configuration of the distribution base station according to the transmission information, and the transmission side and the reception side can transmit and receive each other, and the image related to the identifier described above Features such as voice, voice, and motion are assigned to markup language attributes, and the degree of coincidence between the feature amount extracted from the information provided by the user and the feature amount extracted from the distribution information is evaluated and searched Information processing with arbitrary control and user interaction may be realized by performing recognition, or proper nouns using image recognition dictionaries such as faces and fingerprints and phonetic symbol strings made up of phonemes and phonemes Recognition dictionary with It may be subjected to a personal authentication by the slogan by associating an acoustic model based on the phoneme or phoneme of each finisher.

  In addition, the command dictionary for converting to an associated processing procedure based on the input phoneme sequence or phoneme segment sequence may be on the terminal side or the distribution base station side, and a new control command, media type, format A symbol string such as a phoneme symbol string and an image identifier relating to the type and the device name may be transmitted / received, distributed, or exchanged using a markup language such as XML or HTML, RSS, or CGI, which will be described later.

  A more specific procedure for distributing and exchanging dictionary information will be described. First, by extracting feature quantities and identifiers, or configuring evaluation functions, other terminals and devices in an environment connected to a communication line regardless of any infrared, wireless LAN, telephone line, wired LAN, etc. Exchange information with other classes.

  Next, a case where a phoneme piece is used as processing on the terminal side will be described as an example. The terminal side device analyzes the given voice and converts it into a feature value. Next, the converted feature amount is recognized by various recognition technologies such as HMM and Bayes and converted into an identifier.

  At this time, the converted identifier is information indicating a phoneme, a phoneme piece, and various image identifiers. However, as described elsewhere, if it is a voice, it is a phoneme, an environmental sound, a scale, or an image if it is an image. Or an identifier based on the operation. Then, based on the obtained identifier, a dictionary based on phonemes and phoneme symbol strings is referred to by DP matching to select an arbitrary processing procedure, and the selected processing procedure is transmitted to the target device for control. Thus, it is possible to use the mobile terminal as a remote control using the present invention, or to control home appliances by robots, and the utterance sound notation for smooth communication with the other party at the communication destination An interactive device with a disabled person may be configured by providing a display and a Braille output unit.

  Depending on the CPU performance of the terminal, the information processed in such a procedure can be transmitted as it is without converting natural information such as video and audio into feature values, or can be converted into feature values. It can be sent, sent after being converted into an identifier, sent after the control information is selected, can be selected, and any conversion level can be selected, and the receiving side can process based on information from any state It is configured as a possible receiving side device, and it transmits to the distribution station and control device based on the acquired information, or performs arbitrary processing such as search, recording, mail distribution, machine control, device control based on the acquired information Also good.

  Then, for use in search processing, an identifier string, a character string, and a feature amount that are appropriately used as a query are acquired by recognition, and are transmitted to the distribution-side base station to obtain information according to the query. At this time, advertisements and advertisements may be displayed during the communication waiting time and the search waiting time, and when performing control by voice, a control dictionary is constructed so that control items can be selected by communication. The dictionary information is exchanged / acquired, and the procedure may be performed using the P2P technology, or the information may be sold and distributed.

  In addition, this control command dictionary can be freely renewed and reused by configuring it with arbitrary identifiers such as phonemes and phonemes as described above, feature quantities, and device control information. It may be possible to update trendy search keywords by replacing or reconfiguring dictionary information for search that associates arbitrary identifiers with feature amounts, and the position and configuration of these content information The recognition dictionary information changed accordingly may be a dictionary for face recognition, a dictionary for fingerprint recognition, a dictionary for character recognition, or a dictionary for figure recognition.

  In the control command dictionary, infrared control information to be transmitted to a product that can be controlled by a conventional infrared remote controller is selected as device control information, or a series of operations are continuously performed as a batch process by combining those control information. It may be implemented or only the feature amount information may be transmitted to the speech-controlling information processing apparatus without recognizing the identifier according to the CPU performance of the apparatus.

  Any device capable of providing an infrared remote control signal from a voice information via a conversion dictionary by combining control by an infrared remote controller even with a conventional device that cannot perform voice control by such a method, or a device capable of voice control. In addition to being able to recognize and control commands based on feature quantities and speech waveforms, changing the control dictionary to improve performance, checking with version information of the control dictionary, It is possible to confirm the state of the device.

  In addition, by introducing the server / client model in this way, the server and the client can be divided into arbitrary processing steps, connected by communication, and exchanged arbitrary information between the server and client. Indexing can be realized.

  Also, in order to perform personal authentication by recognizing faces, fingerprints, and voice features, use a phoneme recognition dictionary that uses acoustic models, standard parameters, and standard templates that match the phonetic recognition dictionary information. Therefore, the recognition dictionary with images and sounds can be changed according to the user, and highly versatile personal authentication can be realized. Therefore, a service using various operations and operations such as charging, locking and unlocking keys, selecting a service, granting use, and using a copyrighted work uses the present invention. This can be realized by using an information terminal that performs recognition.

  In addition, using a terminal that performs recognition using the present invention, it is acquired by a client terminal such as a DVD recorder, a network TV, an STB, an HDD recorder, a music recording / playback apparatus, or a video recording / playback apparatus from a core server at a communication destination. Information is transferred to mobile terminals and mobile phones via wireless communication such as infrared communication, FM, VHF frequency band communication, 802.11b, Bluetooth (registered trademark), ZigBee, WiFi, WiMAX, UWB, WUSB (Ultra Wide Band), etc. By providing information, it is possible to use EPG, BML, RSS, teletext data broadcasting, TV video, teletext on mobile terminals and mobile phones, voice input, character string input, swing mobile terminals and mobile phones. The operation of the information terminals, home appliances, information devices, robots, and instructions for control procedures are performed by moving the Terminal or a mobile phone may be subjected to remote operations such as and go an indication of the general consumer electronics as a remote control from the client terminal and information equipment and operation of the robot and control procedures.

  In addition, when a phoneme dictionary based on an attribute extracted in association with an input item such as an HTML FORM tag in information configured in a markup language is registered in the recognition dictionary information 206 in advance, the recognition priority is set. It may be changed to a phoneme dictionary registered in advance, or a recognition target may be limited using a dictionary registered in advance.

  Also, for phonetic symbol strings such as phoneme strings and phoneme string strings based on attribute variables of information configured in markup language, a plurality of phoneme strings and phoneme string phonetic symbol strings that may be recognized simultaneously are written together. Thus, a plurality of recognition dictionary information 206 may be configured, and the same recognition dictionary information 206 may be used for input items having the same attribute variable.

  In addition, a plurality of words that may be recognized using a plurality of phoneme strings, phoneme string strings, and phonetic symbol strings may be described as attribute variables. For example, a classifier such as an arbitrary unit may be represented as a phoneme string, phoneme string string, or table. When it is expressed as a phonetic symbol string, the recognition dictionary information 206 is switched to a number only, switched to a dedicated dictionary according to a menu item, or switched to a limited proper noun dictionary such as a place name or a station name. Also good.

  Further, according to the character code selected to be used for the display based on the markup language, the Bayes discriminant function or HMM used in the step (S506) of converting the speech waveform into an identifier by phonetic symbols such as phonemes and phonemes. Prepare multiple values for each language, such as the standard pattern to be used and the standard template, the values obtained as learning results, the values by eigenvalues / eigenvectors, and the values by covariance matrix. It is possible to support multiple languages by switching to a word standard template, or select a standard template to be used for recognition by acquiring information about the user's information processing device, operating system, or browser-specific language environment from multiple languages You may do it.

  In addition, according to the user's designation, for example, the utterances and dialects that arise due to the difference between the user's native language and native language and the language recognized in the device environment used, such as a standard template when a Russian speaker speaks Chinese The system may be configured so that a standard template can be selected so as to correspond, or the template may be configured by learning the utterance and dialect from the user's utterance.

  Alternatively, a method may be used in which the content of the cookie or session is converted into a phonetic symbol string using phonemes or phoneme pieces according to the attribute variable and the recognition dictionary information 206 is switched, or a phoneme string or phoneme recognized from speech is used. Arbitrary variable transmission such as a method that uses a script such as AJAX, a feature value extracted from a phonetic symbol string such as a single row, a script, a CGI (Common Gateway Interface) parameter, or a socket communication by a program A phonetic symbol string such as a phoneme string or phoneme string recognized based on a voice feature received by the base station or a phonetic symbol string made up of phonemes or phonemes received by the base station The user's utterances are discriminated by using, and arbitrary processing is performed, or search processing of content information, advertisement information, and regional information by configuring search conditions It may be carried out or.

  And display information or music including terminal picture, character, icon, CG (Computer Graphics) or output sound information such as warning sound or robot that changes in accordance with the recognition process of these phonetic symbol strings Feature extraction from motion control information of devices such as mechanical devices, communication devices, electronic devices, electronic musical instruments, recognition dictionary information 206 for recognizing audio, still images, moving images, etc., or video, audio, images For example, a base station can send information for combining and updating arbitrary information such as processing procedure information such as a program, a script, and a function expression, or perform arbitrary processing in a terminal device. May be.

  In addition, when phonetic symbols such as phonemes and phonemes obtained as recognition results are divided into a plurality of frames to obtain recognition results that are continuous in time series, recognition for a plurality of phonemes and phonemes across multiple frames is performed. The distance between the input speech acquired as a result and phonetic symbols such as phonemes and phonemes is used as a feature value to configure the parameters of the Bayes discriminant function, or to span multiple frames to degenerate in time series By configuring the parameters of the HMM using distance information acquired as a recognition result for a plurality of phonemes and phonemes, or by evaluating an identifier evaluated as the first place by a recognition result in a plurality of frames by DP or the like Dynamic speech recognition may be configured in combination with the technology used for conventional speech recognition.

  More specifically, first, markup language information is acquired in steps (S401 and S501) of acquiring content information, and tag attribute detection means for detecting a tag from the markup language information and detecting a tag attribute from the tag is combined. A step of extracting a phonetic symbol string (S402, S502) for extracting a phonetic symbol string associated with the detected attribute and registering it in the recognition dictionary information 206 as a phonetic symbol string used for recognition (S403, S503) To implement. These steps (S401 to S403, S501 to S503) can be produced by character string evaluation processing and detection processing, but operations performed in a conventional speech recognition system, a search by phoneme recognition, a WEB browser by phoneme sequence recognition, and the Internet environment It is not used for searching or browsing content information.

  Next, the step of waiting for the speaker's voice input (S504) is performed, and the step of extracting the feature amount performed by the calculation unit according to the start of the voice input (S505) is performed, and phoneme recognition and / or phoneme pieces are performed. Based on the phonetic symbol recognition program including recognition, a step (S506) of converting the feature value into the identifier by recognition of the phonetic symbol is performed. It is generally known that this step (S506) uses a distance evaluation function or a statistical test method, uses a learning result using multivariate analysis, or uses an algorithm such as HMM. A phonetic symbol string is formed by time-series continuation based on the recognized phonetic symbols.

  Next, the phonetic symbol string is compared with the recognition dictionary information 206 based on the phonetic symbol string extracted from the attribute attached to the tag of the markup language, and the recognition dictionary information 206 is searched, thereby comparing the phonetic symbol string. A step (S507) of evaluating the degree of coincidence between the phonetic symbol strings is performed, and it is evaluated whether or not it is appropriate as a recognition target. For the comparison to determine whether or not it is a recognition target, an algorithm that can be used for symbol string comparison and evaluation such as DP, HMM, and automaton may be arbitrarily used. Various methods have been invented and devised from the past.

  As a result, on the basis of identification information such as a character string or ID associated with the phonetic symbol string specified from the recognition dictionary information 206, a character string is displayed, an arbitrary process is performed, information is exchanged, etc. The recognition process using the phonetic symbols is realized by generating an event, changing the status, or causing the machine device to perform an arbitrary operation, and executing the arbitrary process (S508). As a result, information processing using speech different from conventional grammar dependency and static registered word dependency can be realized.

  At this time, a step of evaluating a match with the phonetic symbol string based on the type information for distinguishing the input items detected by the tag attribute detecting means and having a plurality of recognition dictionary information 206 by the phonetic symbol string ( A phonetic symbol recognition result obtained from the phonetic symbol string and the speech waveform registered in the recognition dictionary information 206 selected according to the attribute of the input item to be recognized while switching the recognition dictionary information 206 used in S507) The recognition efficiency can be improved by limiting the phonetic symbol strings included in the recognition dictionary information 206 to be recognized when the matching score is evaluated by the symbol string comparison evaluation.

  Then, when switching the recognition dictionary information (206) that evaluates the voice input according to the item to be input by the information processing apparatus, the recognition dictionary information (206) of the word associated with the attribute name or attribute is appropriately selected. Therefore, if the information acquired from the attribute is “book”, the unit “book (s / a / ts / u | v / o / ly / u / m)” is used, and the “ Select the recognition dictionary using the phonetic symbol string associated with “number” as the search target for the recognized phonetic symbol string, or if the information obtained from the attribute is “station name”, the suffix Using phonetic symbol strings associated with "stations (e / k / i | s / u / t / e / i / sh / o / N)" and "nouns used as station names" The dictionary can be selected as a search target for recognized phonetic symbol strings, and the information obtained from the attribute is a zip code or telephone number If there are nouns included in a specific framework, simply select a recognition dictionary using a phonetic symbol string of a number as a search target for the recognized phonetic symbol string. Dictionary information 206 to be searched for recognized phonetic symbol strings by switching the plurality of recognition dictionary information 206 according to the attribute associated with the item to be input to the user by restricting Recognition performance can be improved by classifying 206 based on attributes.

  In addition, by outputting voice of words associated with attribute names and attributes in accordance with the order in which voice input is performed and the selection of items that have not been input, the information processing apparatus can input the items to be input. The recognition performance may be improved on the basis of the attribute to be classified by switching the plurality of recognition dictionary information 206 while prompting the item.

  Then, if there is a continuous voice, the next content or markup language is changed in accordance with the step of repeating the conventional processing (S509), the processing step associated with year recognition (S508), or the status change in the device accompanying other external operations. The step (S510) for evaluating whether or not to acquire is executed, and this process ends depending on the situation.

  Note that the status change in the device changes depending on other programs and processes when functioning as a part of other programs or multithread programs, and is used for multithread programs, event-driven programs, etc. Assuming that, it is easy to understand, and similarly, it is possible to rewrite the status or generate an event for another process or another program in the arbitrary processing of the present invention.

  In addition, a character string displayed by designating a markup language by using the method of the present invention or a character string associated with a displayed image or an image feature or output voice / music, voice / music, etc. Arbitrary information displayed by converting various character strings such as character strings associated with acoustic features into phonetic symbol strings and registering them in the dictionary is detected by searching the phonetic symbol strings based on the user's input voice and character strings. In addition to content-related information associated with the detected information, information related to the location of arbitrary information such as advertisements, videos and links, music and voice, etc., it is possible to provide information by user operation, etc. In addition to voice and text input, you can select a character string from a list such as a menu, or use a character string based on a button label in button operation. It may be carried out in interest.

Also, as in the example below (example)
<img href = ”./ flower_lily1.jpg”
recog_dic_type = ”flower_name” recog_dic_url = ”./ flower.prono”
name = ”lily” prono = ”l / i / l / i / y”>
When reading a phonetic symbol according to an attribute related to the phonetic symbol dictionary included in the tag, the phonetic symbol recognition dictionary or table is read according to information such as URL, URI, IP address, directory path, etc. indicating the location of the phonetic symbol dictionary. Reuse frequency is high by using information based on attributes such as “recog_dic_uri” indicating the position and location of information such as a phonetic symbol dictionary, or using information indicating the type of target recognized in the dictionary such as “recog_dic_type” By distinguishing phonetic symbol dictionaries and the like, dictionary information based on phonetic symbol strings and acoustic characteristic template dictionary information for recognition may be provided in association with attributes acquired from the markup language.

  Also, it is possible to save the dictionary information read in the past to some extent by a method generally called a cache, and save the trouble of increasing the priority of the dictionary and reading it again when the above-mentioned attribute indicates a specific word range. Alternatively, a phonetic symbol dictionary that is read for each page as a separate file, such as a style sheet, and is associated with an ID may be incorporated, or a phonetic symbol dictionary that is associated with an ID described in a header block may be incorporated. It is also possible to incorporate a phonetic symbol dictionary given as an attribute for each tag, or to include a phonetic symbol dictionary in header information when reading via a file or communication line. It can also be used as

  In addition, “acoustic OFDM” that can embed text data in speech may be used to embed phonetic symbol strings in speech waveform information, or speech data may be restored by restoring the embedded phonetic symbol strings and related markup language information. Can be used to search for phonetic symbols and display related information in subtitles, etc., so use phonetic symbol strings demodulated from extremely general audio data such as radio and television for searching You can also

  In addition, a database indexed by a phonetic symbol string searched as a search target using a phonetic symbol string acquired by phonetic symbol recognition is a logical space between phonetic symbol strings based on a plurality of keywords. It is also possible to use a combination based on the above, or a configuration in which logicality can be expressed using a Boolean model, and a combination of these can constitute a query and provide it to a database to obtain a search result.

  In this way, unlike the conventional method of stochastically connecting a word and a voice feature group by HMM or the like, the present invention represents a table by associating a phonetic symbol and a voice feature by a distance based on a probability based on a Bayes discriminant function or the like. By acquiring the phonetic symbol string and directly associating the acquired phonetic symbol string and the word character string via the markup language, it is more efficient by restricting the words to be recognized compared to conventional general recognition. It is possible to dynamically provide dictionary information for enabling real recognition in markup languages, and use phonetic symbol strings without using words directly in queries. A database using HMM or DP matching may be constructed and searched by using phonemes or phoneme symbol strings.

  Further, not only attributes based on phonetic symbols such as utterance phonemes and utterance phoneme changes, but also image identifiers, identifiers after image recognition, image-related feature amounts, motion feature amounts, motion identifiers, and the like may be used as attributes.

  The present invention relates to an information processing apparatus that uses phoneme recognition or phoneme piece recognition in speech recognition.

The information processing apparatus and commonly used voice conventionally technique is known for voice recognition. As a method of performing speech recognition, generally, a phoneme or phoneme segment is obtained by phoneme recognition or phoneme recognition using an acoustic model or a standard parameter or template based on a phoneme or phoneme statistically generated from speech accompanying user utterance. There is known a method of extracting phonemes in a time series and acquiring phoneme strings and phoneme string strings.

Then, using the phoneme sequence and phoneme speech recognition dictionary word is recorded consisting of columns, and recognized phoneme sequence and phoneme sequence, the match between the phoneme sequence and phoneme sequence is registered in the speech recognition dictionary The speech recognition and the processing associated with the recognition are realized by evaluating and acquiring a word associated with a phoneme string or phoneme string string having a high degree of coincidence as a result of the evaluation or executing a device control command.

Here, as the user interface for controlling the device, and words specified by the phoneme recognition dictionary as in Non-Patent Document 1, selected by the phoneme recognition process and a device control method which is the dictionary registered in association with the word As a technique for recognizing phonemes and phoneme pieces, as disclosed in Patent Document 1, it has been used as a known technique for a long time.

  In speech recognition, spoken words in human dialogue often change abbreviations and exclamation words and coined words such as “Oo” and “Uon”. Especially in content information, product names and actor names are registered in the dictionary. There were many difficult proper nouns, and not all words could be registered in the dictionary. Therefore, Patent Document 2, Non-Patent Document 2, Non-Patent Document 3, and the like have proposed techniques for searching for content using phoneme recognition.

  Here, Patent Document 3 proposes that a user can easily perform a voice operation by changing the display expression of a recognizable word in use for speech recognition in HTML, which is one of markup languages. Has been made.

  Patent Document 4 proposes a method for dynamically acquiring recognition dictionary data based on an acoustic model with a minimum vocabulary.

Further, according to Patent Document 5, in the use for speech recognition in HTML, which is one of markup languages, a range can be designated by a specific symbol to identify a recognizable word, and speech recognition can be performed. Is proposed to the user, and the words that are difficult to pronounce are described with a recognizable reading method for convenience.
Japanese Patent Laid-Open No. 62-220998 JP 2005-70312 A JP-A-11-25098 JP 2002-91858 A JP 2005-18241 A “Research and development on life support interface for aging society”, Key Project Research Report by Aomori Prefectural Industrial Research Center Vol.5, Apr.1998-Mar.2001 031 Masayuki Nakazawa, Takashi Endo, Kiyoshi Furukawa, Jun Toyoura, Ryuichi Oka (New Information Processing Development Corporation), "Study of speech summaries and topic summaries using phoneme symbol sequences from speech waveforms," IEICE Tech. 28, pp. 61--68, June 1996. Ryuichi Oka, Hironobu Takahashi, Takuichi Nishimura, Nobuhiro Sekimoto, Hidehide Mori, Masanori Ihara, Hiroaki Yabe, Hiroki Hashiguchi, Hiroshi Matsumura. Pattern Search Algorithm Map-What Supports "CrossMediator" Study Group of Society, volume 1, pages 1-6. Japanese Society for Artificial Intelligence, 2001.

In addition, the use method of the phoneme symbol string in the markup language uses a configuration called Segment and Media Locator as a description in MPEG7 used in a moving picture stream such as MPEG2, and uses <Media Locator> to set Segment or Frame in video content. You can specify it directly, or use <Media Locator> to point to the content, <Media Time> to specify the time position inside the content and combine it with a tag that specifies an appropriate proper noun, When assigning, there is a method of using <Series> description method to attach the same kind of metadata at fixed intervals as low level metadata of Visual and Audio. In this case, it's audio in the MPEG7 audio how to specify the Oh Ri as <Scalable Series>, the word is an automatic speech recognition result (word) Lattice and phoneme (phone) those that describe the lattice <Spoken Content DS> There is.

In addition, VoiceXML, a standardized method for speech recognition, proposes a method for expressing a unified method for describing user interfaces that have been divided among products so far, in order to implement recognition that depends on the grammar according to the context. However, no method has been proposed for dynamically constructing dictionary information by assigning attributes to the target range of an arbitrary tag using phonetic symbol identifiers such as phonemes and phoneme pieces regardless of context and grammar.

  According to conventional applications and documents, many phonemes and syllables are confused, but the phoneme in the present invention is the case where the pronunciation of “Akasana” in Japanese is used as an example, and the syllable is used. If there is a phonetic notation, “a / ka / sa / ta / na” or “a / ka / sa / ta / na” is displayed in a single voice, and “a / k / a / s / a / t / a / n / a '' or `` a / cl / k / a / s / a / cl / t / a / n / a '', or `` a / ak / k / ka / a '' / as / s / sa / a / at / t / ta / a / an / n / na / a '' or `` a / a-cl / cl / cl-k / k / ka / a / as / s / sa An example such as `` / a / a-cl / cl / cl-t / t / ta / a / an / n / na / a '' can be considered as bigram, `` aaa / a-cl-cl / cl-cl- cl / cl-cl-k / cl-kk / kka / aaa / aas / sss / saa /… taa / aan / nnn / naa / aaa Arbitrary parts such as the first half, middle board, and second half / Cl / points to the silent or unvoiced part before sounding, and both these phonemes and phonemes are phonemes that represent any sound by any improvement. You may change into a notation symbol piece, a phonetic symbol piece, and a phonetic symbol piece like a notation symbol, a phonetic symbol, a phonetic symbol, and the phoneme piece which decomposed them time-sequentially.

  In addition, the difference between phonetic symbol recognition using phonemes and phonemes and normal speech recognition is explained. Phone recognition and phoneme recognition do not perform vocabulary recognition that interprets meaning and content unlike general speech recognition. And acoustic models are not dynamically constructed according to changes in language models such as words, grammar, and parts of speech. More specifically, phoneme recognition and phoneme recognition do not use grammar-related language models. Does not capture meaning, or does not convert to symbols that contain meaning such as kanji, or does not distinguish homonyms or homonyms, and differentiates parts of speech such as nouns and verbs according to context In this case, phonemes, phonemes, phonetic symbols, phonetic symbols, phonetic symbols, It is denoted as phonetic symbol recognition together recognition using phonetic symbols based on time series of the divided phonetic symbol) phonetic symbols pieces and their symbol string.

  In this way, recognition by phonemes and phonemes is performed by analyzing a speaker's utterance using a static acoustic model for each phonetic symbol and matching the phonetic symbol string accompanying the utterance with the phonetic symbol string in the recognition dictionary. The recognition process and the structure of the recognition dictionary are simplified due to the feature of evaluating only the phoneme, and the phonetic symbols and phonetic symbols such as phonemes and phonemes are used even for unregistered words and exclamation words to evaluate only the sound match. It is possible to recognize the identifier string.

  At this time, as in the past, a dynamic acoustic model that learns according to the speaker's utterance characteristics and improves performance may be used, but depending on words and grammar as in general speech recognition, There is a feature that the process of dynamically switching the model is not performed in phoneme recognition or phoneme recognition.

  For this reason, comparing phoneme strings and phoneme string sequences with registered dictionary contents makes it possible to easily detect unregistered phoneme strings and phoneme string sequences, resulting in recognition results by phonetic symbol recognition using phonemes and phoneme segments. It is also conceivable to implement efficient speech recognition by limiting speech based on words and implementing speech recognition that takes into account general grammar again.

And even if there is a word that is not registered in the dictionary, the recognition method using phonemes and phoneme fragments converts unregistered words in the sentence to be recognized into hiragana character notation, and changes to the transition state of the converted hiragana character string. In addition, a symbol string converted into a phoneme string or phoneme string string based on the prosody obtained from known information is temporarily registered in the recognition dictionary, and acquired after the user's utterance is recognized as a phoneme string or phoneme string string. By comparing the phoneme sequence or phoneme sequence with the phoneme sequence or phoneme sequence of the recognition dictionary, the degree of coincidence between the symbol sequences is measured, and the recognition result is acquired. In addition, it is possible to perform speech recognition having a dictionary configuration with dynamic phonemes and phonemes having a higher degree of freedom than conventional speech recognition.

  At this time, the user can improve the recognition accuracy with a dynamic acoustic model dictionary that does not depend on grammar or words by re-learning the phoneme or phoneme unit acoustic model according to the user's utterance. Re-learning for recognition may be performed by reusing the acoustic information obtained from the utterances as teacher information.

  In conventional speech recognition technology, abbreviations of spoken words in human dialogue, exclamation words (exclamation words) such as “Oo” and “Uon” and coined words have many differences depending on the times and environments, especially in content information Although dynamic proper nouns that depend on trends such as product names and actor names have been inefficiently registered in the recognition dictionary, they have existed for a long time as a challenge when putting huge and varied speech recognition into practical use. Since it is relatively difficult to repeatedly distribute recognition dictionaries including acoustic models and grammatical models, recognition based on vocabularies not registered in the recognition dictionary is virtually impossible.

  Also, in conventional speech recognition, learning of prosodic models and grammatical models is generally indispensable, and such processing procedures have become issues for recognition of unregistered words based on the aforementioned coined words, buzzwords and proper nouns. There is a problem that it is difficult to learn a grammar model based on the prosody related to such unregistered words and the co-occurrence relationship between words.

  Conventionally, audio information in a markup language cannot be searched or operated except for audio information synchronized with video and audio as content information. In order to provide information including a phoneme symbol string to a user, it is necessary to recognize speech information in advance and store the phoneme and a word ID (word identifier) in association with each other.・ There was a problem that there was no method of providing or operating a phoneme.

  Further, in the technique disclosed in Patent Document 3 described above, a method for recognizing a word that is not registered in the dictionary is not presented. Furthermore, in the technique disclosed in Patent Document 4, speech recognition that does not depend on vocabulary cannot be performed, and it is necessary to take measures such as learning prosodic models for unknown words each time. High voice recognition could not be realized. Furthermore, the technique disclosed in Patent Document 5 has a problem that the speech recognition method is not different from the conventional speech recognition method, and recognition using phonemes or phonemes cannot be performed.

  Based on such problems, the object of the present invention is to use a word model, an acoustic model, a grammar model, and a part-of-speech information in a speech recognition dictionary when speech recognition is performed on a word or character string included in content information. An information processing device that can realize more appropriate speech recognition by using recognition dictionary information based on phonetic symbols using phonetic symbol recognition consisting of phonemes and phoneme pieces even if is not registered The purpose is to provide.

  In order to solve the above problems, an information processing apparatus according to a first aspect of the present invention includes a content information acquisition unit that acquires content information including character information and / or meta information, and content information acquired by the content information acquisition unit. And a recognition phonetic symbol string detecting means for detecting a recognized phonetic symbol string consisting of phonetic symbols, and a recognition dictionary information generating means for generating recognition dictionary information using the recognized phonetic symbol string. Features.

  An information processing apparatus according to a second aspect of the present invention includes a content information acquisition unit that acquires content information including character information and / or meta information, and character information and / or meta information from the content information acquired by the content information acquisition unit. By referring to the expansion target character string detection means for detecting the expansion target character string based on the character string, the phonetic symbol storage means for storing the character string and the phonetic symbol in association with each other, and the phonetic symbol storage means, A phonetic symbol conversion unit that converts the expansion target character string into a recognized phonetic symbol string; and a recognition dictionary information generation unit that generates recognition dictionary information using the recognition phonetic symbol string. .

  In addition, in the information processing apparatus according to the second aspect, in the information processing apparatus according to the second aspect, the content information is stored by adding the phonetic symbols converted by the phonetic symbol conversion means to the content information. The apparatus further comprises means.

  According to a fourth aspect of the present invention, in the information processing apparatus according to any one of the first to third aspects, content information stored by the content information storage unit and a recognition dictionary generated based on the content information It is further characterized by further comprising transmission means for transmitting information to another information processing terminal.

  According to a fifth invention, in the information processing apparatus according to any one of the first to fourth inventions, a voice input means for inputting voice, and a feature amount of the voice input by the voice input means are extracted. A feature amount extracting unit; a feature amount phonetic symbol converting unit that converts the feature amount extracted by the feature amount extracting unit into a phonetic symbol; and a phonetic symbol converted by the feature amount phonetic symbol converting unit. And a process executing means for evaluating a phonetic symbol constituting a recognized phonetic symbol string included in the recognition dictionary information and executing a predetermined process corresponding to the most similar phonetic symbol. Features.

  According to a sixth aspect of the present invention, in the information processing apparatus according to the fifth aspect of the invention, the content information includes phoneme information and / or phoneme piece information, and the processing execution means includes the feature quantity phonetic symbol. The phonetic symbols converted by the conversion means and the phonetic symbols constituting the recognized phonetic symbol string included in the recognition dictionary information are evaluated, and the user is notified of the voice corresponding to the most similar phonetic symbol. It is characterized by presenting information by utterance.

  According to a seventh aspect, in the information processing apparatus according to any one of the first to sixth aspects, the phonetic symbol is a phoneme or a phoneme piece.

  The eighth invention is characterized in that in the information processing apparatus according to any one of the first to sixth inventions, the executed process is an authentication process accompanying phoneme recognition.

  According to a ninth aspect of the invention, there is provided a program for interpreting information described using a markup language in a computer, an attribute acquiring step for acquiring an attribute designated by the interpretation, an attribute acquiring step, A phonetic symbol extraction step for extracting a phonetic symbol sequence and / or a phoneme sequence and / or a phoneme fragment sequence associated with the attribute acquired by the phoneme sequence, and a phoneme sequence dictionary used in the phoneme recognition unit by the phonetic symbol extraction step And a dictionary changing step for changing.

  According to a tenth aspect of the present invention, there is provided a program for interpreting information described using a markup language in a computer, an attribute acquiring step for acquiring an attribute designated by the interpretation, an attribute acquiring step, A phonetic symbol extraction step of extracting a phonetic symbol string and / or a phoneme sequence and / or a phoneme fragment sequence associated with the attribute acquired by the user, and a user inputs based on the attribute acquired by the attribute acquisition step An information type evaluation step for evaluating the type of information and a dictionary changing step for changing a phoneme string dictionary used in the phoneme recognition unit are realized by the information evaluation step.

  According to the present invention, in order to use an information processing device using phoneme recognition, a phoneme dictionary necessary for recognition of provided content information is acquired from a markup language associated with the content information or included in the content information. By doing so, it is possible to deal with unspecified words related to display contents. Therefore, in order to perform processing with a high possibility of unspecified words such as product sales in a single device or server / client environment, the name of a phoneme string, phoneme string string, or various identifiers is a tag attribute in the markup language. It is a problem to be able to specify the utterance phoneme dictionary in scene units that span multiple frames of moving images or frames as frames of moving images or frames in content images, sentences in page units or sentence structures It is intended to solve this problem.

  Also, by expanding the phonemes used for these operations, as variables, attributes, and specific tags depending on the file format for distribution and markup language such as HTML, XML, RSS, EPG, BML, MPEG7, CSV sent to the user Indexing using voices easily by incorporating identifiers using phoneme strings and phoneme strings related to voice operations in association with arbitrary markup languages and scripts, and users can acquire, view, and manipulate information using voice Therefore, it is possible to distribute and share voice control information for the purpose of acquisition and to acquire voice control information on the terminal side, and to solve the problem.

  And, when the present invention implements the conventional technique of recognizing unspecified words using phonemes or phonemes, it restricts the words appearing in one scene of content information to various contents changing in the Internet environment. By providing a dynamic dictionary construction method using phoneme sequences and phoneme segment sequences, it is possible to realize speech recognition processing for unspecified words without using prosodic models or grammatical models. We are trying to realize improvements.

  In the case of MPEG7, a scene name, an actor name, and a cast name in a tag representing a scene of video information are specified by attributes, variables, and range specification by a tag using a phoneme symbol string or a phoneme symbol string. Using the information described in the markup language other than the recognition location, and performing a search with any actor name or cast name using the phoneme search technology, any phoneme string corresponding to the scene can be obtained from the markup language information. A device capable of performing instructions and searching can be realized, and the problem can be solved.

  In HTML, a variable or attribute including a phoneme symbol string is provided in the target link or CGI notation, or a range surrounded by a specific tag is converted into a phoneme string and embedded as a tag variable or attribute. , Provide variables and attributes for each table element of the table tag surrounding the product you want to select and give each element tag a name as a variable or attribute with a phoneme string symbol, or give a phoneme string as a variable or attribute for a form tag or input tag, Based on a given phoneme string, the problem can be solved by a method of transmitting information or transitioning to the next page.

  Also, a phoneme string or phoneme string string may be distributed by RSS, ID is associated with a keyword using a tag, and a CSV file is provided as recognition dictionary information that associates the ID with the phoneme string / phoneme string string. You can use a method that identifies the keywords that are to be recognized in, an image recognition dictionary such as a face or fingerprint, a recognition dictionary with proper nouns using phonetic or phonetic symbol strings, and for each speaker Individual authentication using secret words may be performed by associating with an acoustic model based on phonemes or phoneme pieces.

  In this way, the contents of the recognition dictionary based on phonemes and phonemes are acquired from the outside as markup language attributes, arbitrary tags, and dictionary files, thereby enabling operation of the information processing apparatus and solving problems. Can do.

  That is, since words that are not clearly included in the content information are not included in the recognition dictionary unless specified, the probability of misrecognition is reduced, voice operation is performed, and device control by phoneme strings or phoneme string strings The existing markup language can be used for personal authentication with high versatility, and for information exchange between information processing devices. A highly convenient user interface can be realized by adding notations with phonemes and phonemes, and using dictionary information with phonemes and phonemes associated with or associated with markup languages and contents.

  In the present invention, a markup language notation used for content information is changed and saved, or saved and used, or the changed information is used as it is, and a change has been added. It is possible to configure an information processing apparatus that is used for a distribution apparatus that distributes information and a reception terminal that is received and used for recognition and operations and responses associated with recognition. More specifically, as shown in the example of XML or HTML, a method of saving, changing, and distributing information by changing information written in an existing markup language and adding a tag or adding a variable or attribute And receiving the information and operating the information processing apparatus.

<Example of content information>
First, the content and content information to be searched and indexed using the present invention will be described. The content is exclusively a movie, a drama, a photo, a news report, an animation, an illustration, a painting, music, a promotional video, It is generally well known that novels, magazines, games, papers, textbooks, dictionaries, books, comics, catalogs, posters, broadcast program information, etc., but in the present invention, public information, map information, products Information, sales information, advertising information, reservation status, viewing status, viewing status, road status information and questionnaires, surveillance camera images, satellite photos, blogs, models, dolls, robots, and cameras equipped in these devices・ Information obtained by microphone / sensor input, etc., names of such information, status and situation, abstract concepts, superordinate concepts and subordinate concepts It may include information that expand the designation about the symbol string by phonemes or phoneme.

  In addition, there are video time-series changes, audio time-series changes, texts that expect readers' reading positions in time-series, electronic information in markup language notation in HTML, search index information generated by them, and the like. Alternatively, the reading position may be interpreted as a time axis, and a punctuation mark, sentence, chapter, or sentence may be captured as a frame.

  Also, meta information attached to content, EPG and BML as documents and text information by character information, musical scale as musical score information, general still images and moving images, polygon data, vector data and texture data as three-dimensional information And motion data (motion data), still images and moving images based on visualization numerical data, content information for the purpose of advertisement and advertisement, etc., and natural information including visual information, auditory information, text information, and sensor information It is comprised by.

  A method of recognizing the content of a content using a <Spoken Content DS> tag describing a phoneme (phone) lattice such as MPEG7 used conventionally and adding a phoneme sequence has been proposed. Is indexed by a symbol string based on the recognition of audio information generated in the content, so that the user can search for the title and performer of the content by voice operation, phoneme notation and phoneme notation Information is not provided by notation using phonetic symbols and phonetic symbols.

  For this reason, unspecified words and proper nouns related to the names and expressions of content titles and performers cannot always be used for speech recognition, and therefore are related to content information such as scene descriptions, titles, and performer names as in the present invention. Character information is written together by expanding the phoneme, and as a variable or attribute of the tag, an identifier based on a phonetic symbol string, phoneme symbol string, syllable symbol string, or any phonetic symbol or phonetic symbol is embedded in the MPEG information. Use of phoneme recognition technology for speech recognition can be achieved.

  In other words, if the part enclosed by the tag that is the target of speech processing in markup language is an arbitrary character string, the character string is expanded into a phoneme symbol string or phoneme symbol string, and a phoneme symbol or phoneme symbol is used. It can be used for recognition, and it is evaluated by matching the phoneme symbol string and phoneme symbol string recognized from the user's utterance, or by converting the uttered phoneme into arbitrary phonetic characters. The phonetic characters may be matched, or the phoneme symbol string based on the user's utterance recognition result may be evaluated to be the user's operation target or search target. In addition, any character or symbol such as an at sign or a key bracket described in ideograms may be converted to a phoneme symbol or phoneme symbol by an appropriate phonetic symbol string, or a character string from which multiple utterances can be estimated If so, a plurality of phoneme strings, phoneme string strings, and syllable symbol strings may be given as in conventional speech recognition.

  Then, the recognized phoneme sequence or phoneme segment sequence is given to the database as a query and searched by a symbol sequence matching method such as DP or HMM, and the phoneme sequence or phoneme segment sequence is added to the search result so that the user can view it. The search result is presented, the product is selected based on the phoneme sequence included in the search result, and the phoneme sequence and phoneme sequence for performing the charge and purchase procedure are detected from the recognition dictionary with the recognition. By combining a password with a phoneme recognition dictionary composed of user uttered voice features and a recognition dictionary such as fingerprints, irises, faces and palmprints composed of image features By performing the authentication and charging with, it is possible to realize a search / browse / sales / authentication / billing procedure for goods and content information such as goods and rights.

  In this way, an identifier or identifier string recognition dictionary based on phonetic symbols necessary for evaluating an arbitrary word to be acquired as a recognition result in accordance with a position in information such as a content reproduction location, page, or display location. Can be used to create a highly versatile dictionary structure for unspecified words in a variety of usage environments. Words recognized based on a phonetic symbol recognition dictionary using dynamically constructed phonemes and phonemes By presenting, performing arbitrary processing, acquiring the URL of the advertisement, presenting the advertisement, and operating the device, information that is highly convenient for content and advertisement distribution to the user Specifying search conditions and sending them by using phoneme strings and phoneme string sequences as variables used for CGI processing posts and get in the Internet environment such as presentation and Web Or, it is possible to and go the Web page switching and operation.

  The procedure for expanding Japanese into phonemes is well known, and after converting it to “kana notation” using a “split writing” program that converts kanji-kana mixed notation obtained from ideograms into phonograms, There is a method of constructing a symbol string to be used for recognition by using phonetic symbol conversion and phoneme symbol conversion using phonetic symbols such as `` Roman characters '' accompanying `` notation '', and there is also a method of describing with syllable symbols in the same procedure .

  If it is English, it can be converted into a phoneme symbol string using an English phoneme symbol or a phonetic symbol, or converted into a phoneme symbol string using an international phoneme symbol. Appropriate phonetic symbols and phoneme symbols may be used, and there are pronunciation dictionaries in various languages, so phonemes and phonetic symbols as phonogram identifiers are time-sequentially based on phonetic symbols according to the language. Distributing information using markup languages based on arbitrary phonetic symbols by dissociating phonemes as disassembled identifiers and expressing them as appropriate character codes in association with numbers It becomes possible.

  At this time, if necessary, the phoneme symbol string may be converted into a phoneme symbol string to improve the convenience of the search. The environmental sound identifier, the scale identifier, the image identifier, and the motion identifier are assigned to each environment. It may be a sound lattice or a scale lattice, a section of image identifiers or motion identifiers may be provided in the MPEG stream, and a phoneme sequence or a phoneme segment sequence may be given based on pronunciation related to the names of these identifiers.

Next, a more specific procedure will be described with reference to the drawings.
〔Device configuration〕
First, the apparatus configuration of the information processing apparatus 1 when the present invention is applied will be described with reference to FIG. Here, the information processing apparatus 1 is an apparatus that is realized by each information processing device such as a general-purpose computer, a dedicated terminal, and a portable mobile terminal.

  As illustrated in FIG. 1, the information processing apparatus 1 includes a control unit 10, a storage unit 20, a communication unit 30, an input / output unit 40, an operation unit 50, and a display unit 60. . Here, each functional unit is connected to the control unit 10 via a bus. Note that the operation unit 50 and the display unit 60 may be arbitrarily removable devices.

  First, the communication unit 30 is a functional unit for exchanging information with other devices via a LAN (Local Area Network) or a communication network such as the Internet. Here, the communication unit 30 is generally configured by a device capable of transmitting and / or receiving content information such as Ethernet (registered trademark), a modem, a wireless LAN, and a cable television device.

  Next, the input / output unit 40 is a functional unit for inputting / outputting information from / to another device or from the outside, such as an input device such as a microphone, a scanner, a capture board, a camera, or sensors, a speaker, a printer, It consists of output devices such as devices and display devices.

  The storage unit 20 is a functional unit that acquires and stores information in the information processing apparatus 1 and stores a program executed by the control unit 10. The storage unit 20 includes a ROM or RAM as a semiconductor storage element, a hard disk or magnetic tape as a magnetic storage medium, a CD (Compact Disk) or a DVD (Digital Versatile Disk) as an optical storage medium, or the like.

  Specifically, the storage unit 20 stores content information 202, a phonetic symbol conversion table 204, and recognition dictionary information 206, a phonetic symbol addition program 208, a recognition dictionary information update program 210, and the like. The voice operation program 212 is stored.

  The content information 202 stores content acquired from the outside via the communication unit 30 and content input via the input / output unit 40. The phonetic symbol conversion table 204 is a table that is referred to when the content information is converted into phonetic symbols. For example, a character string and a phonetic symbol such as a phoneme are associated with each other and stored. It is a table.

The recognition dictionary information 206 is information that stores a relationship between a word and a phoneme sequence, a phoneme segment sequence, and the like (hereinafter, these phoneme sequences are indicated as phonetic symbols). For example, as shown in FIG. 2 , the item “Title”, “profit campaign” as the target word, and “o / t / o / k / u /” expanded from the target word as a phoneme string (phonetic symbol). “ky / a…” is stored in association with each other. The recognition dictionary information 206 may register proper nouns such as “product name”, “product nickname”, and “phoneme string based on product nickname” in addition to items, etc., and may be registered in a general language dictionary. A recognition dictionary that realizes various recognitions is constructed by dynamically exchanging words including exclamation words and abuse words that cannot be used by phoneme strings or phoneme string strings.

  The operation unit 50 is a functional unit that receives an operation input from a user, and includes an input device that inputs information associated with an operation such as a keyboard, a mouse, a camera, and a remote controller (including wireless). The display unit 60 is a functional unit that outputs information output by the information processing apparatus 1 so that the user can visually recognize the information. The display unit 60 is configured using a display device that performs display related to operations including a display and a projector. ing.

  The control unit 10 calls various programs stored in the storage unit 20 to execute a process for realizing a function corresponding to the program or to control each functional unit of the information processing apparatus 1. It is carried out.

  The control unit 10 reads out the phonetic symbol addition program 208 from the storage unit 20 and executes it, thereby realizing a phonetic symbol addition process to be described later. Moreover, the recognition dictionary information update program 210 mentioned later is implement | achieved by reading and executing the recognition dictionary information update program 210 from the memory | storage part 20. FIG. Further, the voice operation processing is realized by reading and executing the voice operation program 212.

  In addition, the control unit 10 executes a program to acquire phoneme / phoneme recognition processing, tag information acquisition, tag identifier acquisition, phoneme string / phoneme string array, and phoneme / phoneme recognition of user utterance speech. Can select words based on similarity evaluation between phoneme sequence / phoneme segment sequence and phoneme sequence / phoneme segment sequence associated with dictionary registration information, and acquire speech waveform from input / output unit using microphone It may be used for speech recognition, or information may be provided to the user by speech synthesis using a phoneme sequence or phoneme segment sequence obtained by the present invention using a speaker.

  Note that the control unit 10 is usually configured using a CPU (Central Processor Unit), DSP, ASIC, or the like, and can be realized by arbitrarily combining them.

<Operation>
Next, each operation process executed by the information processing apparatus 1 will be described.

<Phonetic symbol addition processing>
First, the phonetic symbol addition processing will be described with reference to FIG. FIG. 3 is an operation flow for explaining the phonetic symbol addition processing, which is realized by the control unit 10 reading and executing the phonetic symbol addition program 208 in the storage unit 20.

  First, the control unit 10 acquires content information 202 stored by being received by the communication unit 30 or input by the input / output unit 40 (step S301).

  Next, a development target character string is detected from the read content information 202 (step S302). Here, the expansion target character string is a character string (information) for identifying a change in display control method. For example, in the case of a markup language, a tag <A> indicating a link or a title It is a tag <TITLE> indicating A character string to be expanded that is expanded into phonetic symbols such as phonemes and phoneme pieces is detected for the range between the tags.

  Next, the expansion target character string is expanded into a phoneme string or a phoneme string string (phonetic symbol) composed of phonetic symbols accompanying the utterance (step S303). Thereby, for example, the title and the name of the link destination are converted into phonetic symbols. When this expanded character string is converted into phonetic symbols, the character string is acquired by referring to other attributes such as the ALT attribute and ID attribute that are information contained in the tag, and converted into a phoneme string or phoneme string string. To compose recognition dictionary information using phonetic symbol strings for dictionary registration, compose phoneme and phoneme string sequences from image file names, music file names, video file names and document file names, A phoneme or phoneme string sequence is constructed using the tag attributes and the text string sandwiched between tags, which are described in files, music files, video files, and document files, and phoneme strings are generated from the text strings sandwiched between tags. Or a phoneme segment string, or by using link information associated with the tag as an attribute, the name of the file at the link destination or the character information contained in the file is sandwiched between the tag or tag attribute or tag String Or configure the phoneme sequence and phoneme sequence by use, how to configure the phonetic symbol string registered in the dictionary by using an arbitrary method such as is conceivable.

  Specifically, the phonetic symbols are converted into phonetic symbols using the phonetic symbol conversion table 204. For example, a character string “main” surrounded by title tags is converted to “m / e / i / n /” as a phonetic symbol by referring to the phonetic symbol conversion table 204.

  In addition, there is a case where the attribute of the phonetic symbol expansion is already given as a tag of the content information itself without performing such development to the phonetic symbol, and the recognition table by the phonetic symbol string to be used for recognition is used. The phonetic symbol string may be configured. For example, after performing step S401 for acquiring the content information, the “pronounce attribute” is detected from the accompanying meta information from the markup language information as shown in FIGS. Step S402 is performed, and a phonetic symbol string composed of phonemes and phoneme pieces described as a variable of the detected “pronounce attribute” is extracted, and the extracted phonetic symbol string and “pronounce attribute” are detected. By executing step S403 of associating with meta information and registering it as dictionary information, a table that uses a voice operation program or expresses an utterance sound that can be recognized by the information processing apparatus A dynamic phonetic symbol string that specifies any processing, procedure, or operation by specifying a processing content or transition destination page using a tag or CGI as meta information associated with the phonetic symbol string, Realize recognition using.

  As a result, the phonetic symbol storage process is executed, and the recognized phonetic symbol string used for phonetic symbol recognition is stored (step S304). The phonetic symbol storage processing is processing for storing the phonetic symbols used for recognition converted in step S303. For example, the phonetic symbols that are already recorded as attributes in the respective tags are extracted, A process (step S304a) of adding a phonetic symbol (phoneme string or phoneme fragment string) as a new attribute by expanding from the sandwiched character string, and a tag or attribute indicating that each tag is a speech recognition target Processing to add to content information (step S304b), processing to configure and update recognition dictionary information 206 by separating proper nouns to be recognized, converting them into phonetic symbols, and forming recognition phonetic symbol strings ( Step S304c) is executed. Thus, processing for clarifying the content information and the recognized phonetic symbol string as a phonetic symbol string including a phoneme string or a phoneme string string of a word to be used for recognition is performed.

  Then, the control unit 10 updates and saves the changed content information 202, and updates and saves recognition dictionary information for phonetic symbol recognition using phonemes and phonemes composed of associated recognition phonetic symbol strings. (Step S305). This makes it possible to use the content information changed to the recognition of the user's utterance and the distribution via the communication unit.

  Although the above-described processing has been described as being executed by the information processing device 1, the processing load of information accompanying conversion to a phoneme string on the receiving side is executed by the distribution device (server) side that distributes content information. You may make it reduce. When the distribution apparatus executes, the distribution apparatus distributes content information accompanied by voice control information in response to a call of content information from a user. Therefore, the information processing device 1 (terminal device) can acquire phoneme information classified according to the page and frame of the content by the information processing device, and can use voice of arbitrary words with less restrictions.

  Here, an operation example when the phonetic symbol addition processing is executed will be described with reference to the drawings. First, FIG. 4 is a diagram showing the state of the content information 202 acquired by the information processing apparatus 1. By executing step S301, the content information 202 is acquired from the communication unit 30 or the input / output unit 40 and stored in the storage unit 20.

  And the information relevant to the tag used as the evaluation object by the phonetic symbol (phoneme sequence / phoneme segment sequence) is detected (step S302). Note that the information in FIG. 4 is an example of content information using an RSS item section when the extraction process is executed in step S302. FIG. 5 shows an example in which a target character string is extracted from the item section and converted. Or it is described as FIG.

  When a tag that is the target of the expansion target character string is detected from the tags included in the acquired content information, the character string in the range specified by the tag is detected. For example, in FIG. 4, a “profit campaign” sandwiched between tags “<title>” and “</ title>” meaning a title is detected as a character string to be expanded into a phonetic symbol string. At this time, unnecessary parenthesis symbols may be deleted, and an arbitrary title character string designated by the distribution side can be acquired by extracting the character string.

  Then, the acquired character string is confirmed and converted into a phonetic symbol string according to the pronunciation of the character string using the phonetic symbol conversion table 204. Then, as shown in FIG. 5, for example, a new pronouncation attribute is added as an attribute or variable to the tag described in the original content information 202 to add a phonetic symbol string (step S304a), and FIG. As shown in the figure, processing for newly setting <pronounce> to </ pronounce> tags, a word and a command recognized as a phonetic symbol are associated and stored as recognition dictionary information 206, and the recognition dictionary information 206 is stored as content information 202. A process of describing and associating a URL with a <META> tag or the like as the acquisition destination of the recognition dictionary information 206 (step S304c) or the like is executed, and phonogram symbol string information used for phonogram recognition is added to or associated with content information It becomes possible.

  Then, the operation based on phonetic symbols (phonemes, phonemes, phonetic symbols, phonetic symbols) by distributing the content information 202 with the above-mentioned changes directly to other terminals or using it in the device. Will be able to.

  In addition, for example, in MPEG7, as shown in FIG. 7, a “<Pronounse DS>” tag is added to describe a phoneme string of content type, or “Rainsound” is generated as an environmental sound behind. The phonetic symbol string “pronounce =” t / o / m / u ”” is added as an attribute to the notation about the performer that can be placed in the phrase tag. In addition, HTML presents an example of a configuration associated with a button or link, and a range between arbitrary tags is searched as a keyword, which is useful for browsing and searching for content, and pronunciation for operation is a phoneme symbol. It may be provided as a sequence, or a phonetic symbol sequence such as acquired phonemes or phonemes may be used in a word dictionary of speech phonemes or speech phonemes for speech synthesis speech in the information processing apparatus 1.

  Thus, according to the phonetic symbol addition processing, a table for incorporation into the phonetic symbol dictionary used for phonetic symbol recognition by adding a phonetic symbol for performing voice operation based on the acquired content information. Content information including phonetic symbol string information can be configured.

<Recognition dictionary information update process>
Next, a recognition dictionary information update process in the case where a phonetic symbol has already been added to the content information 202 will be described with reference to FIG. FIG. 8 is a diagram illustrating an operation flow related to the recognition dictionary information update process, which is a process realized when the control unit 110 executes the recognition dictionary information update program 210 of the storage unit 120.

  First, the control unit 10 acquires content information 202 (step S401). Next, the control unit 10 extracts a phonetic symbol string from the read content information 202 (step S402). In the present embodiment, by extracting a tag (a portion sandwiched between “<” and “>”) included in the content information 202, a tag including a phonetic symbol string is specified and extracted. Become.

  For example, the control unit 10 extracts the “pronounce attribute” of the title tag “<TITLE>” to extract the phoneme symbol string “o / t / o / k / u... To do. Then, the extracted phoneme string is stored as a page title and registered in the recognition dictionary information 206 (step S403).

  In addition, by switching these dictionaries according to the display contents that change according to the page switching, it is possible to avoid erroneous recognition due to words that are not in the display contents, thereby improving the speech recognition rate and improving operability. Then, a necessary phoneme string may be acquired from a URL or the like of dictionary information associated with content information using an arbitrary tag or character string, and dictionary information for associating with a recognized word or control method may be updated.

  In addition, when the phoneme sequence or phoneme sequence is not described in the content information to be distributed or the related phoneme dictionary is not associated, the phoneme or phoneme sequence is performed according to the procedure for embedding the phoneme sequence / phoneme sequence. May be constructed from the content to construct dictionary information, and the constructed dictionary information may be reused if it can be used by detecting whether or not the same word is used. good.

  If the control command does not change the phoneme symbol string when the control recognition dictionary is configured, the control command using the dictionary in which the ID related to the control command, the command word, and the phoneme sequence are associated as shown in FIG. After the distribution of the content information in which the ID for specifying the word is described as the command discrimination ID and the recording to the storage medium, the phoneme is used in the information associated with the content information received from the communication unit or the content information acquired from the storage medium. The command word is specified from the command discrimination ID described in the location where the information or phoneme information is described, and the phoneme string or phoneme string is converted by performing the conversion function from the specified command word to the phoneme or phoneme. Sounds at the time of transmission that tend to be redundant by configuring and using for recognition, or by using a hash value based on a phoneme sequence or phoneme segment sequence associated with the control command as an instruction discrimination ID To shorten the string representation may be working to improve the communication efficiency.

  In addition, regarding the content information 202 acquired via the storage medium or the communication means and stored in the storage unit, when the conversion to the phonetic symbol or addition is not performed, the content is interpreted by the above-described method and the information processing is performed. Conversion to a phonetic symbol by an identifier sequence corresponding to the device 1, and if the content of the content information 202 has already been described, converted or updated, the content of the content information 202 is converted or updated. You don't have to.

  In addition, these conversions are information obtained from an external storage medium by a single device, whether it is converted and distributed on the server side depending on the situation of the content information distributor or user, even if it is properly converted by the client May be converted so that it can be used by the device itself, or may be converted using relay means such as a gateway or a router.

<Voice operation processing>
Next, the voice operation process will be described with reference to FIG. First, the control unit 10 acquires content information acquired in the communication unit 30 or the input / output unit 40 and content information 202 stored in the storage unit 20 (step S501).

  Next, phonetic symbols composed of phonemes and phoneme pieces are extracted from the acquired content information (step S502). Then, the recognition dictionary information 206 is updated and registered based on the extracted phonetic symbols (step S503).

  Next, it waits until there is a voice input based on the utterance from the user from the input / output unit 40 (step S504; No). Here, when a voice input is made by the user (step S504; Yes), the control unit 10 extracts a feature amount of the input voice of the user (step S505). Then, phonetic symbols such as phonemes and phoneme segments are recognized from the extracted feature quantities and converted into phonetic symbols (step S506).

  Then, a coincidence evaluation is performed to determine how much the phonetic symbols converted in step S506 match the phonetic symbols previously registered in the recognition dictionary (step S507). In this coincidence evaluation, the degree of coincidence with a standard model, standard parameter, or standard template of sound or speech stored in the storage unit of the present apparatus is evaluated by an evaluation function, and a phonetic symbol as an evaluation result is specified. Then, the phonetic symbol string is specified by obtaining a plurality of phonetic symbols specified based on the coincidence evaluation in time series. Then, the phonetic symbol string having the highest similarity to the identified phonetic symbol string is set as the phonetic symbol recognition result, and device operation and search processing are executed in accordance with the information associated with the recognition result (step S508). ).

  Here, the process associated with the recognition result is, for example, generation of a character string with proper nouns realized by recognition of a phonetic symbol string using the present invention, execution of a search for each operation command, information or product, and phonetic For example, information is presented to a series of users identified in accordance with the recognition of the symbol string, and an operation is instructed by the user. Specifically, web browser page switching, TV and video operations, robots, navigation devices, computers, audiovisual equipment, cooking appliances, washing machines, air conditioners, responses by voice, text, images, and video, search conditions Is a series of processes and operations such as storing, changing, registering and deleting information presented by the information processing apparatus, specifying and browsing advertisements and program contents associated with recognition results, and personal authentication based on keywords and utterance characteristics. In addition, by using an image recognition dictionary such as face or fingerprint, a recognition dictionary with proper nouns using phonetic symbol strings based on phonemes or phonemes, and an acoustic model based on phonemes or phonemes for each speaker, Authentication may be performed, and charging and service selection can be performed along with the authentication.

  Specifically, by recognizing or recognizing a recognizable word by uttering a word with a phoneme sequence or phoneme sequence registered in the recognition dictionary for speech synthesis to answer a user's question. Performing any operation according to the result, presenting the recognized character string or word string according to the recognition result, or performing an advertisement associated with the phoneme string or phoneme string string, It becomes possible by combining with voice recognition technology.

  Then, it is determined whether or not the next voice input is performed (step S509). If the voice is input again (step S509; Yes), the process returns to step S504 as a process for waiting for the voice to be input. If no voice is input (step S509; No), it is determined whether or not to acquire the next content information (step S510). Here, when acquiring the next content information (step S510; Yes), in order to acquire a new content, a process is repeatedly performed from step S501. If new content information is not acquired (step S510; No), a series of processes such as ending the process and waiting for the user's utterance are performed.

  That is, the apparatus using the present invention marks a place where a user can perform a voice operation by using an identifier by a phonetic symbol such as a phoneme and a phoneme from the information in the acquired markup language and a feature amount for specifying the identifier. Acquired from up-language information, and if necessary, it can be used for personal authentication by acquiring and combining images and actions such as fingerprints, facial expressions, palm prints, etc. It can also be used for operation.

  A selection process conventionally performed by mouse operation is performed based on identifiers and feature values obtained by user's utterances and inputs, and focus is given to any row or column, link or operation button of the table tag, or the cursor is moved. Overlapping, issuing events associated with these operations from the operating system to the browser, controlling other devices using communication means such as infrared, LAN and telephone lines, and agents according to recognized words By changing the robot-corresponding operation, a series of processes associated with recognition of phonetic symbol strings such as phonemes and phoneme pieces can be performed.

  The content information acquired in step S501 detects the “pronounce” attribute information in the tag (step S502) and registers it in the recognition dictionary information 206 (step S503). At this time, by registering what kind of tag is associated with the phonetic symbol string for recognition, each tag in the browser is processed by combining the display position and display item of the screen configuration information with the preceding and following tags. Space for latitude / longitude by specifying the display position at the time of mapping, specifying the scene position by associating it with the tag indicating the scene, title and time-series position in the content information in MPEG7, etc. The physical position can be specified by associating it with the position information, the place name, the area information, or the store information.

  Next, an example of the voice operation process will be described with reference to FIGS. When the user pronounces “first line, drafter” i / ch / i / gy / o / u / m / using the table tag's “pronounce” attribute in the top row in table tag column selection A match between the phoneme sequence such as e and k / i / a / n / sh / a and the user's utterance phoneme is determined from the recognition dictionary. As a result, the “drafter” column in the phoneme column of the utterance is selected, and the “first row” in the tag for designating the row is selected, so that the “first drafter” is selected (step) S506).

  If a phoneme string provided as an attribute for each HTML transmission button shown in FIG. 11 is detected, transmission is performed according to the designation of the form tag, or the phoneme string of the user utterance is recognized and “ts / u If the degree of coincidence with “/ g / i / e” is high, the web browsing process can be performed by moving to the link destination. Then, when moving between pages, an agent such as an agent or a robot that responds to the user such as providing a question such as “Do you want to move?” To the user by voice, character string, image, or video (step S506). Dialogue processing may be performed, image recognition dictionaries such as faces and fingerprints, recognition dictionaries with proper nouns using phonetic symbols and phonemes and acoustic models based on phonemes and phonemes for each speaker It is also possible to perform personal authentication using secret words by associating.

  Next, when the browser image viewed from the user is presented and explained, when FIG. 11 is displayed with an HTML browser, it becomes as shown in FIG. 12. Here, when the utterance “Ichiyoume” is spoken, the attribute “i / ch” is displayed. The focus B300 is set in the first line as shown in FIG. 13 in accordance with the phoneme string of the “pronounced attribute” / i / g / y / o / u / m / e ”, and the phoneme string of the“ pronounced ”attribute is set in accordance with the pronunciation“ shosai ” Is selected as shown in FIG. 14 for the button B302 in which “sh / o / u / s / a / i” is written, a click process is performed and a form is transmitted (step S506).

  At this time, if a lot of detailed buttons are displayed, it will not be understood which button, so it will be announced from the contents of the announcement “How many lines are there from the device side” or “How many draft numbers?” Interactive processing by voice or display may be performed by asking questions so that target phoneme sequences or phoneme segment sequences that can be easily analogized can be obtained. It may be provided in XML.

  A browser that has received such an event performs processing set in advance according to those events. For example, in the case of an <a> tag in HTML, a specified link destination is accessed and an arbitrary web page, image, or video is displayed. , Music and product information, "<INPUT TYPE =" button ">", "<INPUT TYPE =" submit ">", "<INPUT TYPE =" image ">", "<BUTTON TYPE =" ~ If it is an operation input tag such as “>”, the HTML process is shifted to a state where the corresponding button or image is pressed, and if it is a “<FRAME>” tag, a frame is selected according to the name of the frame, or “ If it is a <SELECT> tag, the focus shifts to the select tag with phonetic symbols and phonetic symbol strings generated by the user, configures selection candidates from the option tags, selects any item, If it ’s a “<HR>” or “<A NAME="”>” tag, Scroll to the target line with the related tag using the phonetic symbol string associated with the variable or attribute of the tag as a variable or attribute, or if the tag is a <TITLE> tag, It is possible to execute a process of expanding the phonetic symbol string and storing it in the bookmark in association with its own URL (step S506).

  Of course, recognition by these phonetic symbols may be performed in cooperation with a script or the like, or "<EMBED SRC =" ">", "<OBJECT>", "<APPLET CODE =" " Add arbitrary extension functions with tags such as ``> '', give them as variables and attributes to those programs, and identifiers by phonemes, phonemes and phonetic symbols that are instructions and phonetic symbols for operating them externally It may be used for columns and feature quantities, for information used in conjunction with scripts, or for script control conditions.

  For example, in the case of RSS or MPEG7 using XML or RDF, in order to select an item section as shown in FIG. 4, addition of variables and attributes as shown in FIG. 5 or addition of tags as shown in FIG. You can use a method of adding changes, adding a “pronounce” element to the element type based on the “Dublin Core” of RDF, and naming the scene names, actors, and cast names with phonemes and phonemes. "type" and "img-position" elements are added to describe the display position and feature amount of the image, "motion" elements are added to describe the operation in the screen, and "env-sound" elements are added Then, environmental sound identifiers and feature quantities may be described.

  Further, FIG. 11 shows a syllable notation tag for phonetic pronunciation that is a phonetic character and a tag for phoneme, but these tags may be phoneme segments, image identifiers by input video, etc. For example, when a user's voice or facial expression is recognized as being angry, or when a specific identifier is detected from within an image, processing such as presentation of a script or content, or movement to a link destination is performed. May be.

  These tags, attributes, and elements are generally evaluated by matching character strings in a device that interprets them, and functions, processes, programs, etc. that perform processing recorded in the information processing device according to those character strings. Providing services with information based on their tags and attributes.

  For phoneme and phoneme recognition functions, a phoneme sequence or phoneme sequence is provided and registered in the recognition target dictionary, or if it is another identifier or feature, the evaluation coefficient as a detection result is changed. To output instruction information to peripheral devices, to prepare allophone synonyms with a dictionary structure, and to register a phonetic symbol string to register the allophone synonyms in the dictionary. A column may be described so that it can be discriminated by a boundary symbol, or a process for correcting a result obtained by recognition may be performed.

  Also, simply specify a character string to be displayed by placing a character string to be displayed between pronunciation tags, and use a phoneme or phoneme using a phonetic symbol for pronunciation of a character string in another language such as a kanji mixed sentence, an English sentence, or a Chinese sentence. It can be converted into a symbol string and used for recognition, command control, detection, and search. Not only the phonetic symbols are written in alphabets, they are also replaced with numerical values such as ASCII codes and evidic codes. It may be described in the language.

  Also, it can be used as a tool for creating and producing movies and programs by CG etc. by controlling video and dialogue related to features and identifiers associated with words in this way, screen features and display objects. It is also possible to recognize the utterance status while browsing the content, use the content evaluation based on the vote by the voice operation of the user and the number of browsing, etc. You may evaluate a movie and a program based on the correlation with an identifier.

<Server / client model>
In the above-described mechanism, a search procedure using a markup language may be implemented by a server / client model. Specifically, FIG. 15 shows a state transition in processing in the server / client model.

  First, a terminal device serving as a client generates a query. The query generation method may be a general method using character string input, a method using voice input, or a method of displaying an image and using the feature amount as a query.

  Then, based on the generated query, the distribution device serving as a server searches for an appropriate one, and the distribution base station distributes the search result list information using the present invention to the terminal device according to the search result. Then, the terminal device interprets the markup language of the acquired information, converts the character string in the range sandwiched between specific tags into the above-mentioned identifiers such as phonemes and phonemes, and converts them into voice input information spoken by the user. Therefore, phonemes and phoneme pieces are acquired to form phonetic symbol strings such as phoneme strings and phoneme string strings, and matching processing based on the phoneme strings and phoneme string strings is performed.

  And after performing each process according to operations with high degree of coincidence, keywords, and identifier strings, construct a query with the specified identifier string, and send those queries to the distribution base station to perform a search Then, a search with voice control using a markup language is performed. At this time, the constructed query may be used for a search by a single device.

  In FIG. 16, the insertion and setting of identifier symbol strings such as phonemes and phonemes for speech processing are performed by the markup language interpretation on the terminal side. Operation and processing using the present invention by manually inserting in advance, constructing and inserting identifiers on distribution base station side devices and devices linked with them, or using a single information processing device It is also possible to add variables and attributes, add tags, change the contents of markup language information, or change, add, or delete dictionaries related to identifiers and feature quantities.

In addition, regarding the new identifier combination generated by the present invention, search is performed by giving a word based on an arbitrary name, or by providing a symbol string by a phoneme or phoneme to an arbitrary name, and corresponding to voice control, It may be possible to operate by giving a symbol string using phonemes or phoneme as a keyword for operation, or to associate such a phonetic symbol string with an advertisement, or by adding an advertisement attribute to a URL of a related advertisement. The advertisement and the utterance symbol string may be associated with each other by describing them in the same tag as the utterance symbol attribute.

  In addition, identifiers related to images displayed in the browser, and phonemes and phonemes of keywords to be controlled are converted to be easily usable on terminals by using symbol strings and IDs obtained by compressing identifier strings and identifier strings. Sending information and using the voice in the device without interpreting the markup language can be easily considered, such dictionary information can be acquired via a communication line with a remote control or mobile phone, acquired by e-mail, etc. A highly convenient operating environment may be configured by downloading from the above device.

  In addition, a recognition dictionary based on the file name may be configured by describing the file name as a phoneme string, or the file name is set by the phoneme string / phoneme string sequence and the phoneme / phoneme segment recognition is used in the markup language. It is possible to select various information, and in addition to the stock price search by coupon number or company name or the product search by JAN code, the search by product name, performer name, company name or region name is also performed. You can change the phoneme dictionary according to the location and device, change the phoneme dictionary according to the page, or change the content image, the text in the page unit, and the frame or moving image 1 The phoneme dictionary may be changed according to scene units that span frames as frames or multiple frames of moving images.

  In addition, when indexing according to the present invention is performed for an information format having a chunk header such as the RIFF format as shown in FIG. 17, a tag such as “PRON” is arbitrarily provided as a chunk header, and a phoneme string or phoneme segment is provided. Columns may be written, and if the contents are regular files, general metadata such as the file name, production date and time, producers, etc., or 2D / 3D images, the names of display objects and people, You can list the phonemes and phonemes that accompany the name of the parts, if they are audio files, you can list the phonemes and phonemes of the appearing speech, or you can write the lyrics and titles of music files in phonemes and phonemes, Phonemes / phonemes may be written in the free description area, or may be used for searching.

<Modification>
In this embodiment, the phoneme is mainly used as a phonetic symbol. However, the phoneme portion is changed to a phoneme segment, or the phoneme sequence is a phoneme string of different languages such as an international phoneme symbol, English, or Chinese. Or a selection range or branch content in markup language processing may be configured depending on whether a round image is presented to a computer or a triangular image by an identifier based on the recognized image. , Search based on the presented photos, or perform voice operations by expanding the names associated with the feature quantities of the photos into phoneme strings and phoneme strings and sending and receiving them in markup language and dedicated symbol strings Alternatively, Unicode, JIS code, or ISO code, which is a character encoding method other than ASCII code, may be used, and an arbitrary numerical ID based on phonemes or phonemes is given. And it may be used a unique character code system.

  Further, the identifier string and identifier used in the present invention are scale type, instrument type, mechanical sound type, environmental sound type, image type, face type, facial expression type, person type, action type, landscape type, display position type, character symbol An attribute, a variable name, or an identifier may be designated based on a designation used for identification by one or a plurality of identifiers such as a classification, a label classification, a shape classification, a graphic symbol classification, and a broadcast program classification. The identifiers may be captured as continuously described according to time series transitions, converted into phonemes or phoneme strings based on their names, or used in GET methods or POST in CGI. A search result may be obtained by transmitting those identifiers and identifier strings using a method.

  In this way, the feature names related to speech and their identifiers and identification functions, the feature names and identifiers and identification functions related to still images and moving images, and the mark and language that can be operated by speech by giving attributes and variables to the markup language In addition to searching for content, public information, map information, and the like, the user can realize device control by voice using a phonetic symbol string provided by an information processing device that processes such a markup language. It can be applied to merchandise sales, reservation status, viewing status, questionnaires, surveillance camera images, satellite photos, blogs, robots and equipment control. Note that the search and processing results may be returned from the server to the client using any markup language in response to these requests.

<Procedure example of information processing apparatus used for terminal and base station>
The present invention is also applicable to a processing system using a server and a client related to a base station and a terminal. This device and the terminal are configured as shown in FIG. 18, and are connected via a communication line to acquire information from other devices and distribute information to other devices, so that information related to voice operations can be obtained. It can be exchanged to improve user convenience. The shared line used here is not limited to the Internet but may be used regardless of wired wireless as long as it is a wide area communication network such as a LAN or a telephone line or an indoor communication network, and the target devices are home appliances, remote controllers, robots, It may be a mobile phone or a communication base station, or may be a service such as a web service, a telephone service, or an EPG distribution, and can be implemented for any device or service.

  Further, it is configured by a user terminal, a distribution base station, a device such as a robot controlled by the terminal or the base station, and a remote controller to be controlled, and the remote control and the robot may be used as one form of the terminal or one form of the base station. The user speaks voice to the terminal, and performs an arbitrary processing procedure as described below for recognition processing at the terminal or the base station.

  In the first method, a feature amount is extracted from voice obtained by speech or a captured image, the feature amount is transmitted to a target relay location or base station device, and the base station device that has received the feature amount A phoneme symbol string / phoneme symbol string and other image identifiers are generated according to the feature amount. Then, based on the generated symbol string, a matching control means is selected and executed.

  In the second method, feature amounts are extracted from speech obtained by utterance or captured video, and identifiers associated with recognition such as phoneme symbol strings / phoneme symbol strings and other image identifiers are generated in the terminal. The received symbol string is transmitted to the target relay location or base station apparatus. Then, the base station apparatus to be controlled selects and executes a matching control means based on the received symbol string.

  The third method performs feature amount extraction from voice obtained by speech or captured video, recognizes phoneme strings / phoneme symbol strings, and other image identifiers based on the feature amounts generated in the terminal, The control content is selected based on the recognized symbol string, and transmitted to the base station device that controls the control method and the device that relays information distribution.

  And the 4th method transmits to the base station apparatus which controls the voice waveform and image of the audio | voice obtained by utterance using the terminal, and the imaged image | video as it is, and phoneme symbol string and phoneme symbol in the apparatus to control The system recognizes the sequence and other image identifiers, selects the control means based on the recognized symbol sequence, and the selected relay station or base station apparatus performs the selected control. The same applies to sound and video features and identifiers such as environmental sounds.

  At this time, the terminal may simply transmit only a waveform, transmit a feature amount, transmit a recognized identifier string, or transmit a processing procedure such as a command or a message associated with the identifier string. The client server model may be implemented by changing the configuration of the distribution base station according to the transmission information, and the transmission side and the reception side can transmit and receive each other, and the image related to the identifier described above Features such as voice, voice, and motion are assigned to markup language attributes, and the degree of coincidence between the feature amount extracted from the information provided by the user and the feature amount extracted from the distribution information is evaluated and searched Information processing with arbitrary control and user interaction may be realized by performing recognition, or proper nouns using image recognition dictionaries such as faces and fingerprints and phonetic symbol strings made up of phonemes and phonemes Recognition dictionary with It may be subjected to a personal authentication by the slogan by associating an acoustic model based on the phoneme or phoneme of each finisher.

  In addition, the command dictionary for converting to an associated processing procedure based on the input phoneme sequence or phoneme segment sequence may be on the terminal side or the distribution base station side, and a new control command, media type, format A symbol string such as a phoneme symbol string and an image identifier relating to the type and the device name may be transmitted / received, distributed, or exchanged using a markup language such as XML or HTML, RSS, or CGI, which will be described later.

  A more specific procedure for distributing and exchanging dictionary information will be described. First, by extracting feature quantities and identifiers, or configuring evaluation functions, other terminals and devices in an environment connected to a communication line regardless of any infrared, wireless LAN, telephone line, wired LAN, etc. Exchange information with other classes.

  Next, a case where a phoneme piece is used as processing on the terminal side will be described as an example. The terminal side device analyzes the given voice and converts it into a feature value. Next, the converted feature amount is recognized by various recognition technologies such as HMM and Bayes and converted into an identifier.

  At this time, the converted identifier is information indicating a phoneme, a phoneme piece, and various image identifiers. However, as described elsewhere, if it is a voice, it is a phoneme, an environmental sound, a scale, or an image if it is an image. Or an identifier based on the operation. Then, based on the obtained identifier, a dictionary based on phonemes and phoneme symbol strings is referred to by DP matching to select an arbitrary processing procedure, and the selected processing procedure is transmitted to the target device for control. Thus, it is possible to use the mobile terminal as a remote control using the present invention, or to control home appliances by robots, and the utterance sound notation for smooth communication with the other party at the communication destination An interactive device with a disabled person may be configured by providing a display and a Braille output unit.

  Depending on the CPU performance of the terminal, the information processed in such a procedure can be transmitted as it is without converting natural information such as video and audio into feature values, or can be converted into feature values. It can be sent, sent after being converted into an identifier, sent after the control information is selected, can be selected, and any conversion level can be selected, and the receiving side can process based on information from any state It is configured as a possible receiving side device, and it transmits to the distribution station and control device based on the acquired information, or performs arbitrary processing such as search, recording, mail distribution, machine control, device control based on the acquired information Also good.

  Then, for use in search processing, an identifier string, a character string, and a feature amount that are appropriately used as a query are acquired by recognition, and are transmitted to the distribution-side base station to obtain information according to the query. At this time, advertisements and advertisements may be displayed during the communication waiting time and the search waiting time, and when performing control by voice, a control dictionary is constructed so that control items can be selected by communication. The dictionary information is exchanged / acquired, and the procedure may be performed using the P2P technology, or the information may be sold and distributed.

  In addition, this control command dictionary can be freely renewed and reused by configuring it with arbitrary identifiers such as phonemes and phonemes as described above, feature quantities, and device control information. It may be possible to update trendy search keywords by replacing or reconfiguring dictionary information for search that associates arbitrary identifiers with feature amounts, and the position and configuration of these content information The recognition dictionary information changed accordingly may be a dictionary for face recognition, a dictionary for fingerprint recognition, a dictionary for character recognition, or a dictionary for figure recognition.

  In the control command dictionary, infrared control information to be transmitted to a product that can be controlled by a conventional infrared remote controller is selected as device control information, or a series of operations are continuously performed as a batch process by combining those control information. It may be implemented or only the feature amount information may be transmitted to the speech-controlling information processing apparatus without recognizing the identifier according to the CPU performance of the apparatus.

  Any device capable of providing an infrared remote control signal from a voice information via a conversion dictionary by combining control by an infrared remote controller even with a conventional device that cannot perform voice control by such a method, or a device capable of voice control. In addition to being able to recognize and control commands based on feature quantities and speech waveforms, changing the control dictionary to improve performance, checking with version information of the control dictionary, It is possible to confirm the state of the device.

  In addition, by introducing the server / client model in this way, the server and the client can be divided into arbitrary processing steps, connected by communication, and exchanged arbitrary information between the server and client. Indexing can be realized.

  Also, in order to perform personal authentication by recognizing faces, fingerprints, and voice features, use a phoneme recognition dictionary that uses acoustic models, standard parameters, and standard templates that match the phonetic recognition dictionary information. Therefore, the recognition dictionary with images and sounds can be changed according to the user, and highly versatile personal authentication can be realized. Therefore, a service using various operations and operations such as charging, locking and unlocking keys, selecting a service, granting use, and using a copyrighted work uses the present invention. This can be realized by using an information terminal that performs recognition.

  In addition, using a terminal that performs recognition using the present invention, it is acquired by a client terminal such as a DVD recorder, a network TV, an STB, an HDD recorder, a music recording / playback apparatus, or a video recording / playback apparatus from a core server at a communication destination. Information is transferred to mobile terminals and mobile phones via wireless communication such as infrared communication, FM, VHF frequency band communication, 802.11b, Bluetooth (registered trademark), ZigBee, WiFi, WiMAX, UWB, WUSB (Ultra Wide Band), etc. By providing information, it is possible to use EPG, BML, RSS, teletext data broadcasting, TV video, teletext on mobile terminals and mobile phones, voice input, character string input, swing mobile terminals and mobile phones. The operation of the information terminals, home appliances, information devices, robots, and instructions for control procedures are performed by moving the Terminal or a mobile phone may be subjected to remote operations such as and go an indication of the general consumer electronics as a remote control from the client terminal and information equipment and operation of the robot and control procedures.

  In addition, when a phoneme dictionary based on an attribute extracted in association with an input item such as an HTML FORM tag in information configured in a markup language is registered in the recognition dictionary information 206 in advance, the recognition priority is set. It may be changed to a phoneme dictionary registered in advance, or a recognition target may be limited using a dictionary registered in advance.

  Also, for phonetic symbol strings such as phoneme strings and phoneme string strings based on attribute variables of information configured in markup language, a plurality of phoneme strings and phoneme string phonetic symbol strings that may be recognized simultaneously are written together. Thus, a plurality of recognition dictionary information 206 may be configured, and the same recognition dictionary information 206 may be used for input items having the same attribute variable.

  In addition, a plurality of words that may be recognized using a plurality of phoneme strings, phoneme string strings, and phonetic symbol strings may be described as attribute variables. For example, a classifier such as an arbitrary unit may be represented as a phoneme string, phoneme string string, or table. When it is expressed as a phonetic symbol string, the recognition dictionary information 206 is switched to a number only, switched to a dedicated dictionary according to a menu item, or switched to a limited proper noun dictionary such as a place name or a station name. Also good.

  Further, according to the character code selected to be used for the display based on the markup language, the Bayes discriminant function or HMM used in the step (S506) of converting the speech waveform into an identifier by phonetic symbols such as phonemes and phonemes. Prepare multiple values for each language, such as the standard pattern to be used and the standard template, the values obtained as learning results, the values by eigenvalues / eigenvectors, and the values by covariance matrix. It is possible to support multiple languages by switching to a word standard template, or select a standard template to be used for recognition by acquiring information about the user's information processing device, operating system, or browser-specific language environment from multiple languages You may do it.

  In addition, according to the user's designation, for example, the utterances and dialects that arise due to the difference between the user's native language and native language and the language recognized in the device environment used, such as a standard template when a Russian speaker speaks Chinese The system may be configured so that a standard template can be selected so as to correspond, or the template may be configured by learning the utterance and dialect from the user's utterance.

  Alternatively, a method may be used in which the content of the cookie or session is converted into a phonetic symbol string using phonemes or phoneme pieces according to the attribute variable and the recognition dictionary information 206 is switched, or a phoneme string or phoneme recognized from speech is used. Arbitrary variable transmission such as a method that uses a script such as AJAX, a feature value extracted from a phonetic symbol string such as a single row, a script, a CGI (Common Gateway Interface) parameter, or a socket communication by a program A phonetic symbol string such as a phoneme string or phoneme string recognized based on a voice feature received by the base station or a phonetic symbol string made up of phonemes or phonemes received by the base station The user's utterances are discriminated by using, and arbitrary processing is performed, or search processing of content information, advertisement information, and regional information by configuring search conditions It may be carried out or.

  And display information or music including terminal picture, character, icon, CG (Computer Graphics) or output sound information such as warning sound or robot that changes in accordance with the recognition process of these phonetic symbol strings Feature extraction from motion control information of devices such as mechanical devices, communication devices, electronic devices, electronic musical instruments, recognition dictionary information 206 for recognizing audio, still images, moving images, etc., or video, audio, images For example, a base station can send information for combining and updating arbitrary information such as processing procedure information such as a program, a script, and a function expression, or perform arbitrary processing in a terminal device. May be.

  In addition, when phonetic symbols such as phonemes and phonemes obtained as recognition results are divided into a plurality of frames to obtain recognition results that are continuous in time series, recognition for a plurality of phonemes and phonemes across multiple frames is performed. The distance between the input speech acquired as a result and phonetic symbols such as phonemes and phonemes is used as a feature value to configure the parameters of the Bayes discriminant function, or to span multiple frames to degenerate in time series By configuring the parameters of the HMM using distance information acquired as a recognition result for a plurality of phonemes and phonemes, or by evaluating an identifier evaluated as the first place by a recognition result in a plurality of frames by DP or the like Dynamic speech recognition may be configured in combination with the technology used for conventional speech recognition.

  More specifically, first, markup language information is acquired in steps (S401 and S501) of acquiring content information, and tag attribute detection means for detecting a tag from the markup language information and detecting a tag attribute from the tag is combined. A step of extracting a phonetic symbol string (S402, S502) for extracting a phonetic symbol string associated with the detected attribute and registering it in the recognition dictionary information 206 as a phonetic symbol string used for recognition (S403, S503) To implement. These steps (S401 to S403, S501 to S503) can be produced by character string evaluation processing and detection processing, but operations performed in a conventional speech recognition system, a search by phoneme recognition, a WEB browser by phoneme sequence recognition, and the Internet environment It is not used for searching or browsing content information.

  Next, the step of waiting for the speaker's voice input (S504) is performed, and the step of extracting the feature amount performed by the calculation unit according to the start of the voice input (S505) is performed, and phoneme recognition and / or phoneme pieces are performed. Based on the phonetic symbol recognition program including recognition, a step (S506) of converting the feature value into the identifier by recognition of the phonetic symbol is performed. It is generally known that this step (S506) uses a distance evaluation function or a statistical test method, uses a learning result using multivariate analysis, or uses an algorithm such as HMM. A phonetic symbol string is formed by time-series continuation based on the recognized phonetic symbols.

  Next, the phonetic symbol string is compared with the recognition dictionary information 206 based on the phonetic symbol string extracted from the attribute attached to the tag of the markup language, and the recognition dictionary information 206 is searched, thereby comparing the phonetic symbol string. A step (S507) of evaluating the degree of coincidence between the phonetic symbol strings is performed, and it is evaluated whether or not it is appropriate as a recognition target. For the comparison to determine whether or not it is a recognition target, an algorithm that can be used for symbol string comparison and evaluation such as DP, HMM, and automaton may be arbitrarily used. Various methods have been invented and devised from the past.

  As a result, on the basis of identification information such as a character string or ID associated with the phonetic symbol string specified from the recognition dictionary information 206, a character string is displayed, an arbitrary process is performed, information is exchanged, etc. The recognition process using the phonetic symbols is realized by generating an event, changing the status, or causing the machine device to perform an arbitrary operation, and executing the arbitrary process (S508). As a result, information processing using speech different from conventional grammar dependency and static registered word dependency can be realized.

  At this time, a step of evaluating a match with the phonetic symbol string based on the type information for distinguishing the input items detected by the tag attribute detecting means and having a plurality of recognition dictionary information 206 by the phonetic symbol string ( A phonetic symbol recognition result obtained from the phonetic symbol string and the speech waveform registered in the recognition dictionary information 206 selected according to the attribute of the input item to be recognized while switching the recognition dictionary information 206 used in S507) The recognition efficiency can be improved by limiting the phonetic symbol strings included in the recognition dictionary information 206 to be recognized when the matching score is evaluated by the symbol string comparison evaluation.

  Then, when switching the recognition dictionary information (206) that evaluates the voice input according to the item to be input by the information processing apparatus, the recognition dictionary information (206) of the word associated with the attribute name or attribute is appropriately selected. Therefore, if the information acquired from the attribute is “book”, the unit “book (s / a / ts / u | v / o / ly / u / m)” is used, and the “ Select the recognition dictionary using the phonetic symbol string associated with “number” as the search target for the recognized phonetic symbol string, or if the information obtained from the attribute is “station name”, the suffix Using phonetic symbol strings associated with "station (e / k / i | s / u / t / e / i / sh / o / n)" and "nouns used as station names" The dictionary can be selected as a search target for recognized phonetic symbol strings, and the information obtained from the attribute is a zip code or telephone number If there are nouns included in a specific framework, simply select a recognition dictionary using a phonetic symbol string of a number as a search target for the recognized phonetic symbol string. Dictionary information 206 to be searched for recognized phonetic symbol strings by switching the plurality of recognition dictionary information 206 according to the attribute associated with the item to be input to the user by restricting Recognition performance can be improved by classifying 206 based on attributes.

  In addition, by outputting voice of words associated with attribute names and attributes in accordance with the order in which voice input is performed and the selection of items that have not been input, the information processing apparatus can input the items to be input. The recognition performance may be improved on the basis of the attribute to be classified by switching the plurality of recognition dictionary information 206 while prompting the item.

  Then, if there is a continuous voice, the next content or markup language is changed in accordance with the step of repeating the conventional processing (S509), the processing step associated with year recognition (S508), or the status change in the device accompanying other external operations. The step (S510) for evaluating whether or not to acquire is executed, and this process ends depending on the situation.

  Note that the status change in the device changes depending on other programs and processes when functioning as a part of other programs or multithread programs, and is used for multithread programs, event-driven programs, etc. Assuming that, it is easy to understand, and similarly, it is possible to rewrite the status or generate an event for another process or another program in the arbitrary processing of the present invention.

  In addition, a character string displayed by designating a markup language by using the method of the present invention or a character string associated with a displayed image or an image feature or output voice / music, voice / music, etc. Arbitrary information displayed by converting various character strings such as character strings associated with acoustic features into phonetic symbol strings and registering them in the dictionary is detected by searching the phonetic symbol strings based on the user's input voice and character strings. In addition to content-related information associated with the detected information, information related to the location of arbitrary information such as advertisements, videos and links, music and voice, etc., it is possible to provide information by user operation, etc. In addition to voice and text input, you can select a character string from a list such as a menu, or use a character string based on a button label in button operation. It may be carried out in interest.

Also, as in the example below (example)
<img href = ”./ flower_lily1.jpg”
recog_dic_type = ”flower_name” recog_dic_url = ”./ flower.prono”
name = ”lily” prono = ”l / i / l / i / y”>
When reading a phonetic symbol according to an attribute related to the phonetic symbol dictionary included in the tag, the phonetic symbol recognition dictionary or table is read according to information such as URL, URI, IP address, directory path, etc. indicating the location of the phonetic symbol dictionary. Reuse frequency is high by using information based on attributes such as “recog_dic_uri” indicating the position and location of information such as a phonetic symbol dictionary, or using information indicating the type of target recognized in the dictionary such as “recog_dic_type” By distinguishing phonetic symbol dictionaries and the like, dictionary information based on phonetic symbol strings and acoustic characteristic template dictionary information for recognition may be provided in association with attributes acquired from the markup language.

  Also, it is possible to save the dictionary information read in the past to some extent by a method generally called a cache, and save the trouble of increasing the priority of the dictionary and reading it again when the above-mentioned attribute indicates a specific word range. Alternatively, a phonetic symbol dictionary that is read for each page as a separate file, such as a style sheet, and is associated with an ID may be incorporated, or a phonetic symbol dictionary that is associated with an ID described in a header block may be incorporated. It is also possible to incorporate a phonetic symbol dictionary given as an attribute for each tag, or to include a phonetic symbol dictionary in header information when reading via a file or communication line. It can also be used as

  In addition, “acoustic OFDM” that can embed text data in speech may be used to embed phonetic symbol strings in speech waveform information, or speech data may be restored by restoring the embedded phonetic symbol strings and related markup language information. Can be used to search for phonetic symbols and display related information in subtitles, etc., so use phonetic symbol strings demodulated from extremely general audio data such as radio and television for searching You can also

  In addition, a database indexed by a phonetic symbol string searched as a search target using a phonetic symbol string acquired by phonetic symbol recognition is a logical space between phonetic symbol strings based on a plurality of keywords. It is also possible to use a combination based on the above, or a configuration in which logicality can be expressed using a Boolean model, and a combination of these can constitute a query and provide it to a database to obtain a search result.

  In this way, unlike the conventional method of stochastically connecting a word and a voice feature group by HMM or the like, the present invention represents a table by associating a phonetic symbol and a voice feature by a distance based on a probability based on a Bayes discriminant function or the like. By acquiring the phonetic symbol string and directly associating the acquired phonetic symbol string and the word character string via the markup language, it is more efficient by restricting the words to be recognized compared to conventional general recognition. It is possible to dynamically provide dictionary information for enabling real recognition in markup languages, and use phonetic symbol strings without using words directly in queries. A database using HMM or DP matching may be constructed and searched by using phonemes or phoneme symbol strings.

  Further, not only attributes based on phonetic symbols such as utterance phonemes and utterance phoneme changes, but also image identifiers, identifiers after image recognition, image-related feature amounts, motion feature amounts, motion identifiers, and the like may be used as attributes.

1 is a block diagram of an information processing apparatus using the present invention. The figure which showed an example of the data structure of recognition dictionary information. The figure which showed the operation | movement flow of the phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure for demonstrating the operation | movement of a phonetic symbol provision process. The figure which showed the operation | movement flow of recognition dictionary update processing. The figure which showed the data structure from which recognition dictionary information differs. The figure which showed the operation | movement flow of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure for demonstrating operation | movement of recognition dictionary information update processing. The figure which showed the operation | movement flow at the time of applying to a server client model. The figure which showed the operation | movement flow at the time of applying to a server client model. The figure for demonstrating the modification in this embodiment. The figure for demonstrating the modification in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 10 Control part 20 Storage part 202 Content information 204 Phonetic symbol conversion table 206 Recognition dictionary information 208 Phonetic symbol addition program 210 Recognition dictionary information update program 212 Voice operation program 30 Communication part 40 Input / output part 50 Operation part 60 Display section

Claims (10)

Content information acquisition means for acquiring content information including character information and / or meta information;
Recognized phonetic symbol string detecting means for detecting a recognized phonetic symbol string consisting of phonetic symbols from the content information acquired by the content information acquiring means;
Recognition dictionary information generating means for generating recognition dictionary information using the recognition phonetic symbol string;
An information processing apparatus comprising: Content information acquisition means for acquiring content information including character information and / or meta information;
A development target character string detection unit that detects a development target character string based on character information and / or meta information from the content information acquired by the content information acquisition unit, and stores the character string and a phonetic symbol in association with each other. Phonetic symbol storage means for
A phonetic symbol conversion means for converting the expansion target character string into a recognized phonetic symbol string by referring to the phonetic symbol storage means;
Recognition dictionary information generating means for generating recognition dictionary information using the recognition phonetic symbol string;
An information processing apparatus comprising:   3. The information processing apparatus according to claim 2, further comprising content information storage means for storing the phonetic symbol converted by the phonetic symbol conversion means to the content information by storing the content information. .   4. The transmission apparatus according to claim 1, further comprising a transmission unit configured to transmit the content information stored by the content information storage unit and the recognition dictionary information generated based on the content information to another information processing terminal. The information processing apparatus according to any one of the above. Voice input means for inputting voice;
Feature quantity extraction means for extracting the feature quantity of the voice input by the voice input means;
A feature value phonetic symbol conversion unit for converting the feature amount extracted by the feature amount extraction unit into a phonetic symbol;
Evaluating the phonetic symbols converted by the feature value phonetic symbol conversion means and the phonetic symbols constituting the recognized phonetic symbol string included in the recognition dictionary information, and corresponding to the most similar phonetic symbols Processing execution means for executing predetermined processing;
The information processing apparatus according to claim 1, further comprising: The content information includes phoneme information and / or phoneme piece information,
The processing execution means evaluates the phonetic symbols converted by the feature value phonetic symbol conversion means and the phonetic symbols constituting the recognized phonetic symbol string included in the recognition dictionary information, and the most similar table is evaluated. 6. The information processing apparatus according to claim 5, wherein information is presented by voice utterance to a user corresponding to a phonetic symbol.   The information processing apparatus according to claim 1, wherein the phonetic symbol is a phoneme or a phoneme piece.   The information processing apparatus according to claim 5, wherein the executed process is an authentication process associated with phoneme recognition. On the computer,
A markup language interpretation step of interpreting information described using a markup language, and an attribute acquisition step of acquiring an attribute specified by the interpretation;
A phonetic symbol extraction step of extracting a phonetic symbol sequence and / or a phoneme sequence and / or a phoneme segment sequence associated with the attribute acquired by the attribute acquisition step;
A dictionary change step for changing a phoneme string dictionary used in the phoneme recognition unit by the phonetic symbol extraction step;
A program characterized by realizing. On the computer,
A markup language interpretation step of interpreting information described using a markup language, and an attribute acquisition step of acquiring an attribute specified by the interpretation;
A phonetic symbol extraction step of extracting a phonetic symbol sequence and / or a phoneme sequence and / or a phoneme segment sequence associated with the attribute acquired by the attribute acquisition step;
An information type evaluation step for evaluating the type of information input by the user based on the attribute acquired by the attribute acquisition step;
A dictionary change step for changing a phoneme string dictionary used in the phoneme recognition unit by the information evaluation step;
A program characterized by realizing.

Images