JP2007251581A - Voice transmission terminal and voice reproduction terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the technique which provides versatile or various communication even to a voice speech. <P>SOLUTION: When an operator of a cellular phone set 10 generates words, a voice input part 11 inputs the voice, and a voice recognition part 12 performs voice recognition. A voice synthesizing part 13 obtains voice data (a sound effect, BGM, etc.) coordinated with a voice recognition result by referring to a voice material data base 101, and synthesizes the voice data obtained from the data base to the voice input from the voice input part 11. A communication part 14 transmits synthesized voice to a cellular phone set 20. The cellular phone set 20 performs the same processing as a normal speech voice, and receives the synthesized voice to reproduce it from a speaker. Thus, the voice obtained by synthesizing the sound effect and BGM to the words of a speaking person is reproduced by a receiving side. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique used in a device such as a mobile phone that transmits and receives audio. More specifically, the present invention relates to a voice or data transmission technology that enables communication with a variety or variety.

  Various functions have been added to mobile phones in recent years, and higher functionality has been advanced. In addition to the original voice call function, e-mail transmission / reception function, Internet connection function, sound playback function, movie playback function, camera function, movie function, etc. are added.

  For example, by connecting to the Internet and downloading a sound file or movie file, it is possible to play the sound or movie on a mobile phone. It is also possible to shoot still images and videos using the camera function and movie function and view them on a mobile phone, and to send still images and videos to friends using e-mail .

  As described above, with the increase in the number of functions of mobile phones, it is possible to realize new communication methods and provide users with new ways of enjoying by using these functions in combination. However, the voice call function has not been changed conventionally. In other words, there are many techniques aimed at improving the quality of voice to be transmitted and reducing noise, but these are only for the purpose of accurately transmitting the voice spoken by the caller.

  Patent Document 1 below relates to a technique for outputting sound based on a human emotional state. A voice signal is acquired from the voice flowing on the telephone line, and the emotional state is judged from the acquired voice to output appropriate music.

JP 2005-352151 A

  As described above, the current situation is that a simple method has been adopted for voice calls. Therefore, if there is technology to enhance expressiveness for voice calls and express communication in various ways, diversity and variety can be added to communication using mobile phones, etc. It is expected that mobile phones can be used as communication tools with higher added value.

  The technique disclosed in Patent Document 1 determines an emotional state from voice received by a voice output device. Accordingly, since the receiving device must have a function for recognizing voice, in order for many users to use such a function, the telephones used by all those users need to support this function. There is. Therefore, it is not easy to construct a mechanism that allows many users to receive services using this technology widely.

  In view of the above problems, an object of the present invention is to provide a technique that enables communication with a great variety and variety even for voice calls.

  In order to solve the above problem, the invention according to claim 1 is a speech material database for storing speech material data associated with a speech recognition result, speech input means, and speech recognition for speech input from the speech input means. Voice recognition means for performing voice recognition, and voice material data associated with a recognition result by the voice recognition means from the voice material database, and means for synthesizing the acquired voice material data with the voice input from the voice input means And communication means for transmitting the voice synthesized with the voice material data to the partner terminal.

  According to a second aspect of the present invention, in the audio transmitting terminal according to the first aspect, the video material database for storing the video material data associated with the voice recognition result and the recognition result by the voice recognition means Means for acquiring the video material data being recorded from the video material database, and the communication means sends the video acquired from the video material database together with the audio synthesized with the audio material data to the partner terminal. It is characterized by transmitting.

  According to a third aspect of the present invention, in the voice transmitting terminal according to the first aspect, a vibration setting database in which a voice recognition result and vibration setting data are associated with each other, and a vibration setting based on the recognition result by the voice recognition means Means for determining data, wherein the communication means transmits vibration setting data to the partner terminal together with the voice synthesized with the voice material data.

  According to a fourth aspect of the present invention, there is provided the voice transmitting terminal according to the second or third aspect, further comprising means for acquiring terminal information of the partner terminal, and video according to a type of the partner terminal. The transmission of material data or vibration setting data is stopped.

  According to a fifth aspect of the present invention, in the voice transmission terminal according to any one of the second to fourth aspects, the communication unit further transmits a recognition result by the voice recognition unit to the counterpart terminal. Features.

  The invention according to claim 6 is a video material database that stores video material data associated with a voice recognition result, a voice input unit, a voice recognition unit that performs voice recognition on a voice input from the voice input unit, Means for acquiring video material data associated with the recognition result by the voice recognition means from the video material database; and the video acquired from the video material database together with the voice input from the voice input means. And a communication means for transmitting to.

  According to a seventh aspect of the present invention, there is provided a vibration setting database in which voice recognition results and vibration setting data are associated with each other, voice input means, voice recognition means for performing voice recognition on voice input from the voice input means, and the voice It comprises: means for determining vibration setting data based on the recognition result by the recognition means; and communication means for transmitting the vibration setting data to the partner terminal together with the voice input from the voice input means.

  According to an eighth aspect of the present invention, in the voice transmitting terminal according to any one of the first to seventh aspects, the voice recognition result is obtained by converting the voice input by the voice input means into a text and / or the voice. It includes information related to the tone of the sound determined from the sound input by the input means.

  According to a ninth aspect of the present invention, in the voice transmitting terminal according to the first aspect, the voice material database is stored in a memory card, and the voice material database is inserted by inserting the memory card into the voice transmitting terminal. It can be used.

  According to a tenth aspect of the present invention, in the audio transmitting terminal according to the second or sixth aspect, the video material database is stored in a memory card, and the memory card is inserted into the audio transmitting terminal to insert the memory card. The video material database can be used.

  According to an eleventh aspect of the present invention, in the voice transmitting terminal according to the third or seventh aspect, the vibration setting database is stored in a memory card, and the memory card is inserted into the voice transmitting terminal to insert the memory card. A vibration setting database is available.

  A twelfth aspect of the invention is a terminal that receives a voice transmitted from the voice transmitting terminal according to the first aspect, and outputs a voice synthesized with voice material data from a speaker.

  A thirteenth aspect of the invention is a terminal for receiving voice and data transmitted from the voice transmitting terminal according to the second aspect, wherein the video material data received while outputting the synthesized voice data from the speaker. Is output to a monitor.

  The invention according to claim 14 is a terminal for receiving the voice and data transmitted from the voice transmitting terminal according to claim 3, wherein the received vibration setting is output while outputting the voice synthesized with the voice material data from the speaker. The vibrator is driven based on the data.

  A fifteenth aspect of the present invention is a terminal that receives the synthesized voice and the voice recognition result transmitted from the voice transmitting terminal according to the fifth aspect, wherein the receiving terminal is video material data associated with the voice recognition result. A receiving-side video material database for storing the received video material data, when receiving the video material data from the audio transmitting terminal, the video data received together with the synthesized audio is reproduced or based on the received recognition result. The corresponding video material data is acquired from the receiving-side video material database, and it is possible to select whether to reproduce the acquired video material data together with the synthesized audio.

  The invention according to claim 16 is a terminal that receives the synthesized speech and the speech recognition result transmitted from the speech transmitting terminal according to claim 5, wherein the receiving terminal associates the speech recognition result with the vibration setting data. A receiving side vibration setting database, and when the receiving terminal receives vibration setting data from the voice transmitting terminal, it plays a synthesized voice and drives or receives a vibrator based on the received vibration setting data. The vibration setting data is determined by referring to the reception side vibration setting database based on the voice recognition result, and it is possible to select whether to play the synthesized voice and to drive the vibrator based on the determined vibration setting data. And

  According to the seventeenth aspect of the present invention, there is provided a voice material database that stores voice material data associated with key operations, a voice input unit, and voice material data associated with key operations input during a voice call. Means for synthesizing the obtained voice material data with the voice inputted from the voice input means and the voice inputted from the voice input means; and a communication means for sending the voice synthesized with the voice material data to the other terminal. Features.

  According to the eighteenth aspect of the present invention, there is provided a video material database for storing video material data associated with key operations, a voice input means, and video material data associated with key operations input during a voice call. It is characterized by comprising means for acquiring from a video material database and communication means for transmitting video material data acquired together with the audio input from the audio input means to a partner terminal.

  According to a nineteenth aspect of the present invention, the vibration setting database for storing the vibration setting data associated with the key operation, the voice input means, and the vibration setting data associated with the key operation input during the voice call are Means for acquiring from a vibration setting database; and communication means for transmitting the vibration setting data acquired together with the voice input from the voice input means to a partner terminal.

  The voice transmitting terminal according to the present invention acquires the voice material data associated with the voice recognition result, and synthesizes the acquired voice material data with the voice input from the caller. Then, the voice synthesized with the voice material data is transmitted to the partner terminal. Thereby, when a caller utters a word, a sound effect or BGM associated with the word is synthesized and transmitted, so that it is possible to enhance the expressive power in communication. Further, since the synthesized terminal is generated and transmitted by the terminal on the transmission side, the terminal on the reception side only needs to have normal voice reception and playback functions. That is, since it is not necessary for both terminals that perform transmission and reception to correspond to functions, a service that is easy to introduce and spread can be realized.

  Furthermore, the audio transmission terminal of the present invention acquires video material data associated with the audio recognition result, and transmits the video data together with the synthesized audio to the partner terminal. Therefore, it is possible to convey the emotion and intention of the caller in an expressive manner using the video.

  Furthermore, the voice transmitting terminal according to the present invention determines the vibration setting data based on the voice recognition result, and transmits the vibration setting data to the partner terminal together with the synthesized voice. Therefore, it is possible to convey the emotions and intentions of the caller in a more realistic expression form by using vibration.

{First embodiment}
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a usage image of a communication system using a mobile phone of the present invention. The mobile phone 10 and the mobile phone 20 have a function of connecting to a network and transmitting / receiving data in addition to a voice call function. In the present invention, when a caller operating the mobile phone 10 speaks, various special effects are given to the voice and the voice is expressed on the mobile phone 20. For example, as shown in the figure, when the operator of the mobile phone 10 utters the word “liar”, the word “liar” is reproduced on the mobile phone 20 and an image related to the word “liar” is displayed. It is displayed on the monitor of the mobile phone 20. Alternatively, the BGM related to the word “liar” is played on the mobile phone 20.

  FIG. 2 is a block diagram of the mobile phone 10 and the mobile phone 20. First, the configuration and function of the mobile phone 10 will be described. The voice input unit 11 inputs a caller's voice. The input voice is output to the voice recognition unit 12 and the voice synthesis unit 13.

  The voice recognition unit 12 performs voice recognition processing on the voice input by the voice input unit 11 in real time. A known method may be used as the voice recognition process. In general, the frequency of input speech is analyzed to recognize phonemes. Furthermore, the input speech is analyzed by recognizing words and sentences. Finally, the voice recognition unit 12 converts the input voice into text data.

  Further, the voice recognition unit 12 specifies the tone of the caller's voice from the sound pressure (sound intensity), tempo, frequency, etc. of the input voice. In order to specify the voice tone of the caller, the voice recognition unit 12 includes a table that associates the sound pressure, tempo, frequency, and the like of the voice with the caller's state pattern (voice tone pattern) in advance. Yes. In other words, numerical values such as sound pressure, tempo, frequency, etc. are set for predetermined state patterns such as “excited tone”, “sunk tone”, “screaming voice”, “laughing voice”, etc. . If the input sound value, such as the sound pressure, tempo, and frequency, is close to the set value of any of the registered status patterns (within the set threshold range), the current call It is determined that the person's voice is in one of the registered state patterns.

  The voice synthesizer 13 determines the voice to be synthesized based on the voice recognition result input from the voice recognizer 12 and synthesizes the determined voice with the voice input from the voice input unit 11. The voice material database 101 is used to determine the voice to be synthesized from the voice recognition result.

  FIG. 3 is a diagram showing a registration example of the audio material database 101. The voice material database 101 is a database that associates the voice recognition result with the voice data in the voice recognition unit 12 and stores the corresponding voice data. In the speech recognition result field of the speech material database 101, keywords or caller status patterns are registered. Actually, the database for associating keywords with audio data and the database for associating state patterns with audio data are managed by different databases, but they are managed by one database for the sake of simplicity. Explain that it is. In FIG. 3, when the registered keyword is a keyword, only the keyword character is displayed. When the registered is a status pattern, (status pattern) is displayed together with the status pattern name. Yes.

  The audio material database 101 is stored in the memory card 30 (shown in FIG. 1) and provided by, for example, a content provider. The user can use the audio material database 101 by purchasing the memory card 30 storing the audio material database 101 and inserting it into the memory card slot of the mobile phone 10. However, the audio material database 101 may be downloaded from the network and stored in a built-in memory or a memory card of the mobile phone 10.

  The speech synthesizer 13 searches the speech text data input from the speech recognizer 12 for a keyword registered in the speech material database 101. If the keyword is hit, the speech synthesizer 13 responds to the keyword. The voice data to be determined is determined as synthesized voice. Alternatively, the voice synthesizer 13 determines voice data corresponding to the caller's state pattern input from the voice recognizer 12 as synthesized voice.

  In the case of the example in the figure, for example, if the voice uttered by the caller includes keywords such as “worst”, “blue”, “depressed”, “depressed”, a dark atmosphere (Doyo ~ Sound effect data is selected. On the other hand, if the speech produced by the caller includes keywords such as “happy”, “happy”, and “fun”, sound effect data with a bright and brilliant atmosphere (a sparkling image) is selected. When the state pattern input from the voice recognition unit 12 is “screams”, explosive sound effect data is selected.

  When the voice synthesis unit 13 acquires voice data from the voice material database 101 based on the voice recognition result, the voice synthesis unit 13 synthesizes the voice acquired from the voice material database 101 with the voice input from the voice input unit 11 and outputs the synthesized voice to the communication unit 14. To do.

  The voice recognition unit 12 also outputs the voice recognition result to the video determination unit 15. The video determination unit 15 determines a video to be transmitted to the mobile phone 20 based on the voice recognition result input from the voice recognition unit 12. The video material database 102 is used to determine a video to be selected from the voice recognition result.

  FIG. 4 is a diagram illustrating a registration example of the video material database 102. The video material database 102 is a database that associates the voice recognition result in the voice recognition unit 12 with the video data and stores the associated video data. In the voice recognition result field of the video material database 102, a keyword or a caller's state pattern is registered. The video determination unit 15 searches the text data of the voice input from the voice recognition unit 12 for a keyword registered in the video material database 102. If the keyword is hit, the video corresponding to the keyword is searched. Get the data. Alternatively, the video determination unit 15 acquires video data corresponding to the caller's state pattern input from the voice recognition unit 12.

  In the example of the figure, for example, if the voice uttered by the caller contains keywords such as “lie”, “lie sober”, and “lie”, it expresses the appearance of the liar Selected video data is selected. If the voice uttered by the caller includes keywords such as “don't forgive”, “can't forgive”, or “unforgivable”, it expresses a harsh man angry Video data is selected. When the state pattern input from the voice recognition unit 12 is “screaming”, a thunder image representing a state of raising a yelling voice is selected. In the example shown in the figure, moving image data is registered as video data, but still image data may be registered as video data.

  The video material database 102 is also stored in the memory card 30 and provided. However, the video material database 102 may be downloaded from the network and stored in a built-in memory or a memory card of the mobile phone 10.

  When the video determination unit 15 acquires video data from the video material database 102 based on the voice recognition result, the video determination unit 15 outputs the acquired video data to the communication unit 14.

  Further, the voice recognition unit 12 outputs the voice recognition result to the vibration determination unit 16. The vibration determination unit 16 determines the vibration setting data to be transmitted to the mobile phone 20 based on the voice recognition result input from the voice recognition unit 12. In order to acquire the vibration setting data from the voice recognition result, the vibration setting database 103 is used.

  FIG. 5 is a diagram illustrating an example of registration in the vibration setting database 103. The vibration setting database 103 is a database in which a voice recognition result in the voice recognition unit 12 is associated with vibration setting data. In the voice recognition result field of the vibration setting database 103, a keyword or a caller's state pattern is registered. The vibration determination unit 16 searches the voice text data input from the voice recognition unit 12 for a keyword registered in the vibration setting database 103, and if a hit is found, the vibration corresponding to the keyword is searched. Get configuration data. Alternatively, the vibration determination unit 16 acquires vibration setting data corresponding to the caller's state pattern input from the voice recognition unit 12.

  In the example shown in the figure, for example, if the voice uttered by the caller includes keywords such as “liar”, “lie sober”, and “lie”, a powerful vibration can be performed only once. Vibration setting data to be generated is selected. Further, when the state pattern input from the voice recognition unit 12 is “screaming”, vibration setting data for generating a strong vibration five times in succession is selected. In the vibration setting data, the number of vibrations, the vibration time, the vibration interval, the strength of vibration, and the like may be set individually or in combination.

  The vibration setting database 103 is also stored in the memory card 30 and provided. However, the vibration setting database 103 may be downloaded from the network and stored in the built-in memory or the memory card of the mobile phone 10.

  When the vibration determination unit 16 acquires the vibration setting data from the vibration setting database 103 based on the voice recognition result, the vibration determination unit 16 outputs the acquired vibration setting data to the communication unit 14.

  The communication unit 14 includes a functional unit that transmits / receives audio to / from another mobile phone and a functional unit that transmits / receives data via a network. As described above, the communication unit 14 inputs the voice of the caller synthesized with the sound effects and BGM from the voice synthesis unit 13. In addition, the communication unit 14 inputs video data determined based on the voice recognition result from the video determination unit 15. Further, the communication unit 14 inputs the vibration setting data determined based on the voice recognition result from the vibration determination unit 16. Then, the communication unit 14 transmits these synthesized speech and data to the mobile phone 20.

  Here, the method by which the communication unit 14 transmits the synthesized voice is the same as the method for transmitting normal call voice. That is, since the sound acquired from the caller and the sound effect and BGM are synthesized in the voice processing unit of the mobile phone 10, the communication unit 14 can process and send the synthesized voice in the same way as normal voice. It is. As a result, the mobile phone that receives this synthesized voice does not need a special function. The mobile phone 20 on the receiving side only has to receive the synthesized voice and output the synthesized voice from the speaker in the same manner as normal call voice.

  Video data and vibration setting data are transmitted using a data transmission / reception protocol such as TCP / IP separately from voice communication. The communication unit 14 has a data communication function that enables such communication. Alternatively, the communication unit 14 may superimpose video data and vibration setting data on a specific frequency band for voice communication and transmit the data.

  Note that the data format of the vibration setting data transmitted from the mobile phone 10 may be composed of, for example, a vibrator command and an argument indicating the number of vibrations, vibration time, vibration interval, vibration intensity, and the like.

  Next, the configuration and function of the mobile phone 20 will be described. The communication unit 21 receives the synthesized audio and video data and vibration setting data transmitted from the communication unit 14. The communication unit 21 has a function of transmitting / receiving audio signals and a data communication function. As described above, when video data and vibration setting data are transmitted using a data communication function separately from an audio signal, the communication unit 21 uses the data communication function to transmit video data and vibration setting data. Receive. When the video data and the vibration setting data are superimposed on the audio signal, the communication unit 21 acquires the video data and the vibration setting data separately from the audio signal.

  The communication unit 21 outputs the received audio signal to the audio reproduction unit 22. The voice reproduction unit 22 outputs synthesized voice from a speaker included in the mobile phone 20. In this way, when a caller utters a word on the mobile phone 10, a sound effect or BGM is synthesized with the word, and the synthesized sound is reproduced on the mobile phone 20.

  Therefore, the caller of the mobile phone 20 can not only simply listen to the words uttered by the caller of the mobile phone 10, but can also communicate with a sense of presence by adding sound effects and BGM to the words. It is. For example, when a caller of the mobile phone 10 utters the word “happy”, a sound effect of a sparkling image is reproduced on the mobile phone 20 together with the word, so that the caller of the mobile phone 20 expresses the emotion of the other party. I can feel enough.

  In addition, as described above, this synthesized voice is synthesized in the transmitting-side mobile phone 10 and transmitted in the same manner as normal voice. Therefore, the receiving-side mobile phone 20 is synthesized using an ordinary mobile phone. Audio can be played back. Therefore, in order to use such a synthesized voice reproduction function, it is not necessary for both the transmitting and receiving terminals to support the function. If a mobile phone equipped with the voice synthesis function of the present invention is prepared, it is rich in expression for any voice playback device (not limited to a mobile phone, but may be a fixed phone or a video phone). Simple synthesized speech can be transmitted.

  The communication unit 21 also outputs the received video data to the video playback unit 23. The video playback unit 23 plays back the input video data on the monitor of the mobile phone 20. The video reproduction unit 23 reproduces the video data on the monitor in synchronization with the received synthesized sound being reproduced from the audio reproduction unit 22. When the video data is moving image data, the video reproducing unit 23 reproduces the moving image on the monitor, and when the video data is still image data, the still image is displayed on the monitor. As a result, the mobile phone 20 reproduces the received synthesized voice and the video determined from the voice recognition result.

  For example, in the case of using the database shown in FIGS. 3 and 4, for example, when a word “don't allow” is issued on the mobile phone 10, the mobile phone 20 will say “blow” The sound effect of the sound of a finger is synthesized and reproduced, and further, a video of a strict man's finger is reproduced on the monitor of the mobile phone 20.

  In addition, the communication unit 21 outputs the received vibration setting data to the vibrator 24. The vibrator 24 generates vibration based on the input vibration setting data. That is, the vibrator 24 can generate vibrations according to parameters such as the number of vibrations, vibration time, vibration interval, and vibration intensity specified in the vibration setting data. As a result, the mobile phone 20 reproduces the received synthesized speech and generates vibration determined from the speech recognition result.

  For example, in the case of using the database shown in FIGS. 3, 4, and 5, for example, when a word “liar” is issued on the mobile phone 10, the sound “liar” is played on the mobile phone 20. At the same time, the video that plays the person who lies is played back, and a strong vibration occurs once in a single shot.

  As described above, by using the mobile phone communication system of the present embodiment, the operator of the mobile phone 10 can convey his / her feelings in an expressive manner. Images and vibrations enhance the expressive power of emotions and feelings that are not communicated by words alone. In addition, even if the caller of the mobile phone 10 is not good at expressing emotions, the mobile phone 10 enhances the expressive power if only speaking the words that he wants to convey. On the other hand, the operator of the mobile phone 20 can receive the feelings of the other party in an emotional manner. In addition, it is possible not only to give diversity to simple communication but also to incorporate elements of play and to communicate with high variety.

{Second Embodiment}
Next, a second embodiment of the present invention will be described. As described above, since the mobile phone 10 transmits the synthesized voice in the same manner as the normal voice, it has been described that the receiving side only needs to have a normal voice reproduction function. On the other hand, for the video data and the vibration setting data, a function corresponding to the terminal on the receiving side is required. The cellular phone 20 includes a video playback unit 23 and a vibrator 24. When these processing units receive data together with an audio signal, the mobile phone 20 plays back video and drives the vibrator in synchronization with playback of the synthesized voice.

  In the second embodiment, the mobile phone 10 selects data to be transmitted according to the terminal type of the destination mobile phone 20. As shown in FIG. 6, the mobile phone 20 includes a terminal information storage unit 25. The terminal information storage unit 25 stores information such as the manufacturer name, model name, and manufacturing number name of the mobile phone 20.

  On the other hand, the mobile phone 10 includes a terminal information acquisition unit 17 as shown in FIG. The terminal information acquisition unit 17 acquires information on the terminal model from the mobile phone 20 before transmitting synthesized speech and other data. Also in the second embodiment, synthesized sound is generated and video data and vibration setting data are determined by the same method as in the first embodiment. In the first embodiment, when the synthesized voice is generated and the video data and the vibration setting data are determined, the synthesized voice and the data are transmitted as they are. In the second embodiment, first, the terminal information acquisition unit 17 acquires the terminal model information of the mobile phone 20 and determines the processing method.

  Specifically, when the terminal information acquisition unit 17 generates an information acquisition command for the terminal model at the time when the voice call is started (when the terminals are connected to each other), the command is sent from the communication unit 14 to the mobile phone 20. Sent. In the mobile phone 20, the terminal model information of the mobile phone 20 is read from the terminal information storage unit 25, and the read information is transmitted to the mobile phone 10. In this way, the terminal information acquisition unit 17 obtains the terminal model information of the mobile phone 20.

  The terminal information acquisition unit 17 outputs the terminal model information to the video determination unit 15 and the vibration determination unit 16. The video determination unit 15 determines from the terminal model information whether the destination mobile phone has a video playback function during a voice call. If it is determined that the destination mobile phone has a video playback function during a voice call, the video data acquired from the video material database 102 is output to the communication unit 14 as in the first embodiment. To do. On the other hand, when it is determined that the destination mobile phone does not have the function, the output of the video data to the communication unit 14 is stopped.

  The vibration determining unit 16 determines from the terminal model information whether the destination mobile phone has a vibration function during a voice call. If it is determined that the destination mobile phone has a vibration function for voice calls, the vibration setting data determined with reference to the vibration setting database 103 is transmitted to the communication unit, as in the first embodiment. 14 is output. On the other hand, when it is determined that the destination mobile phone does not have the function, the output of the vibration setting data to the communication unit 14 is stopped.

  Thereby, the communication unit 14 transmits only the synthesized voice and the video data when the destination mobile phone has the video playback function at the time of the voice call but does not have the vibration function. If the destination mobile phone has a vibration function for voice calls, but does not have a video playback function, only the synthesized voice and vibration setting data are transmitted. When the destination mobile phone has both a video playback function and a vibration function during a voice call, video data and vibration setting data are transmitted in addition to the synthesized voice.

  Thus, since the data to be transmitted is determined according to the type of the mobile phone of the transmission destination, useless data not supported by the mobile phone of the transmission destination is not transmitted. Thereby, it is possible to reduce the cost of data transmission, and it is possible to prevent the communication band from being compressed with useless data.

{Third embodiment}
Next, a third embodiment of the present invention will be described. In the first embodiment, the mobile phone 20 reproduces the video data transmitted from the mobile phone 10 as it is. Alternatively, the vibrator 24 is driven based on the vibration setting data transmitted from the mobile phone 10. On the other hand, in the third embodiment, the mobile phone 20 also determines video data and vibration setting data.

  In the third embodiment, as shown in FIG. 7, the voice recognition unit 12 outputs a voice recognition result to the communication unit 14. The communication unit 14 transmits the voice recognition result to the mobile phone 20 together with the synthesized voice, the video data, and the vibration setting data. The communication unit 14 transmits the voice recognition result by the same method as the video data or the like. That is, the speech recognition result is transmitted using the data communication function or superposed on the speech signal.

  In the mobile phone 20, the communication unit 21 separates and acquires the voice recognition result together with the video data and the vibration setting data from the voice signal. The communication unit 21 outputs the voice recognition result together with the received video data to the video reproduction unit 23.

  The mobile phone 20 includes a video material database 201 as shown in FIG. The database structure of the video material database 201 is the same as the video material database 102 provided in the mobile phone 10. That is, the voice recognition result and the video data are associated with each other and the associated video data is stored.

  However, the correlation between the voice recognition result and the video data in the video material database 201 may be different from that in the video material database 102. For example, different video data may be associated with the same keyword or the same state pattern. For example, if the video material database is acquired from the same content provider, the content is the same, but if the video material database is acquired from a different content provider, the content is different. Alternatively, a plurality of different video material databases may be provided from one content provider. The video material database 201 may also be stored in a memory card and provided to the user, or may be downloaded by the user via a network.

  When the video reproduction unit 23 receives the video data and the voice recognition result from the communication unit 21, the video reproduction unit 23 may reproduce the input video data, that is, the video data transmitted from the mobile phone 10 as it is on the monitor. Alternatively, the video reproduction unit 23 may determine the corresponding video data by referring to the video material database 201 based on the input voice recognition result, and cause the monitor to reproduce the determined video data. Which video is to be reproduced may be set in advance in the mobile phone 20. If you want to enjoy the video sent from the other party's terminal, you can set it to play the received video data, or if you want to use the video material database 201 used by your mobile phone 20 It may be set to use the received voice recognition result.

  Further, the communication unit 21 outputs the voice recognition result together with the vibration setting data received to the vibrator 24. The mobile phone 20 includes a vibration setting database 202 as shown in FIG. The database structure of the vibration setting database 202 is the same as the vibration setting database 103 provided in the mobile phone 10. That is, the vibration setting data is associated with the voice recognition result.

  Similarly, the vibration setting database 103 and the vibration setting database 202 may be different databases or the same database. The vibration setting database 202 may also be stored in a memory card and provided to the user, or may be downloaded by the user via a network.

  When vibrator 24 receives vibration setting data and a voice recognition result from communication unit 21, vibrator 24 may generate vibration based on the input vibration setting data, that is, vibration setting data transmitted from mobile phone 10. Alternatively, the vibrator 24 may determine the vibration setting data with reference to the vibration setting database 202 based on the input voice recognition result, and may generate vibration based on the determined data.

  As described above, in the third embodiment, the video reproduced together with the synthesized audio is obtained using the transmission-side mobile phone 10 or is acquired using the reception-side mobile phone 20. It is possible to select whether to use. Therefore, if the receiving side has a richer database, it is possible to use the receiving side database and perform communication with richer expressiveness. In addition, by using the database on the receiving side, it is possible to enjoy that the unintended video on the transmitting side is reproduced on the receiving side.

  In the third embodiment, the recognition result of the voice recognition unit 12 is directly transmitted to the mobile phone 20 via the communication unit 14. In other words, data obtained by converting the caller's voice into text data or data designating the state pattern specified by the voice recognition unit 12 is transmitted to the mobile phone 20.

  As another method, only the keyword adopted on the mobile phone 10 side may be transmitted. That is, when the speech synthesis unit 13, the video determination unit 15, and the vibration determination unit 16 input text data as a speech recognition result from the speech recognition unit 12, the keyword search is performed with reference to the databases 101 to 103, respectively. When a keyword is searched, audio data, video data, and vibration setting data are acquired from the keyword. Therefore, when a keyword is searched, this keyword is output to the communication unit 14. And the communication part 14 does not transmit all the text data which converted the audio | voice, but transmits only a keyword to the mobile telephone 20. FIG. As a result, the data transmission amount can be reduced.

  Note that the functions of the second embodiment may be incorporated into the third embodiment. In other words, the terminal model information of the destination mobile phone is acquired at the start of the voice call, and the video data and the voice recognition result are transmitted only when the destination mobile phone has a video playback function at the time of the voice call. It may be. Moreover, the vibration setting data and the voice recognition result may be transmitted only when the transmission destination mobile phone has a vibration function during a voice call.

{Fourth embodiment}
In the first to third embodiments, the voice recognition process is executed in real time on the voice input in the mobile phone 10. Then, the audio material data, the video material data, and the vibration setting data are obtained by referring to the databases 101 to 103 based on the voice recognition result. That is, the databases 101 to 103 associate speech recognition results with material data and vibration setting data.

  In the fourth embodiment, the sound material data registered in the sound material database is associated with the key operation of the mobile phone 10. Similarly, the video material data registered in the video material database and the vibration setting data registered in the vibration setting database are associated with key operations of the mobile phone 10.

  Therefore, the mobile phone 10 does not perform voice recognition processing, and performs voice synthesis and transmission of video or vibration setting data based on the key operation of the operator. For example, when the operator presses the “1” key during a voice call, the corresponding BGM is selected according to the key, and the selected BGM is synthesized with the call voice and transmitted to the mobile phone 20. Alternatively, when the operator depresses the “2” key during a voice call, the corresponding video data and vibration setting data are selected according to the key, and the selected video data and vibration setting data are displayed along with the call voice. It is transmitted to the mobile phone 20.

  Thus, BGM and sound effects can be synthesized with the call voice at any timing by performing key operations assigned at any timing during the voice call. In addition, video data and vibration setting data can be transmitted to the partner terminal at an arbitrary timing. In the above example, one key operation is associated with audio material data or the like, but may be associated with a plurality of key operations. For example, a method may be used in which voice data is designated by selecting a key from “1” to “9” in response to an instruction for synthesizing voice with the key “1”.

  Also by this method, it is possible to add a special effect that enhances the expressive power to the call voice. In addition, since real-time voice recognition processing is not performed, the processing load on the mobile phone 10 can be reduced.

{Modifications}
In each of the above embodiments, the video data acquired from the voice recognition result is transmitted together with the synthesized voice. As another embodiment, voice data may not be synthesized, and video data acquired from a voice recognition result may be transmitted together with voice uttered by a caller.

  In each of the above embodiments, the vibration setting data acquired from the speech recognition result is transmitted together with the synthesized speech. As another example, the voice setting data acquired from the voice recognition result may be transmitted together with the voice uttered by the caller without performing voice synthesis.

  As described above, the embodiments of the present invention have been described by way of example in which the transmitting and receiving terminals are mobile phones, but one or both of them may be terminals other than the mobile phone. For example, it may be a landline telephone having a video playback function and a vibration function. Alternatively, it may be a messenger executed on a personal computer.

  In addition, it is desirable that the terminal on the transmission side has a function for turning on / off the function in each of the above embodiments. That is, it is only necessary to individually enable / disable the function regarding whether to synthesize audio, to transmit video data, or to transmit vibration setting data. For example, each function can be turned off for an important work call.

  Further, in each of the above embodiments, it has been described that each database can be obtained by being stored in a memory card or downloaded. However, by providing a simple authoring tool in the terminal, these databases can be stored by the user himself / herself. May be created.

It is an image figure of the communication system using a mobile phone. It is a block diagram of a mobile phone on a transmission side and a reception side. It is a figure which shows the example of registration of an audio material database. It is a figure which shows the example of registration of a video material database. It is a figure which shows the example of registration of a vibration setting database. It is a block diagram of a mobile phone on the transmission side and the reception side in the second embodiment. It is a block diagram of a mobile phone on the transmitting side and the receiving side in the third embodiment.

Explanation of symbols

10 (Transmission side) Mobile phone 20 (Reception side) Mobile phone 101 Audio material database 102 Video material database 103 Vibration setting database 201 (Reception side) Video material database 202 (Reception side) Vibration setting database

Claims (19)

An audio material database for storing audio material data associated with the speech recognition results;
Voice input means;
Voice recognition means for performing voice recognition on the voice input from the voice input means;
Means for acquiring the voice material data associated with the recognition result by the voice recognition means from the voice material database, and synthesizing the acquired voice material data with the voice input from the voice input means;
A communication means for transmitting the synthesized voice data to the other terminal;
A voice transmitting terminal comprising: The voice transmission terminal according to claim 1, further comprising:
A video material database that stores video material data associated with the speech recognition results;
Means for acquiring video material data associated with a recognition result by the voice recognition means from the video material database;
With
The communication means transmits the video acquired from the video material database together with the audio synthesized with the audio material data to the partner terminal. The voice transmission terminal according to claim 1, further comprising:
A vibration setting database in which voice recognition results are associated with vibration setting data;
Means for determining vibration setting data based on a recognition result by the voice recognition means;
With
The communication means transmits vibration setting data to the partner terminal together with the voice synthesized with the voice material data. The voice transmission terminal according to claim 2 or 3, further comprising:
Means for obtaining terminal information of the partner terminal;
With
An audio transmitting terminal, wherein transmission of video material data or vibration setting data is stopped according to a type of the partner terminal. The voice transmission terminal according to any one of claims 2 to 4, further comprising:
The voice transmission terminal characterized in that the communication means transmits a recognition result by the voice recognition means to the partner terminal. A video material database that stores video material data associated with the speech recognition results;
Voice input means;
Voice recognition means for performing voice recognition on the voice input from the voice input means;
Means for acquiring video material data associated with a recognition result by the voice recognition means from the video material database;
Communication means for transmitting the video acquired from the video material database together with the voice input from the voice input means to a partner terminal;
A voice transmitting terminal comprising: A vibration setting database in which voice recognition results are associated with vibration setting data;
Voice input means;
Voice recognition means for performing voice recognition on the voice input from the voice input means;
Means for determining vibration setting data based on a recognition result by the voice recognition means;
A communication means for transmitting vibration setting data to the partner terminal together with the voice input from the voice input means;
A voice transmitting terminal comprising: The voice transmission terminal according to any one of claims 1 to 7,
The speech recognition result includes the result of text-converting the speech input by the speech input means and / or information related to the tone of the speech determined from the speech input by the speech input means. . The voice transmission terminal according to claim 1,
The audio material database is stored in a memory card, and the audio material database can be used by inserting the memory card into the audio transmission terminal. In the voice transmitting terminal according to claim 2 or 6,
An audio transmitting terminal, wherein the video material database is stored in a memory card, and the video material database can be used by inserting the memory card into the audio transmitting terminal. In the voice transmitting terminal according to claim 3 or 7,
The vibration setting database is stored in a memory card, and the vibration setting database can be used by inserting the memory card into the voice transmitting terminal. A terminal for receiving voice transmitted from the voice transmitting terminal according to claim 1,
An audio reproduction terminal characterized by outputting audio synthesized with audio material data from a speaker. A terminal for receiving voice and data transmitted from the voice transmitting terminal according to claim 2,
An audio reproduction terminal characterized by outputting received video material data to a monitor while outputting audio synthesized with audio material data from a speaker. A terminal for receiving voice and data transmitted from the voice transmitting terminal according to claim 3,
An audio reproduction terminal characterized in that a vibrator is driven based on received vibration setting data while outputting a synthesized voice of audio material data from a speaker. A terminal that receives the synthesized voice and the voice recognition result transmitted from the voice transmitting terminal according to claim 5,
The receiving terminal is
Receiving-side video material database that stores video material data associated with voice recognition results,
With
The receiving terminal is
When the video material data is received from the audio transmission terminal, the video material data received together with the synthesized audio is reproduced, or the corresponding video material data is acquired from the receiving video material database based on the received recognition result, and synthesized. An audio reproduction terminal characterized in that it can select whether to reproduce video material data acquired together with audio. A terminal that receives the synthesized voice and the voice recognition result transmitted from the voice transmitting terminal according to claim 5,
The receiving terminal is
Receiving side vibration setting database that associates voice recognition results with vibration setting data,
With
The receiving terminal is
When vibration setting data is received from the voice transmitting terminal, the synthesized voice is reproduced and the vibrator is driven based on the received vibration setting data, or the reception side vibration setting database is referred to based on the received voice recognition result. A voice reproduction terminal characterized in that vibration setting data is determined to reproduce a synthesized voice and to select whether to drive a vibrator based on the determined vibration setting data. An audio material database that stores audio material data associated with key operations;
Voice input means;
Means for acquiring voice material data associated from a key operation input during a voice call from the voice material database, and synthesizing the acquired voice material data with the voice input from the voice input means;
A communication means for transmitting the synthesized voice data to the other terminal;
A voice transmitting terminal comprising: A video material database that stores video material data associated with key operations;
Voice input means;
Means for acquiring video material data associated with key operations input during a voice call from the video material database;
A communication means for transmitting the video material data acquired together with the sound input from the sound input means to a partner terminal;
A voice transmitting terminal comprising: A vibration setting database for storing vibration setting data associated with key operations;
Voice input means;
Means for acquiring vibration setting data associated with key operations input during a voice call from the vibration setting database;
Communication means for transmitting the vibration setting data acquired together with the voice input from the voice input means to the partner terminal;
A voice transmitting terminal comprising:

Images