JP2005208367A - Audio reproducing processing apparatus and telephone terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an audio reproducing processing apparatus capable of changing the display styles of an image according to the output of audio data without, increasing the amount of data. <P>SOLUTION: The audio reproducing processing apparatus is equipped with a mouth-shape generating means 21 of reading in face image data on a caller, corresponding to an origination-source telephone number from among of a telephone directory database 23, when sound-recording data are reproduced and analyzing the sound-recording data to generate a mouth-shape image, corresponding to the analysis result and an expression changing image generating means 26 of putting the mouth-shape image and a face image together, to generate an expression changing image in which the expression of the face changes, and an expression changing image which is generated each time an analysis is conducted is displayed by a display means 14. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an audio reproduction processing apparatus that displays an image in conjunction with output of audio data and a telephone terminal equipped with the same.

  Conventionally, various answering machine service systems have been proposed in order to convey a message message recorded in an absence to a receiver. For example, a message message is registered in a message center of a base station using a telephone line, and a receiver makes an inquiry to the message center to receive the message message. According to this, the recipient can indirectly communicate with the caller.

  In this answering machine service system, only voice information is used as a message message medium, so the message message becomes inorganic, and the actual expression intention may not be accurately transmitted to the receiver. On the other hand, the message message of the caller is stored in the database, and the message message can be played back according to the request of the receiver, and the emotion of the caller is expressed by an agent (proxy) image when the message message is played back. And the message system which changes the agent image continuously and displays it like a moving image is proposed (for example, refer to patent documents 1).

JP 2002-41279 A

  However, in such a conventional message system, it is necessary to prepare a large number of agent image frame data in order to display a moving image when a message message is reproduced. Furthermore, in order to reproduce different moving images over a long period of time, it is necessary to accumulate and process a huge amount of image data. For this reason, a large-capacity memory and a high-speed arithmetic processing circuit are required, which is not suitable for mounting on a portable terminal.

  The present invention has been made in view of the above circumstances, and is equipped with an audio reproduction processing apparatus capable of changing the display form of an image in accordance with the output of audio data without increasing the amount of data, and the same. The purpose is to provide a telephone terminal.

  An audio reproduction processing apparatus according to the present invention is an audio reproduction processing apparatus that displays an image in conjunction with output of audio data, and includes analysis means for analyzing the audio data, and an analysis result every time the audio data is analyzed. And an image generation unit that generates a new image in which the display form of the original image is changed, and a display control unit that displays the new image every time the new image is generated.

  According to the configuration described above, the image generation unit that generates a new image in which the display form of the original image is changed according to the analysis result every time the sound data is analyzed is provided each time the sound data is analyzed. Since images corresponding to analysis results are sequentially generated and output from a single image, there is no need to prepare multiple images in advance, and as a result, images can be displayed in accordance with the output of audio data without increasing the amount of data. The form can be changed.

  In the audio reproduction processing apparatus of the present invention, the analysis unit analyzes the audio data at unit time intervals. According to the above configuration, the number of images to be generated can be increased by adjusting the time interval, and the continuity of images can be improved. Further, the processing load can be reduced by reducing the number of images to be generated.

  In the audio reproduction processing apparatus of the present invention, the image generation unit generates a new face image in which the facial expression is changed. According to the above configuration, the facial expression of the face image is changed in conjunction with the output of the sound, so that the sound can be output as if the sound is emitted from the face image.

  The speech reproduction processing apparatus of the present invention further includes mouth shape creation means for changing the facial expression of the face image based on the mouth shape created based on the analysis result of the speech data. According to the above configuration, since the display form of the image can be changed simply by changing the mouth shape of the original image to the newly created mouth shape, a new face image can be easily generated with a small amount of data. .

  The speech reproduction processing apparatus of the present invention further includes emotion estimation means for changing the expression of the face image based on the emotion estimated based on the analysis result of the audio data. In the audio reproduction processing apparatus of the present invention, the emotion estimation means estimates the speaker's emotion based on at least one selected from the loudness, height and speaking speed of the audio data. . According to the above configuration, since the speaker's emotion can be expressed, a more familiar face image can be generated.

  The audio reproduction processing apparatus according to the present invention further includes selection means for selecting an arbitrary original image from a plurality of original images. According to the above configuration, by selecting the original image that matches the audio data, it is possible to display the image linked with the output of the audio data without a sense of incongruity.

  The telephone terminal of the present invention is a telephone terminal that is equipped with the audio reproduction processing apparatus of the present invention and displays an image in conjunction with the output of recorded audio data, and associates the original image with a telephone number. And storage means for storing the recorded voice in association with the telephone number of the caller, and the selection means is configured to store the caller telephone number from a plurality of stored original images. Each time the image generation means analyzes the recorded voice data, the display form of the original image selected corresponding to the caller telephone number is changed according to the analysis result. A new image is generated.

  According to the above configuration, by storing the original image in association with the telephone number and storing the face image of the message recorder associated with the caller telephone number, You can play a message as if you were talking.

  The telephone terminal of the present invention is a telephone terminal that is equipped with the voice reproduction processing apparatus of the present invention and displays an image in conjunction with the output of a call ringtone, and stores the original image in association with a telephone number. Storing means for selecting, and the selecting means selects an image corresponding to the telephone number of the caller from a plurality of stored original images, and the image generating means outputs an analysis result each time the ringtone is analyzed. In response, a new image is generated by changing the display form of the original image selected corresponding to the telephone number of the caller.

  The telephone terminal of the present invention is a telephone terminal that is equipped with the voice reproduction processing apparatus of the present invention and displays an image in conjunction with the output of a mail ringtone, and stores the original image in association with a mail address. Storing means for selecting the image corresponding to the mail address from the plurality of stored original images, the image generating means according to the analysis result every time the ring tone is analyzed, A new image is generated by changing the display form of the original image selected corresponding to the e-mail address.

  According to the above configuration, by providing the storage unit that stores the original image in association with the telephone number, it is as if the voice is emitted from the face image associated with the caller telephone number or mail address during the incoming call. Can output a ringtone.

  The telephone terminal of the present invention is a telephone terminal that is equipped with the voice reproduction processing apparatus of the present invention and displays an image in conjunction with the output of the call voice, and stores the original image in association with the telephone number. Storage means, wherein the selection means selects an image corresponding to the telephone number of the caller from a plurality of stored original images, and the image generation means responds to the analysis result each time the incoming call voice is analyzed. Then, a new image in which the display form of the original image selected corresponding to the caller telephone number is changed is generated.

  According to the above configuration, the facial expression of the face image can be changed as if the other party is talking in accordance with the voice of the other party, so that it can be used as a videophone with a small memory usage.

  The audio reproduction processing method of the present invention is an audio reproduction processing method for displaying an image in conjunction with output of audio data, and analyzes the audio data and responds to the analysis result each time the audio data is analyzed. Thus, a new image in which the display form of the original image is changed is generated, and the new image is displayed every time the new image is generated.

  Furthermore, the audio reproduction processing program of the present invention is an audio reproduction processing program for displaying an image in conjunction with output of audio data, and the computer analyzes the audio data each time the audio data is analyzed. For generating a new image in which the display form of the original image is changed according to the analysis result, and a display control unit for displaying the new image every time the new image is generated. This is a sound reproduction processing program.

  According to the present invention, the image generation unit that generates a new image in which the display form of the original image is changed according to the analysis result every time the audio data is analyzed is provided. Since the images corresponding to the analysis results are sequentially generated and output from the images, it is not necessary to prepare a plurality of images in advance, and as a result, the image data can be matched to the output of the audio data without increasing the data amount. The display form can be changed.

  FIG. 1 is a configuration diagram of an answering machine reproduction system for explaining an embodiment of the present invention. The terminal 11 shown in the figure is a telephone terminal equipped with a voice reproduction processing device that displays an image in conjunction with the output of voice data, and outputs the recorded message while the facial expression of the message recorder's face image. Is displayed. The terminal 11 includes a microphone 12 for inputting sound, a speaker 13 for outputting sound, a monitor (LCD) 14 for outputting an image, a pointing device such as a mouse, and manual input such as a keyboard. An operation unit 15 is provided externally. The input operation unit 15 enables execution of various functions of the terminal 11 and setting input.

  The terminal 11 includes a control unit 16. The control unit 16 includes a CPU and the like, and performs execution control of various means related to recording and reproduction of message data, which will be described later, and image processing of the caller, access control to a database, and the like. Moreover, whenever the control part 16 produces | generates the new image (expression change image) which changed the display form of the original image (face image), this new image is displayed on the display means 14. FIG.

  The terminal 11 also includes a recording unit 17 for recording a message message (voice data), a recording database 18 for storing the recorded voice in association with the telephone number of the caller, and input of voice data into the recording database 18. Recording data management means 19 for managing the output, reproduction means 20 for reproducing the audio data, analyzing the audio data, and changing the expression of the face image according to the mouth shape created based on the analysis result of the audio data Mouth shape creation means 21, emotion estimation means 22 for analyzing voice data and changing the facial expression based on the emotion estimated based on the analysis result of the voice data, and the original image associated with the telephone number A telephone directory database 23 to be stored, a telephone directory data management means 24 for managing input / output of data to / from the telephone directory database 23, and a caller as a telephone number A user identifying unit 25 for identifying based, and an expression modified image creating means 26 for generating a new image obtained by changing the display mode of the original image according to the analysis result each time analyzing the audio data. The user specifying unit 25 and the phone book data management unit 24 select an image corresponding to the caller telephone number from the plurality of face images stored in the phone book database 23.

  The telephone directory database 24 stores the telephone number of the calling party terminal in association with the still image of the calling party corresponding to this telephone number. The caller's still image is captured in advance from a digital camera (not shown) held by the receiver terminal 11 itself, or provided from the caller. This still image is stored in association with the telephone number of each caller.

  Further, the recording database 18 stores the recording date and time (incoming date and time) in association with the recording data of the caller and the caller telephone number at the time of the absence recording.

  The mouth shape creating means 21 creates a plurality of mouth shapes of the caller's face image corresponding to the caller telephone number based on the recorded data when the recorded data is reproduced. Here, for example, five mouth shapes corresponding to five vowels of Japanese vowels “A”, “I”, “U”, “E”, “O”, which are spoken words of the caller, are created. The mouth shape creating means 21 analyzes the speech data by speech recognition or the like at unit time intervals, and creates a mouth shape image corresponding to the first vowel of the analyzed phrase each time it is analyzed. By sequentially creating mouth shape images at the time of reproducing audio data, it is not necessary to hold a plurality of images, so that an increase in the amount of data to be handled can be suppressed. By changing the mouth shape, it is possible to express on the screen how the caller is talking. Note that five types of mouth shape images may be created in advance at the start of audio data reproduction. In this case, an increase in the amount of data to be handled can be minimized by storing only the difference data of the mouth shape with respect to the original image.

  The emotion estimation means 22 is for estimating the speaker's emotion based on at least one selected from the loudness, pitch and speaking speed when reproducing the voice data. This emotion information can be reflected in the facial expression created by the facial expression change image creating means 26. In the emotion estimation means 22, emotion information of the caller's emotions is parameterized for each loudness, height and speaking speed, and as emotion expression information for changing the display form of the face image It is remembered. The emotion estimation means 22 is, for example, that the voice data is loud and high in tone, the speaker is angry if the voice is fast, the voice is loud or normal, and the speaking speed is slow. Presume that the speaker is in a normal state.

  Each time the voice data is analyzed, the facial expression changed image creating means 26 synthesizes the created mouth shape image and the original image to generate a facial expression modified image in which the facial expression of the original image changes. The facial expression change image creation means 26 adds the emotion information obtained by the emotion estimation means 22 using the mouth shape image created by the mouth shape creation means 21, and adds the mouth shape and eyes of the original image. An expression-changed image is generated in which the shape or the like (for example, the eyes are lowered or the mouth is opened wide) is changed. Each time the facial expression change image is generated, it is displayed on the display means 14.

  FIG. 2 is a diagram for explaining the process from reading of recorded data to changing the facial expression of an image, and FIG. 3 is a diagram showing the flow from reading of recorded data to changing the facial expression of an image. When the input operation unit 15 attached to the terminal 11 is operated, the control unit 16 checks whether there is an absence recording by referring to the recording database 18 through the recording data management means 19. When there is an absence recording, the recording data management means 19 sends the voice data recorded from the recording database 18 at the time of the incoming call, that is, the message data of the caller, and the telephone number of the caller terminal corresponding to this message data (outgoing call) (Original telephone number) is read (step S31).

  Next, the user specifying means 25 checks whether or not the caller telephone number is registered in the telephone book database 23 through the telephone book data management means 24 (step S32). If the telephone number is registered in the telephone book database 23 (step S33), it is then checked whether or not the still image of the caller corresponding to the telephone number is registered in the telephone book database 23 (step S33). S34).

  If a still image is registered in the telephone directory database 23, the control unit 16 causes the user specifying means 25 to read and hold the still image (step S35). In this way, when the process related to the series of recorded data is completed, the process waits until the reproduction process is started.

  When the user of the terminal 11 confirms that there is absence recording data by the display means 14 or the like and operates the input operation unit 15, the recording data reproduction process is started. The control unit 16 checks whether or not the caller's image data is held in the user specifying means 25 (step S37). If face image data is held, the recording data is analyzed for each unit time (step S38), and mouth shape data is created each time (step S39), and the emotion of the speaker is estimated (step S39). Step S40).

  The expression change image creation means 26 reflects the emotion information estimated by the emotion estimation means in addition to the facial expression change due to the mouth shape change, and changes the expression according to the analysis result of the recorded data from one original image. An image is created (step S41). Thereafter, the display means 14 displays the facial expression change image (step S42) and the recorded data is reproduced for a unit time (step S43). Then, the processes after step S37 are repeated until there is no unreproduced recording data (steps S44 and S36). If it is determined in step S37 that the face image has not been read, the process proceeds to step S43, and the recorded data is reproduced.

  In this way, when playing back a message recorded on the receiver's terminal, not only the voice is output but also the caller's face image is displayed and the facial expression is changed to Message playback can be performed as if the caller is speaking. By this image display, it is possible to entertain the receiver with the effect that the caller is near the receiver.

  In addition, the audio | voice reproduction process demonstrated above can also be used for another use. For example, it is possible to change the display shape, pattern, and color (display form) of an image that matches the ring tone and the face image of the caller, etc., together with the output of the ring tone. In addition, the face image of the other party can be changed and displayed during a call. In this case, even a telephone terminal that does not have a videophone function can realize the same function as a videophone. In the case of a telephone terminal that has a videophone function, compared to the case of transmitting and receiving audio and images. This has the effect of greatly reducing the amount of data handled.

  The voice reproduction processing device and the telephone terminal of the present invention have an effect that the display form of an image can be changed in accordance with the output of voice data without increasing the amount of data, and a message recorded by absence is called. This is useful for a voice reproduction processing apparatus that displays an image in conjunction with output of voice data, such as an answering machine system that reproduces the user's own face image, and a telephone terminal equipped with the same.

Configuration diagram of answering machine playback system for explaining an embodiment of the present invention Diagram explaining the process from reading the recorded data to changing the facial expression of the image Diagram showing the flow from reading the recorded data to changing the facial expression of the image

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Terminal 16 Control part 17 Recording means 18 Recording database 19 Recording data management means 20 Playback means 21 Mouth shape creation means 22 Emotion estimation means 23 Telephone book database 24 Telephone book data management means 25 User specification means 26 Expression change image creation means

Claims (13)

An audio reproduction processing device that displays an image in conjunction with output of audio data,
Analyzing means for analyzing the audio data;
Image generating means for generating a new image in which the display form of the original image is changed according to the analysis result every time the audio data is analyzed;
Display control means for displaying the new image every time the new image is generated;
An audio reproduction processing apparatus comprising: The audio reproduction processing device according to claim 1,
The analysis means is an audio reproduction processing device that analyzes the audio data at unit time intervals. The audio reproduction processing device according to claim 1 or 2,
The voice reproduction processing device, wherein the image generation means generates a new face image in which the facial expression is changed. The audio reproduction processing device according to claim 3,
An audio reproduction processing device comprising a mouth shape creating means for changing the facial expression of the face image according to the mouth shape created based on the analysis result of the sound data. The sound reproduction processing device according to claim 3 or 4,
An audio reproduction processing apparatus comprising emotion estimation means for changing an expression of the face image based on an emotion estimated based on an analysis result of the audio data. The sound reproduction processing device according to claim 5,
The speech reproduction processing device, wherein the emotion estimation means estimates a speaker's emotion based on at least one selected from the volume, height and speaking speed of the voice data. The sound reproduction processing device according to any one of claims 1 to 6,
An audio reproduction processing apparatus comprising selection means for selecting an arbitrary original image from a plurality of original images. A telephone terminal that is equipped with the voice reproduction processing device according to claim 7 and displays an image in conjunction with output of recorded voice data,
Storage means for storing the original image in association with a telephone number;
Storage means for storing the recorded voice in association with the telephone number of the caller,
The selection means selects an image corresponding to the caller's telephone number from a plurality of stored original images,
The image generation means generates a new image in which the display form of the original image selected corresponding to the caller telephone number is changed according to the analysis result every time the recorded voice data is analyzed. Phone terminal. A telephone terminal equipped with the voice reproduction processing device according to claim 7 and displaying an image in conjunction with output of a call ringtone,
Storing means for storing the original image in association with a telephone number;
The selection means selects an image corresponding to the caller's telephone number from a plurality of stored original images,
The image generation means is a telephone terminal that generates a new image in which the display form of the original image selected corresponding to the caller telephone number is changed according to the analysis result every time the ring tone is analyzed. A telephone terminal equipped with the voice reproduction processing device according to claim 7 and displaying an image in conjunction with output of a mail ringtone,
Storing means for storing the original image in association with a mail address;
The selecting means selects an image corresponding to the mail address from a plurality of accumulated original images,
The image generation means is a telephone terminal that generates a new image in which the display form of the original image selected corresponding to the mail address is changed according to the analysis result every time the ring tone is analyzed. A telephone terminal that is equipped with the voice reproduction processing device according to claim 7 and that displays an image in conjunction with output of a call voice,
Storing means for storing the original image in association with a telephone number;
The selection means selects an image corresponding to the caller's telephone number from a plurality of stored original images,
The image generation means is a telephone terminal that generates a new image in which the display form of the original image selected corresponding to the caller telephone number is changed according to the analysis result every time the incoming call sound is analyzed. An audio reproduction processing method for displaying an image in conjunction with output of audio data,
Analyzing the audio data;
Each time the voice data is analyzed, a new image is generated by changing the display form of the original image according to the analysis result,
An audio reproduction processing method for displaying a new image every time the new image is generated. An audio reproduction processing program that displays an image in conjunction with output of audio data, the computer
Analyzing means for analyzing the voice data;
Image generating means for generating a new image in which the display form of the original image is changed according to the analysis result every time the audio data is analyzed,
An audio reproduction processing program for functioning as display control means for displaying the new image every time the new image is generated.

Images