JP2005173772A - Image communication system and image formation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image communication system capable of always improving communication ability with a called party during communication and an image formation method. <P>SOLUTION: In this information communication system 10, an expression characteristic extraction part 20 extracts a parameter 38 from a digital image 36 acquired in a transmission terminal device 12 via an image input part 18, an emotion designation part 42 of an emotion change part 22 separately designates deformation to a part giving an expression, a magnifying power designation part 44 specifies a deformation ratio to provide it to the emotion designation part 42 and provides a pair of values 50 of the corresponding part to a shape deformation part 40, and the shape deformation part 40 outputs a parameter 52 replaced by the values 50 provided by the shape deformation part 40 and installs a display model storage part 24 and a display model creation part 26 in either of the transmission terminal device 12 or a reception terminal device 14 according to the output data from the transmission terminal device 12. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image communication system and an image generation method, and the image communication system of the present invention is capable of transmitting image information and audio information via a wired or wireless communication line such as a public line network or an IP (Internet Protocol) network. In particular, the present invention relates to a system for converting a facial expression image of a sender's face into a two-dimensional or three-dimensional CG (Computer Graphics) image or the like different from the actual one and sending it to a communication partner. This image generation method relates to a method of generating a character image having a facial expression on the transmission side or the reception side, for example, based on information sent to a communication partner.

  With the increase in speed and capacity of communication lines, it has been widely used to communicate both voice and images by using telephones, etc., compared to communication that has been performed only by voice. . An example of such communication means is a videophone system. This system has recently been applied to general home telephones and mobile phones due to the small size of the system. The communication including such an image can simultaneously convey information that does not appear in the voice expression due to the facial expression of the speaker, etc., compared to the communication of the conventional telephone in which the transmission medium is limited to the voice. This communication makes the emotional expression rich.

  By the way, in a recent videophone system, particularly a mobile phone, communication including an image is generally performed in an unspecified place or an unspecified time zone. At this time, if communication is performed while ignoring the situation of the communication partner, there arises a problem that the privacy of the speaker is easily violated. For this reason, users who care about makeup, hairstyles, clothes, etc. are expected to appear in the users of this system in preparation for a call. This is likely to be a burden on the user who uses this system. In addition, when receiving such an unexpected call, the user can select a method of making a call using only voice as in the past, without transmitting the image as a non-call state, but communication in this case is not possible. If you do not use the functions that you have, it will cause a decline in communication compared to using it.

  As a method of dealing with such a problem, for example, so-called animation that extracts facial expressions from a caller's face image, reflects the extracted facial expressions on the selected image, and generates a moving image of the selected image. The method is proposed in the review of Non-Patent Document 1. This review summarizes the committee draft proposed in 1998 by the Working Group (WG) 11 / MPEG of the Moving Picture Experts Group (MPEG). This review describes a method of analyzing facial features by image processing, encoding facial features using a plurality of predefined facial parameters, etc., and integrating the generated graphics and animation at a low bit rate, It also describes that processing corresponding to privacy protection is also performed.

Eric Petajan, "Very Low Bitrate Face Animation Coding in MPEG-4", 2000 face2face Animation. Inc., pp.1-33.

  However, as described above, in a communication system that extracts facial expressions from a caller's face image, graphicizes or animates them, and transmits information such as these images to the communication destination, even if the communication face is an animated face, etc. Even if the image is transmitted, if the change in facial expression of the caller is scarce, sufficient communication effect cannot be expected, and improvement of communication ability is not always achieved.

  It is an object of the present invention to provide an image communication system and an image generation method capable of solving such drawbacks of the prior art and constantly communicating with a communication partner while improving communication ability.

  In order to solve the above-described problems, the present invention captures a sender's face image at a transmission terminal device, and displays an image different from the face image on the reception terminal device via a wired or wireless communication line. The transmitting terminal device captures the face image of the sender and converts it into a digital image; and facial expression feature extraction means for extracting feature point data representing each part of the face from the digital image as a facial expression parameter; And an emotional deformation means for designating a part to be deformed independently for deformation of a part that causes an expression in the face image and replacing the value of the corresponding part of the facial expression parameter. An emotion designation means for designating deformation for a part that brings about, a magnification designating means for specifying a deformation ratio for the designated part, and extraction A shape deforming means for replacing the expression parameter value of the specified part with the prescribed deformation ratio, and according to the data output from the transmitting terminal device, as a character to be displayed at the communication destination One of the transmission terminal device and the reception terminal device includes display model storage means for storing model data, and character generation means for generating a character image by associating model data as a character with facial expression parameters. .

  In the image communication system of the present invention, facial expression parameters are extracted by a facial expression feature extraction unit from a digital image obtained via an image input unit, and the transformation to the part that causes the facial expression by the emotion designating unit of the emotion transformation unit is independently performed. Specify, specify the deformation ratio with the magnification specifying means, supply it to the emotion specifying means, supply the values of the corresponding part to the shape deforming means as a set of values, and replace with the values supplied by the shape deforming means According to the request from the user, the facial expression parameter is output and the display model storage means and the character generation means are arranged in either the transmission terminal apparatus or the reception terminal apparatus according to the data output from the transmission terminal apparatus. It is possible to provide the communication partner with exaggerated character actions and expressions based on the intentions.

  In order to solve the above-described problem, the present invention provides an image generation method for capturing a sender's face image and displaying an image different from the face image on the receiving side via a wired or wireless communication line. The method includes a first step of pre-selecting a character as an image different from the face image, a second step of capturing a sender's face image and converting it into a digital image, and features indicating each part of the face from the digital image A third step of extracting point data as a facial expression parameter, a fourth step of determining whether or not to specify a deformation independently for a deformation of a part that causes a facial expression in a face image, and specifying this deformation If not, the display parameter is simply output, and if this deformation is specified independently, the value of the corresponding part of the facial expression parameter is specified, and the value of the corresponding part is replaced A sixth step of outputting model data as a character to be displayed at the communication destination according to data output from the transmission side, and a seventh process of generating a character image by associating the model data as a character with the facial expression parameter This process is performed by either one of the processing on the transmission side and the processing on the reception side.

  The image generation method of the present invention selects a character in advance, converts a sender's face image into a digital image, extracts facial expression parameters from this digital image, targets deformation of the part, and specifies whether to specify independently If the value is specified independently according to the judgment of the expression, the value of the corresponding part of the facial expression parameter is specified, the value of the corresponding part is replaced, and displayed at the communication destination according to the data output from the transmission side As the character, the model data is output and the character image is generated by either the processing on the transmission side or the processing on the reception side, so that the behavior and expression of the character is based on the intention according to the request from the user. Can be provided to the other party, especially when model data is output and character images are generated for processing on the receiving side, reducing the transmission burden. It is possible.

  Next, an embodiment of an image communication system according to the present invention will be described in detail with reference to the accompanying drawings.

  In this embodiment, the image communication system of the present invention is applied to the information communication system 10. The illustration and description of parts not directly related to the present invention are omitted. In the following description, the signal is indicated by the reference number of the connecting line in which it appears.

  As shown in FIG. 3, the information communication system 10 of the present embodiment is a system that exchanges audio or images between a transmission terminal device 12 and a reception terminal device 14 via an IP (Internet Protocol) network 16. The transmission terminal device 12 and the reception terminal device 14 are not limited to the IP network 16, but may be a wired communication line or a wireless line such as a public line network. 1 includes an image input unit 18, an expression feature extraction unit 20, an emotion change unit 22, a display model storage unit 24, a display model generation unit 26, a background model storage unit 28, an image generation unit 30, a code A conversion unit 32 and a transmission unit 34.

  The image input unit 18 includes a light receiving unit in which light receiving elements 35 such as a CCD (Charge Coupled Device) and a CMOS (Complimentary-Metal Oxide Semiconductor) are arranged in a two-dimensional array through the optical system. The optical image formed by the photoelectric conversion function in the light receiving unit is converted into an analog electric signal, and the analog electric signal is converted into a digital signal. The two-dimensional digital image signal 36 thus obtained is supplied to the facial expression feature extraction unit 20.

  The facial expression feature extraction unit 20 extracts the contours of eyes, nose, mouth, eyebrows, etc., which are each part of the face, from the digital image signal 36 acquired from the image input unit 18 as feature points, and detects the feature points as coordinate data It has a function to do. Each of the detected coordinate data is output to the emotion change unit 22 as a parameter 38 for changing the expression.

  The emotion changing unit 22 includes a shape deforming unit 40, an emotion designating unit 42, and a magnification designating unit 44. The emotion changing unit 22 has a function of targeting each part of the face to be included in the selected model data, replacing the corresponding parameter independently for each part of the supplied parameter 38, and outputting it as a new parameter 52. Have. This function recognizes input data 46 expressed by the operation of a key button (not shown) as emotion, specifies the emotion type and emotion magnitude according to the recognition, and changes the parameter 38 according to this specification. ing. The magnification designating unit 44 has a function of defining the magnification as a ratio of changing the parameter 38 in accordance with the number of key button operations, pressing duration (input duration), input numerical value, and the like. Output to the designation unit 42. The numerical value may be a value representing the slope of the function. It is preferable to prepare in advance a parameter table for designating a magnification in accordance with these inputs, and to determine the magnification (enlargement or reduction) based on this table. The magnification specifying unit 44 described below has the same function. The emotion designation unit 42 has an emotion recognition function, and supplies the shape modification unit 40 with designation data 50 that is a set of a part corresponding to the recognized emotion and magnification data for the part.

  The shape deforming unit 40 has a function of associating the supplied parameter 38 with the corresponding designation data 50 and converting it into a value corresponding to the magnification data, that is, coordinate data. The shape deforming unit 40 outputs the parameter 52 varied by the new conversion to the display model generating unit 26.

  The model data 54 selected from the display model storage unit 24 is supplied to the display model generation unit 26. The display model storage unit 24 is a storage that stores model data to be displayed at the communication destination. The display model storage unit 24 stores a plurality of model data, and the display model storage unit 24 can select a character to be displayed to a communication partner according to a user's request in advance by operating a selection unit (not shown).

  The display model generation unit 26 acquires the parameter 52 from the shape deformation unit 40, that is, the coordinate data of the facial expression feature and the model data 54 from the display model storage unit 24, and associates them with each part of the model data 54 based on the parameter 52. By taking the coordinates, it has a function of generating a display model image in which the movement of the user's facial expression and the movement of the model data are synchronized. The display model generation unit 26 outputs model data (display model image) 56 synchronized with the movement of the facial expression of the user to the image generation unit 30.

  Here, the model data described above is data in which a three-dimensional CG wireframe model or the like is provided, and a model representing muscles is three-dimensionally arranged on this model. When this model data represents a human face, the muscles of the face move according to the change by changing the model data. This change can express facial expressions, such as joy, anger, and sadness, and the movement of the mouth at the time of speech. The model data may be based on a two-dimensional image such as a person or a cat. In this embodiment, such model data is called a character, a character image, or a display model image.

  The display model generating unit 26 three-dimensionally arranges muscle models on the wire frame and contracts each muscle, thereby deforming the wire frame itself and giving the character a facial expression. The display model generation unit 26 preferably includes a rendering processing function. As a result, the display model generation unit 26 outputs the rendered model data 56 to the image generation unit 30.

  Similar to the display model storage unit 24, the background model storage unit 28 is a storage that stores background image data 58 of model data to be displayed at the communication destination. The background model storage unit 28 also selects a background image in advance by a user operation on a selection unit (not shown), and the background model storage unit 28 outputs background image data 58 corresponding to the selected background image to the image generation unit 30.

  The image generation unit 30 has a function of generating the composite image data 60 by combining the model data 56 generated by the display model generation unit 26 and the background image data 58 acquired from the background model storage unit 28. The image generation unit 30 outputs the generated combined image data 60 to the encoding unit 32.

  The encoding unit 32 compresses the supplied composite image data 60 and sends the compressed image data 62 to the transmission unit 34. The compression process is performed by applying a known technique. Although not described so far, the audio data is also compressed and sent to the transmission unit 34. The transmitting unit 34 transmits the compressed data including the compressed audio data together with the compressed image data 62 as the transmission signal 64 to the receiving terminal device 14 that is a communication partner via the IP network 16.

  The receiving terminal device 14 includes a receiving unit 66, a decoding unit 68, and an output unit 70 as shown in FIG. The reception unit 66 receives the transmission signal 64 from the transmission unit 34 of the transmission terminal device 12, and sends the acquired reception signal 72 to the decoding unit 68. The decoding unit 68 performs a decoding process on the received signal 72 and restores the supplied digital image signal or digital audio signal. The decoding unit 68 supplies these restored signals 74 to the output unit 70.

  Although not shown, the output unit 70 includes a monitor, a speaker, an earphone, a headphone, a vibrator, and the like. The monitor has a function of displaying the supplied digital image signal. The speakers, earphones, and headphones convert the analog audio signal D / A converted by the restoration unit 68 into audio and output it.

  With this configuration, an image of a character reflecting the intention of the user who uses the transmission terminal device 12 is transmitted to the reception terminal device 14, and the image supplied by the reception terminal device 14 is reproduced.

  Next, the operation of the information communication system 10 will be described with reference to the transmission terminal device 12 in FIG. 4 and the function in FIG. 5 and the operation procedure of the reception terminal device 14 in FIG. The transmission terminal apparatus 12 selects some model data such as a character and a background by the user after activation and before transmission (step S10). For selection, the user selects a desired display model from a plurality of display models and background models stored in the display model storage unit 24 and the background model storage unit 28 by a user interface such as a selection button of the transmission terminal device 12. . This selection may be set so that it can be selected before the call starts. After this setting, the process proceeds to image input processing (to step S12).

  In the image input unit 18, incident light 35 from the object scene is converted into an electrical signal by a photoelectric conversion function, and an optical image such as a captured person or scene is converted into a digital image signal 36 and output to the facial expression feature extraction unit 20 ( Step S12). The facial expression feature extraction unit 20 acquires the supplied digital image signal 36, and uses a known method such as a feature point search or a template matching process using an image processing filter for the digital image signal 36 to extract a facial region. Extract (step S14). The contours of each part of the face, such as the eyes, nose, mouth and eyebrows, are predefined. Each defined part is represented by position coordinate data. This position coordinate is a feature point. The feature of the facial expression is extracted as a parameter 38 that associates the part and the position coordinate data in this way. A specific feature point sampling method can also be detected as 68 FAPs (Face Animation Parameters) described in the WG11 / MPEG committee draft.

  Next, the emotion changing unit 22 determines whether or not there is an emotion instruction for giving an emotion effect to the supplied parameter 38 (step S16). The determination is made based on the input data 46 supplied via the user interface (not shown) of the transmission terminal device 12 (step S16). If it is determined that the input data 46 is supplied (YES), the process proceeds to step S18.

  In step S18, it is recognized what kind of emotion is to be expressed from the input data 46 supplied to the emotion designating unit 42, and a part to be changed according to the recognition is specified. In addition, the magnification designation unit 44 changes the size of the emotion based on the input data 46, and sets this size. The magnification designating unit 44 may have a conversion function 76 as shown in FIG. 5 (a), and the conversion function 76 may be used as a model to determine the change magnification. In some cases, it is desirable to exaggerate a certain facial expression during communication. In this case, it is preferable to further amplify the behavior shown in the previous expression. In order to realize this operation, as shown in FIG. 5 (b), by changing the slope of the above-described function model to a broken line 78, a one-dot chain line 80, and a solid line 82 so as to increase in time series, The behavior of the expressed face can be gradually amplified. The magnification specifying unit 44 is supplied with the magnification data 48 indicating the size to the emotion specifying unit 42.

  In this manner, the parameter to be changed in accordance with the emotion and the designation data 50 that is a set of the parameters are sent to the shape deforming unit 40. The shape deforming unit 40 converts the supplied parameter 38 into coordinate data based on the designated data 50, and replaces or updates the value of the corresponding parameter (step S20). The shape deforming unit 40 supplies all the parameters 52 including the updated values to the display model generating unit 26.

  Note that the deformation reference position coordinates (the deformation origin) corresponding to the magnification or ratio of the operation can be arbitrarily set in advance. When there is no input from the emotion designating unit 42, the parameter 38 acquired from the facial expression feature extracting unit 20 is transferred to the display model generating unit 26 as it is.

  If it is determined that the input data 46 is not supplied (NO), the process proceeds to step S22.

  Next, the display model generation unit 26 associates the parameter 52 (expression feature data) with each part of the model data 54 acquired from the display model storage unit 24, and moves the facial expression of the user and the movement of the model data. Is generated and output to the image generation unit 30 as model data 56 (step S22).

  The image generation unit 30 is supplied with the background image data 58 according to the selection in step S10 among the plurality of background images stored in the background model storage unit 28. The image generation unit 30 synthesizes the supplied model data 54 and background image data 58 by a known image synthesis method (step S24). The image composition unit 30 supplies the composite image data 60 to the encoding unit 32.

  The encoding unit 32 compresses the supplied composite image data 60 and sends it to the transmission unit 34 (step S26). For image compression, for example, compression according to the MPEG4 standard may be used. The transmitting unit 34 converts the supplied compressed data according to the communication protocol and transmits the converted data to the receiving terminal device 14 (step S28). The transmitting terminal device 12 determines whether or not to end this series of processing. When this process is continued (NO), the transmission terminal device 12 returns to the image input process and repeats the above-described process. When the process is to be ended (YES), the transmission terminal device 12 ends the communication by operating a disconnect button (not shown) that disconnects the communication.

  More specifically, emotional change will be described. For example, “surprise” may be expressed as the emotional change, and here, an eye is selected as an arbitrary part representing the emotional change. In this case, the emotional change visually expresses the emotion by enlarging the size and shape of the character's eyes. When the input change amount 84 of the eye is supplied as the input data 46 as shown in FIG. 5 (a), the output change amount 86 can be amplified from the original size by the function of FIG. 5 (a). .

  In general, the user can specify the transformation of the behavior of the facial expression part of the model data without being limited to the eyes, and replace the FAP data with different data. Can change behavior or emotion.

  In some cases, the expression of continuous “surprise” is exaggerated during communication. In this case, the size of the eyes may be expressed by further amplifying the behavior shown in the previous “surprise” expression during the dialogue. The exaggerated expression can be realized by gradually changing the slope of the function as shown in FIG. 5 (b). The request for this expression varies the slope of the function in accordance with the number of times the key button that outputs the input data 46 is pressed, the duration of pressing the key button, or a numerical value input by the user. As a result, the transmission terminal device 12 has a function of generating the parameter 52 in which the parameter 38 for deforming the model data is dynamically changed according to the designation by the user.

  Next, the operation of the receiving terminal device 14 will be described with reference to FIG. The reception terminal device 14 is in a reception standby state so that a signal transmitted from an unspecified transmission terminal device can be generated (start). In this state, the receiving terminal device 14 receives the transmission signal 64 from the transmitting terminal device 12 described above by the receiving unit 66 (step S50). The receiving unit 66 sends the received signal 72 to the decoding unit 68.

  The decoding unit 68 performs decompression processing on the encoded compressed data included in the received signal 72 and decodes it to restore a digital image signal or digital audio signal (step S52). The decoding unit 68 converts these decoded signals into analog signals according to the form handled by the output unit 70 and sends them to the output unit 70. The output unit 70 outputs the supplied signal 74 from each output device (step S54). Among the supplied signals 74, the image signal displays an image on a monitor, and the audio signal is output as sound from a speaker which is an audio output device. At this time, since the supplied image exaggeratedly expresses the character's motion in response to a request from the user, the receiver expresses emotional expressions that cannot be conveyed by the words from the sender who is the other party. It can be received as visual information.

  Next, after the output process, it is determined whether or not to end this series of reception processes, that is, communication (step S56). When this communication is continued (NO), the process returns to the reception process and is repeated. Further, when the communication is disconnected (YES), for example, the communication is disconnected by pressing the “disconnect” key button.

  As described above, the information communication system 10 controls the deformation rate of the character according to the number of times the user interface key is pressed and the duration of the pressing operation of the key when the motion of the character is deformed. This allows users to communicate emotional changes to the other party without having to make detailed settings when expressing their emotions, so they can improve their emotional expression between speakers and improve their communication skills. be able to.

Next, a modified example in which the configuration of the transmission terminal device 12 in the information communication system 10 is modified will be described. In the following embodiments, the transmission terminal device 12 is constructed with substantially the same components. For this reason, in the following embodiments, the reference numerals used in the previous embodiments are attached to the respective components, and the description of the same reference numerals is omitted so as to avoid the complexity due to the repetition of the same description.
<< First Modification >>
The difference from the previous embodiment in the first modification is in the components of the emotion changing unit 22 and its connection, as shown in FIG. In addition to the shape deforming unit 40, the emotion designating unit 42, and the magnification designating unit 44, the emotion changing unit 22 is provided with a history managing unit 88 that manages a call history. In addition, the emotion designation unit 42 is connected not only to the magnification designation unit 44 but also to the history management unit 88. The history management unit 88 records, for each communication partner, for example, the magnitude of the behavior performed by the previous character, that is, the magnification, and uses each of these records as a reference magnification when each character changes emotions next time. It has a function of managing information on the communication partner as history information. This management function includes a function of searching history information. This search is performed with respect to the history management unit 88 in a situation where the magnification designation is not supplied. Accordingly, the history management unit 88 exchanges information with the emotion designating unit 42, and stores, as history information, a part to be deformed with respect to the deformation of the facial expression designated by the user, deformation magnification, communication partner information, and the like. ing.

  Further, the emotion designating unit 42 receives not only the shape deforming unit 40 but also the history managing unit 88, the information received from the user by the deformed part designated by the input data 46 and the deformation rate acquired by the magnification designating unit 44. Also supply. In this case, the emotion designating unit 42 has a function of designating a size for a part to be changed based on a search result (stored information) for the partner information currently being communicated and generating an update parameter.

  Next, the operation of the transmission terminal apparatus 12 including these functions will be described with reference to FIGS. The same steps as those in the previous embodiment are denoted by the same reference numerals, and description thereof is omitted. The transmission terminal device 12 in the present embodiment is the same from the selection by the user (step S10) to the emotion instruction determination processing (step S16). If it is determined that there is no emotion instruction (NO), the process proceeds to the CG image generation process for the character shown in FIG. 9 via the connector A (to step S22). Also, upon receiving an emotion instruction (YES), the process proceeds to a magnification instruction determination process (to step S32). This determination process is a process for determining whether or not the data indicating the magnification in the input data 46 is supplied within a predetermined time, for example. When there is a magnification instruction (YES), magnification data 48 indicating the instructed magnification is supplied to the emotion designation unit 42. The emotion designating unit 42 creates instruction data 50 using the instructed emotion part and its magnification data as a set of changing parameters (step S18). The emotion designating unit 42 supplies the instruction data 50 to the shape deforming unit 40. The shape deforming unit 40 updates a parameter corresponding to the instruction data 50 among the supplied parameters 38 with the instruction data 50 (step S20). After this process, the process proceeds to the history information recording process of FIG. 9 via the connector B (to step S34). This process will be further described later.

  On the other hand, returning to the magnification instruction determination process (step S32), if it is determined that there is no magnification instruction (NO), the process proceeds to the search process of the history management unit 88 (to step S36). The history management unit 88 stores history information that associates communication partner information with the previous parameter. This search is processing for detecting the current communication partner based on the communication partner information from the history information.

  Next, it is determined whether there is a history as the determination of the search result (step S38). When the history information corresponding to the history management unit 88 is detected (YES), the process proceeds to the parameter designation process corresponding to the history (to step S40). If no history information is detected (NO), the process proceeds to parameter specification processing (to step S42). The parameter designation process corresponding to the history (step S40) is performed by the emotion designation unit. The emotion designation unit 42 receives supply of history information detected from the history management unit 88, and designates an update parameter to be changed based on this information. The emotion designating unit 42 supplies the designated update parameter to the shape deforming unit 40 as instruction data 50.

  The parameter designation process (step S42) is also performed by the emotion designation unit 42. At this time, data indicating that there is no history information is supplied from the history management unit 88 to the emotion designation unit 42. Since there is no data with a size to compare anything, the emotion designating unit 42 generates a predetermined parameter defined in advance for such a case as a parameter to be updated and supplies it to the shape deforming unit 40.

  The shape deforming unit 40 updates the corresponding parameter (step S20), and thereafter proceeds to a history information recording process (to step S34). In this recording process (step S34), the processed parameter supplied from the shape deforming unit 40 via the signal line for sending the instruction data 50 and the magnification data 48 is associated with the communication partner information and recorded in the history managing unit 88. It is processing. By performing the recording process at this position, the history management unit 88 records the latest information as a history. Further, this recording may be performed at every predetermined time interval, or at least only for a communication partner for which no history exists.

  Subsequent processing is performed by the same operation procedure as the previous embodiment from the character CG image generation processing (step S22) to the end determination processing (step S30). When continuing in the end determination process (NO), the process is returned to the image input process via the connector C.

  In this embodiment, the change in the shape of the display model is made different depending on the other party information. However, the change in the shape of the display model may be made different for each character used. In this case, it can be realized by adding a character ID (Identification) or the like used for information managed by the history management unit 88.

In this way, the emotional change made to the communication partner is reproduced based on the communication history, so that the user does not have to specify the emotional change for each communication partner each time compared to the previous embodiment. It is possible to improve user operability.
<< Second Modification >>
Next, a second modification will be described. The difference from the previous embodiment in the second modification is in the components of the emotion change unit 22 and its connection destination. The emotion changing unit 22 includes a shape deforming unit 40, a background deformation specifying unit 90, and a magnification specifying unit 44. The background deformation designation unit 90 has a function of designating deformation of the arrangement of each feature point with respect to the feature points of the background image set in advance by the user. As for the deformation instruction by the background deformation designating unit 90, the feature point of the background is designated by the input data 46.

  The shape deforming unit 40 generates a background image deformed according to the magnification designated by the instruction data 50 supplied to the background data 58, that is, the rate of change. The shape deforming unit 40 supplies the generated background image data 92 to the image generating unit 30. The background image data 92 is data corresponding to the parameter 52 of FIG. 7 that has undergone the update process in the shape deformation process.

  If there is no input from the background deformation specifying unit 90, the background image stored in the background model storage unit 28 is simply transferred to the image generation unit 30.

  With this configuration, the background image can be exaggerated and deformed into a shape according to the user's request.

  This operation procedure will be briefly described with reference to FIG. Since this operation procedure is performed based on the configuration of FIG. 10, the CG image generation process (step S22) of the character is performed immediately after the facial expression extraction (step S14). This process may be performed immediately before the image composition process (step S24) as before. Further, the process from the user selection to facial expression extraction and the image synthesis to the end determination process are the same processes as before.

  In the operation of the present embodiment, a background shape determination process (step S44), a parameter change process (step S46), and a background image deformation process (step S48) are performed with the shape deformation target as the background. There is a feature in the point. In the background shape determination process (step S44), the background deformation designating unit 90 determines whether or not there is input data 46 indicating an instruction for the background shape. If there is no instruction for deformation (NO), the flow proceeds to image composition processing (to step S24). If there is an instruction for deformation (YES), the process proceeds to a process for designating a change in parameters (go to step S46).

  Here, the background image is represented based on, for example, a so-called parametric expression in which each coordinate value in the image region is expressed by one parameter. Parametric representation is a conventionally known mechanism in which (x, y, z) is uniquely determined for a certain parameter (t) if it is a curve, or (u, v) if it is a curved surface. In addition, the background image can also be realized by expressing it using a lettering method using fine particles called particles.

  The parameter changing process (step S46) to be changed is performed by the background deformation specifying unit 90, and the feature point to be changed is specified based on the input data 46 among the feature points preliminarily defined in the background image and supplied. A combined parameter is generated according to the magnification data 48. The magnification data 48 is data obtained from the input data 46 supplied to the magnification designating unit 44.

  Specifically, in the case of a background image using parametric expression, deformation refers to deforming a curved surface by changing the coordinate value at an arbitrary position. In this case, the magnification of deformation is an arbitrary This corresponds to the magnitude of fluctuation of the coordinate value of the position. In the case where the deformation is performed on a background image using particles, the change in the parameter value indicated by the number of particles or the like corresponds to the magnification of the deformation.

  In the background image deformation process (step S48), the shape deformation unit 40 reads the background model 58 desired by the user from the background model storage unit 28, and is supplied with the change rate of the feature points arranged in the background image. This is a process of determining the arrangement of the feature points by changing the shape according to the designated data 50.

  Specifically, for example, when expressing the feeling of “anger”, a background image displaying a flame may be displayed due to a change in the shape of the background image. In order to express the expression of this emotion more exaggeratedly, it is possible to express the emotion by changing the amount of movement of the feature points in the background image, such as changing the size and fire power of the flame, and making the movement different. .

By operating in this way, the operation of the background image can be expressed exaggeratedly according to the request from the user, and even if there is no change in the character image, it can be expressed in words to the other party from the change in the background image. It is possible to convey emotional expressions that cannot be transmitted as visual information.
<< Third Modification >>
Next, a third modification will be described. The third modification is different from the second modification in the object whose shape is to be deformed. As shown in FIG. 12, since the transmission terminal device 12 uses a deformation target as a component, the emotion deformation unit 22 includes a component deformation designation unit 94, and a component model storage unit 96 that provides model data of the deformation target is also provided. It is arranged. Model data is supplied to the shape deforming unit 40 from the background model storage unit 28 and the part model storage unit 96, respectively. Here, the component is an item that assists the character's emotional expression such as “sweat” and “tears”, for example, and each item is expressed using parametric expressions and particles as in the background image described above.

  The component deformation designation unit 94 has a function of designating a feature point of a component to be deformed according to input data 46 supplied in the same manner as the background deformation designation unit 90 of the second modification. Input data 46 is supplied to the component deformation designating unit 94, feature points of the component are selected, and magnification data 48 for the component is supplied. The component deformation designating unit 94 supplies a parameter combining the feature point of the selected component and its magnification data 48 to the shape deforming unit 40 as instruction data 50.

  In addition to the supplied instruction data 50, the shape deforming unit 40 is supplied with image data 58 including background image data and component image data. The shape deforming unit 40 updates the part image data corresponding to the instruction data 50 in the image data 58 to the parameter indicated by the instruction data. The shape deforming unit 40 supplies the image generating unit 30 with the image data 98 including the updated component data together with the background image. The image generation unit 30 combines the model data 56 of the character having a facial expression with the supplied image data 98. As a result, the image generation unit 30 outputs the composite image data 60 to the encoding unit 32, encodes and compresses it by the encoding unit 32 as in the previous embodiments, and sends it through the transmission unit 34. To do.

  This operation procedure will be briefly described with reference to FIG. This operation procedure is the same as the selection by the user, image input, facial expression extraction, and character image generation in FIG.

  Characteristic operations in the present embodiment are: a part shape determining process (step S60), a parameter changing process (step S62), and a part image deforming process (step S64) using a shape deformation target as a part. There is in going. In the part shape determination process (step S60), it is determined whether or not there is input data 46 indicating an instruction for the part shape in the part deformation specifying unit 94. If there is no instruction (NO), the process proceeds to image composition processing (to step S24). If there is an instruction (YES), the process proceeds to a process for designating a change in parameters (go to step S62).

  Here, the component image is also expressed using, for example, a parametric expression and a lettering method using fine particles called particles, like the background image described above.

  The process for specifying the parameter to be changed (step S62) is performed by the component deformation specifying unit 94, and the feature point to be changed is specified based on the input data 46 among the feature points defined in advance in the component image. A combined parameter is generated in accordance with the magnification data 48 for. The magnification data 48 is data obtained from the input data 46 supplied to the magnification designating unit 44.

  In the part image deformation process (step S64), the model data 58 including the part model desired by the user from the part model storage unit 96 together with the background model desired by the user from the background model storage unit 28 is stored in the shape deforming unit 40. This is a process of reading out, changing the change rate of the feature points arranged in the component image in accordance with the designated data 50, and determining the arrangement of the feature points.

  Thereafter, three images, that is, a character image, a background image, and a component image are synthesized, the obtained synthesized image data is subjected to an encoding process, and compressed data is transmitted. If it is continued by the end determination process, the process returns to image input and repeats a series of processes. When the call is to be terminated, for example, the power is turned off and the call is disconnected.

By operating in this way, it is possible to exaggerate the behavior of the component image in response to a request from the user, and even if the character image or the background image is poorly changed, this exaggerated expression can be used to call Emotional expressions that cannot be conveyed in words can be conveyed as visual information to the other party.
<< 4th modification >>
Next, a fourth modification will be described. The information communication system 10 of the fourth modified example generates parameters and code data used in the transmission terminal device 12 and transmits them to the reception terminal device 14, and the reception terminal device 14 receives parameters and codes from the transmission terminal device 12. This is a system that receives data and generates a display model at the receiving terminal device.

  As shown in FIG. 14, the transmission terminal device 12 in the present embodiment includes an image input unit 18, an expression feature extraction unit 20, an information change unit 22, a display model storage unit 24, a background model storage unit 28, an encoding unit 32, and A transmission unit 34 is provided. The transmission terminal device 12 of the present embodiment is configured without the display model generation unit 26 and the image generation unit 30 as compared with the transmission terminal device 12 of FIG. The transmission terminal device 12 of the present embodiment can be simplified in configuration.

  In the display model storage unit 24, a code table corresponding to the character is set in advance, and corresponding code data is output according to the user's character selection. The background model storage unit 28 also transmits code data corresponding to the selected background image, instead of transmitting the background image selected in response to the transmission. In the background model storage unit 28, a code table corresponding to the background image is set in advance, and corresponding code data is output according to the user's selection of the background image. On the transmission side, the display model storage unit 24 and the background model storage unit 28 need only have a code table. Since all the model data is not stored, the display model storage unit 24 and the background model storage unit 28 of this embodiment can be smaller in capacity than the previous embodiment.

  With such a configuration, the encoding unit 32 is directly supplied with the parameter 52 from the parameter generation unit 40, and is also supplied with the code data 100 representing the desired character model and the code data 102 representing the background model. Is done.

  In addition to the receiving unit 66, the decoding unit 68, and the output unit 70, the receiving terminal device 14 of the present embodiment includes a display model storage unit 24, a background model storage unit 28, and a display that have been provided in the transmitting terminal device 12 so far. A generation model 26 and an image generation unit 30 are included. The display model storage unit 24, the background model storage unit 28, the display generation model 26, and the image generation unit 30 are the same as those of the transmission terminal device 12 in FIG. In particular, the display model storage unit 24 and the background model storage unit 28 both have code tables. The receiving unit 66 receives the data 64 transmitted from the transmitting unit 34, and outputs a compressed received signal 72 (compressed data) to the decoding unit 68. The decoding unit 68 performs decoding processing on the supplied compressed data, and separately outputs a parameter 104 that is coordinate data of facial expression features, data 106 that specifies a character image, and data 108 that specifies a background image. Parameter 104 corresponds to parameter 52 on the transmission side, data 106 corresponds to data 100 on the transmission side, and data 108 corresponds to data 102 on the transmission side.

  The data 106 is supplied to the display model storage unit 24, and the display model storage unit 24 outputs the model data 54 of the character image corresponding to the data 106 to the display model generation unit 26. The display model generation unit 26 applies the parameter 104 to the supplied model data 54 to generate model data (image) 56 having a facial expression, and outputs the model data (image) 56 to the image generation unit 30. The display model generation unit 26 performs a rendering process on the supplied digital image signal. In this case, the information communication system 10 does not transmit the image data generated on the transmission side, and only a small transmission capacity such as parameters and code data is required. Therefore, the user can save a communication fee.

  The data 108 is supplied to the background model storage unit 28, and the background model storage unit 28 outputs model data 58 of the background image corresponding to the data 108 to the image generation unit 30. The image generation unit 30 synthesizes an image based on the model data 56 and the model data 58. The image generation unit 30 supplies the composite image data 60 to the output unit 70. It goes without saying that not only image data but also audio data is supplied to the output unit 70. On the monitor of the output unit 70, the model image selected in synchronization with the facial expression of the sender of the communication partner can be moved to display the background image.

  Next, the operation of the information communication system 10 will be briefly described. As shown in FIG. 16, the operation is basically the same as the procedure of FIG. However, there is a feature in that there is no generation of a character image and synthesis of a background image. A character and a background image are selected in advance (step S10). For example, the selection is performed by the user with the selection button before the start of the call. In particular, in this embodiment, instead of transmitting the selected character data 54 from the display model storage unit 24, the code data 100 corresponding to the selected character is transmitted. The background model storage unit 28 also transmits code data 102 corresponding to the selected character.

  Thereafter, the transmitting terminal device 12 sequentially inputs the user's images (step S12), extracts facial expressions (step S14), and changes the extracted facial expression parameters according to the emotional change. It is determined whether or not (step S16). If no change is specified, the current parameter 38 is output. When changing is designated, the designation of the parameter to be changed and its magnification are set as one set (step S18), and the parameter 38 is updated (step S20). That is, a new parameter 52 is generated by updating each set of corresponding parameters, and is output to the encoding unit 32.

  The encoding unit 32 is supplied with parameters 52 and code data 100 and 102. The encoding unit 32 performs compression processing on these data and outputs the result to the transmission unit 34 (step S26). The transmitting unit 34 transmits the supplied compressed data 64 to the receiving terminal device 14 (step S28). If the end determination process (step S30) continues, the process returns to image input and a series of processes are repeated. When the call is to be terminated, for example, the power is turned off and the call is disconnected.

  Next, the receiving terminal device 14 is brought into a communication connection state with the transmitting terminal device 12 (negotiation), and then receives the compressed data 64 from the transmitting terminal device 12 (step S50). The receiving terminal device 14 performs a decoding process on the acquired compressed received signal 72 (compressed data) (step S52). The parameters are divided into the parameter 104 and the code data 106 and 108 by the decoding process, and supplied to the display model generation unit 26, the display model storage unit 24, and the background model storage unit 28, respectively. Although the display model storage unit 24 and the background model storage unit 28 are not shown in FIG. 17 as processes, the display model storage unit 24 displays the character model data 54 and the background image model data 58 according to the supplied code data 106 and 108, respectively. The data is supplied to the generation unit 26 and the image generation unit 30. That is, the display model storage unit 24 and the background model storage unit 28 select a model or background image of the character to be output based on the correspondence between the code data supplied from the plurality of stored model data and the code table. Output.

  Note that the receiving terminal device 14 transmits default model data when there is no character or background image corresponding to the code data of the character designated from the transmitting terminal device 12 or the code data of the background image.

  Next, a CG image of the character is generated using the supplied parameter 104 and the character model data 54 (step S22). The CG image is generated in an exaggerated manner by associating each part of the model data with the expression data that is the parameter 104. The movement of the user's facial expression is reflected in the exaggerated model data 56. As a result, the character is expressed in synchronization with the movement of the sender.

  Next, the model data 56 and the background image model data 58 are synthesized (step S24). The combined image data 60 is output from the image generation unit 30 to the output unit 70. The output unit 70 displays the image supplied to the monitor (step S54). The receiver can know the delicate feelings of the communication partner by viewing the character image synchronized with the movement of the sender. Next, when continuing in the end determination process (step S56), the process returns to the reception process and repeats the series of processes described above. When the call is to be terminated, for example, the power is turned off and the call is disconnected.

  By operating in this way, the information communication system 10 can reduce the load on transmission by compressing and transmitting parameters and code data instead of the image data 64 completed from the transmission terminal device 12. The receiving terminal device 14 can decode the supplied compressed data, acquire parameters representing the sender's facial expression, obtain a corresponding character or background image from the code data, and generate an image to be displayed. Become.

  In addition, although the embodiment of the present invention has been described by limiting the number of modifications to one for ease of explanation, the embodiment and the modifications are combined to generate and communicate more complex images. It goes without saying that it can be done.

  By configuring as described above, the information communication system 10 extracts the parameter 38 from the digital image 36 obtained from the transmission terminal apparatus 12 via the image input unit 18 by the facial expression feature extraction unit 20, and the emotion change unit 22 The emotion designating part 42 independently designates deformation for the part to be given a facial expression, and the magnification designating part 44 specifies the ratio of the deformation and supplies it to the emotion designating part 42, and sets the corresponding part values. Is supplied to the shape deforming unit 40 as the value 50, and the parameter 52 replaced with the value 50 supplied by the shape deforming unit 40 is output, and the display model storage unit 24 and the display model are generated according to the output data of the transmission terminal device 12. By arranging the unit 26 in one of the transmission terminal device 12 and the reception terminal device 14, it is possible to provide the communication partner with a character action or expression based on the intention according to the request from the user. . Compared with the case where a character image with a simple expression is simply provided, intentional intentions can always be expressed with respect to the other party, so that the communication ability can be further improved. Further, when the display model storage unit 24 and the display model generation unit 26 are arranged in the transmission terminal device 12, the configuration of the reception terminal device 14 can be greatly simplified, and when these are arranged in the reception terminal device 14, The amount of transmission can be greatly reduced.

  Depending on the data output from the transmission terminal device 12, the background model storage unit 28 and the image synthesis unit 30 are included in either the transmission terminal device 12 or the reception terminal device 14, thereby improving the quality of the displayed image. In particular, even if the transmission terminal device 12 is provided with the background model storage unit 28 and the image synthesis unit 30, the configuration of the reception terminal device 14 can be simplified, while the reception terminal device 14 has a background model storage unit. Providing 28 and the image composition unit 30 can also reduce the burden of transmission.

  The magnification designating unit 44 is performed in response to an instruction from the user, and dynamically changes and designates in accordance with any of the numerical value representing the number of times, duration, and magnification supplied as the input data. Thus, the user's emotion can be easily expressed as an operation.

  In addition, the emotion change unit 22 includes a history management unit 88, stores the part, magnification, and communication partner information corresponding to the past emotion as history information, and outputs the history information to the emotion designation unit 42 according to conditions. Thus, the operability of the user can be improved, and the emotion designating unit 42 sets the feature points of the target model data supplied according to the target to be deformed and the magnification for the feature points of the model data as a set. Thus, by supplying to the shape deforming unit 40, data replacement of the supplied parameter 38 can be easily performed.

  Here, the target is at least one of a part that characterizes the facial expression, a background image, and an item that assists the emotional expression of the character. This makes it easier to express the sender's emotion in the image than the representation of the character image in which the image displayed on the communication partner has a simple expression.

  In addition, by applying the image generation method of the present invention, a character is selected in advance, the face image of the sender is converted into a digital image, facial expression parameters are extracted from the digital image, and the deformation of the part is targeted, and is independently Depending on the data to be output from the transmission side, specify the value of the corresponding part of the facial expression parameter when assigning it independently according to the judgment whether or not to specify, and replace the value of the corresponding part A character based on the intention according to the request from the user by outputting the model data and generating the character image as the character to be displayed at the communication destination by either the processing on the transmission side or the processing on the reception side. Can be provided to the other party in the form of movements and expressions, and in particular, model data is output and character images are generated for processing on the receiving side. And, it is possible to reduce the burden of transmission.

  The transformation process of the object associated with the emotion specifies the deformation for the part that brings about the facial expression of the facial image, specifies the deformation ratio for the designated part, that is, the magnification, and the extracted facial expression parameters of the designated part. By replacing the value of the expression parameter with the rate of deformation and outputting it as a parameter 52, it is possible to more easily express the sender's emotion in the image than the expression of the character image with a simple expression.

  The image generation method performs output of model data as a background image to be displayed at a communication destination and image synthesis on either the transmission side or the reception side in accordance with data output from the transmission side, thereby generating a character image. Various image representations can be provided in addition to the display. In this case as well, the transmission burden can be reduced by outputting the model data and generating the character image for the processing on the receiving side.

  Also, the image generation method generates a set of feature points of model data to be supplied according to the object to be deformed and a magnification for the feature points, and facilitates data replacement of the supplied parameter 38. It can be performed, and the target is at least one of the items that characterize the facial expression, the background image, and the character's emotional expression. Thus, the sender's emotion can be expressed in an easy-to-understand manner.

  The image generation method instructs deformation of the target in accordance with input data from a user, and dynamically changes in response to any of the numerical values representing the number of times, duration, and magnification supplied by the input data. By doing so, the emotion of the user can be easily expressed as an operation.

  The image generation method determines whether or not the magnification of the target to be changed independently between the emotion determination and the emotion processing according to the determination is specified, and if there is a magnification specification, proceeds to the emotion processing according to the determination, If the magnification is not specified, search based on the history information whether or not the currently communicating party is included, determine whether there is history information corresponding to this search result, and when history information exists, By specifying a value corresponding to this history information and when there is no history information, specifying a predetermined value will provide some value for each communication partner, greatly increasing the user's operation Can be improved.

  In the embodiments of the present invention, the encoding and decoding processes have been described as communicating with either images or parameters. However, the present invention is not limited to this. For example, audio and acoustic data are also encoded and decoded. However, it may be multiplexed with the image and transmitted.

It is a block diagram which shows the schematic structure of the transmission terminal device in the Example of the information communication system to which the image communication system of this invention is applied. It is a block diagram which shows the schematic structure of the receiving terminal device in the Example of the information communication system to which the image communication system of this invention is applied. It is a figure which shows the structure in the Example of the information communication system to which the image communication system of this invention is applied. It is a flowchart explaining the operation | movement procedure of the transmission terminal device of FIG. It is a figure explaining the function model which shows the magnification in the magnification designation part of FIG. 1, and its change. It is a flowchart explaining the operation | movement procedure of the receiving terminal device of FIG. It is a block diagram which shows the schematic structure of the 1st modification in the transmission terminal device of FIG. It is a flowchart explaining the operation | movement procedure of the transmission terminal device (1st modification) of FIG. It is a flowchart explaining the operation | movement procedure following FIG. It is a block diagram which shows the schematic structure of the 2nd modification in the transmission terminal device of FIG. 11 is a flowchart for explaining an operation procedure of the transmission terminal device (second modification) in FIG. 10. It is a block diagram which shows the schematic structure of the 3rd modification in the transmission terminal device of FIG. 13 is a flowchart for explaining an operation procedure of the transmission terminal device (third modification) in FIG. It is a block diagram which shows the schematic structure of the 4th modification in the transmission terminal device of FIG. FIG. 15 is a block diagram showing a schematic configuration of a receiving terminal device corresponding to the transmitting terminal device of FIG. 15 is a flowchart for explaining an operation procedure of the transmission terminal device (fourth modified example) in FIG. 14. 16 is a flowchart for explaining an operation procedure of the receiving terminal device of FIG.

Explanation of symbols

10 Information communication system
12 Sending terminal device
14 Receiving terminal equipment
16 IP network
18 Image input section
20 Facial feature extraction unit
22 Emotion change department
24 Display model storage
26 Display model generator
28 Background model storage
30 Image generator
32 Encoder
34 Transmitter
40 Shape deformation part
42 Emotion designation part
44 Magnification specification part
66 Receiver
68 Decryption unit
70 Output section

Claims (12)

In an image communication system in which a sender's face image is captured by a transmission terminal device and an image different from the face image is displayed on the reception terminal device via a wired or wireless communication line,
The transmitting terminal device captures the sender's face image and converts it into a digital image; and
Facial expression feature extraction means for extracting feature point data representing each part of the face from the digital image as facial expression parameters;
An emotional deformation means for designating a part to be deformed independently for deformation of a part that causes an expression in the face image, and replacing a value of a corresponding part of the expression parameter;
The emotional deformation means includes an emotion designating means for designating a deformation for a part that causes an expression in the face image;
A magnification specifying means for defining a deformation ratio with respect to the specified portion;
A shape deformation means for replacing the value of the facial expression parameter of the designated part among the extracted facial expression parameters with the prescribed deformation ratio;
A display model storage means for storing model data as a character to be displayed at a communication destination in accordance with data output from the transmission terminal device, and a character that generates a character image by associating the model data as the character with the facial expression parameter An image communication system, comprising: a generation unit included in one of the transmission terminal device and the reception terminal device.   The system according to claim 1, wherein the system stores background model storage means for storing model data as a background image to be displayed at a communication destination, the character image, and the background according to data output from the transmission terminal device. An image communication system comprising an image composition means for compositing model data as an image in either one of the transmission terminal device and the reception terminal device.   3. The system according to claim 1, wherein the magnification specifying means is any one of a numerical value representing a number of times of being supplied as the input data, a duration of the input, and a magnification, in response to an instruction from a user. An image communication system characterized by being dynamically changed in accordance with an instruction of the image.   The system according to claim 1, 2, or 3, wherein the emotion modifying means stores, as history information, a part, magnification, and communication partner information corresponding to past emotions, and outputs them to the emotion designating means according to conditions. An image communication system characterized by including history management means.   5. The system according to claim 1, wherein the emotion designating unit sets a feature point of the model data to be supplied according to a target to be deformed and a magnification for the feature point of the model data. An image communication system characterized by being supplied to the shape deforming means.   6. The image communication system according to claim 5, wherein the target is at least one of a part characterizing the facial expression, the background image, and an item for assisting emotional expression of the character. In an image generation method for capturing a sender's face image and displaying an image different from the face image on a receiving side via a wired or wireless communication line, the method includes:
A first step of pre-selecting a character as an image different from the face image;
A second step of capturing the face image of the sender and converting it to a digital image;
A third step of extracting feature point data representing each part of the face from the digital image as a facial expression parameter;
A fourth step of determining whether or not to specify deformation independently for deformation of a part that causes an expression in the face image;
When the deformation is not specified, the display parameter is simply output, and when the deformation is specified independently, the value of the corresponding part of the facial expression parameter is specified, and the value of the corresponding part is replaced. ,
A sixth step of outputting model data as the character to be displayed at the communication destination in accordance with data output from the transmission side, and a character image is generated by associating the model data as the character with the facial expression parameter 7. An image generation method, wherein the step 7 is performed by any one of the processing on the transmission side and the processing on the reception side. The method according to claim 7, wherein the fifth step is an eighth step of designating a deformation for a part that causes a facial expression in the facial image;
A ninth step of defining a deformation rate with respect to the designated portion;
And a tenth step of replacing the value of the facial expression parameter of the designated part with the prescribed deformation ratio among the extracted facial expression parameters and outputting as the facial expression parameter Method.   The method according to claim 7 or 8, wherein the method includes an eleventh step of outputting model data as a background image to be displayed at a communication destination according to data output from the transmission side, the character image, and the character image. An image generation method characterized in that a twelfth step of synthesizing model data as a background image is performed on either the transmission side or the reception side.   10. The method according to claim 7, 8, or 9, wherein the method generates a set of feature points of model data to be supplied according to an object to be deformed and a magnification for the feature points, and Is at least one of a part that characterizes the facial expression, the background image, and an item that assists the emotional expression of the character.   11. The method according to any one of claims 7 to 10, wherein the method instructs transformation of the object according to input data from a user, and continues the input for the number of times supplied by the input data. An image generation method characterized by dynamically changing according to an instruction of any one of numerical values representing time and magnification. 12. The method according to any one of claims 7 to 11, wherein the method determines whether a target magnification to be independently changed between the fourth step and the fifth step is designated. The process of
When the magnification is designated, the process proceeds to the fifth step. When the magnification is not designated, the information corresponding to the past emotion, the magnification and the communication partner is used as history information, and the current communication is performed based on the history information. The 14th step of searching for the partner in
A fifteenth step of determining whether or not the history information exists by the search;
When the history information exists, a sixteenth step of designating a value changed corresponding to the history information;
And a seventeenth step of designating a predetermined value when there is no history information.

Images