JP2015015623A - Television telephone set and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a television telephone set capable of allowing smooth communication without letting a called person misunderstand by translating a gesture that a calling person takes.SOLUTION: A television telephone set (1) includes: gesture dictionaries (31A-31D) in which reference feature data representing features of a gesture and translation information representing a translation of the gesture are made to correspond to each other; a gesture analysis part (33) which generates photography-time feature data representing features of a gesture of a subject on the basis of an image of the subject, and selects translation information corresponding to photography-time feature data as a translation result from translation information of the gesture dictionary (31B); and a translation result composition part (34) which generates a translation result composite image by putting the translation result together with the image of the subject.

Description

  The present invention relates to a video phone for making a call using video and audio, and a program.

  When a videophone is used to make a call between a speaker and a receiver, the videophone of the speaker has call data including the image of the speaker captured by the camera and the voice of the speaker captured by the microphone. , To the receiver's video phone via the telephone network. As a result, the listener can view the image of the speaker while listening to the voice of the speaker.

  When the language of the speaker is different from that of the receiver, the receiver instantly translates the speaker's voice and has a conversation on the video phone. However, if the speaker has little knowledge of the speaker's language or if the speaker's voice cannot be translated instantaneously, the conversation cannot be performed smoothly. In order to solve this problem, Patent Document 1 describes a technique for translating a speaker's voice.

  In the technique described in Patent Document 1, a speech translation unit (not shown) is provided between a speaker's video phone and a receiver's video phone. The voice translation unit (not shown) converts a speaker's voice (for example, English voice) included in the call data transmitted from the speaker's videophone into text data (English notation text data). A text language translation unit (not shown) that translates the language described in the text data into the language specified by the listener (for example, Japanese), and the display unit of the receiver's videophone using the translated text data as subtitles An image text synthesizing unit (not shown) that synthesizes the image of the speaker included in the call data and transmits it to the receiver's video phone so that the image is displayed at the end (for example, the lower end). . Thus, the listener can see the translated subtitles along with the image of the speaker while listening to the voice (English) of the speaker.

JP-A-5-260193

  However, in the technique described in Patent Document 1, the voice of the speaker included in the call data is translated and transmitted to the receiver's video phone as subtitles, but the image of the speaker included in the call data is It is transmitted to the receiver's video phone as it is. For this reason, depending on the nationality, culture, and customs of the other party (listener), the gesture taken by the subject (speaker) of the sender may be perceived as an inappropriate gesture for the other party, and the communication may be frustrated There is. For example, in Japan, when you greet the other person lightly, you may perform a “raise your hand to show your palm” action, but in Europe this action may be recognized as an “insulting act”. .

  The present invention has been made in view of the above points, and by translating a gesture taken by a subject (speaker) of a transmission source, smooth communication without causing misunderstanding to the other party (listener). An object of the present invention is to provide a videophone and a program capable of achieving the above.

  The video phone according to the present invention includes a gesture dictionary that associates reference feature data indicating a feature of a gesture and translation information indicating translation of the gesture, and feature data at the time of shooting that indicates the feature of the subject based on the image of the subject. A gesture analysis unit that selects, as translation results, translation information corresponding to the shooting feature data from translation information in the gesture dictionary, and synthesizes the translation results with the subject image. A translation result synthesis unit for generating an image.

  ADVANTAGE OF THE INVENTION According to this invention, smooth communication can be aimed at without producing misunderstanding with respect to an other party (listener) by translating the gesture which the to-be-photographed object (speaker) takes.

It is a schematic block diagram which shows the structure of the video telephone 1 which concerns on 1st Embodiment of this invention. FIG. 2 is a flowchart for explaining the operation (videophone processing) of the videophone 1 according to the first embodiment of the present invention, using the configuration of FIG. FIG. 3 is a schematic block diagram showing the configuration of the videophone 1 according to the second embodiment of the present invention. FIG. 4 is a flowchart for explaining the operation (videophone processing) of the videophone 1 according to the second embodiment of the present invention, using the configuration of FIG. FIG. 5 is a schematic block diagram showing the configuration of the gesture translation unit 30 in the videophone 1 according to the third embodiment of the present invention. FIG. 6 shows an example of the determination storage unit 36 of FIG. It is a flowchart for demonstrating operation | movement (video telephone process) of the video telephone 1 which concerns on 3rd Embodiment of this invention using the structure of FIG.5 and FIG.6. The example of the gesture dictionary in the dictionary memory | storage part 31 of the gesture translation part 30 in the video telephone 1 which concerns on 4th Embodiment of this invention is shown. It is a figure for demonstrating operation | movement of the video telephone 1 which concerns on 4th Embodiment of this invention. The example of the gesture dictionary in the dictionary memory | storage part 31 of the gesture translation part 30 in the video telephone 1 which concerns on 5th Embodiment of this invention is shown. It is a figure for demonstrating operation | movement of the video telephone 1 which concerns on 5th Embodiment of this invention. It is a figure for demonstrating operation | movement of the video telephone 1 which concerns on 6th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The video phone according to the present embodiment is applied to a mobile phone, a smartphone, and the like.

[First Embodiment]
FIG. 1 is a schematic block diagram showing the configuration of the videophone 1 according to the first embodiment of the present invention. The video phone 1 includes a camera 11, a transmission unit 12, a microphone 13, a reception unit 21, a display unit 22, a speaker 23, a control unit (not shown), and a storage device (not shown). Yes.

  The control unit controls the camera 11, the transmission unit 12, the microphone 13, the reception unit 21, the display unit 22, and the speaker 23. The control unit includes, for example, a CPU (Central Processing Unit). The storage device stores a computer program executable by the computer, and the CPU reads and executes the computer program. As the display unit 22, an LCD (Liquid Crystal Display) is exemplified.

  The video phone 1 is used for a speaker and a receiver. Here, the video phone 1 used by the speaker is referred to as the speaker's video phone 1A, and the video phone 1 used by the receiver is referred to as the receiver's video phone 1B.

  In FIG. 1, for convenience of explanation, the camera 11, the transmission unit 12, and the microphone 13 are illustrated only in the video phone 1 </ b> A of the speaker. The camera 11 photographs a subject (speaker). The microphone 13 captures the voice of the speaker. The transmission unit 12 transmits call data including the image of the speaker captured by the camera 11 and the voice of the speaker captured by the microphone 13 to the receiver's video phone 1B via the telephone network 2. .

  In FIG. 1, for the sake of explanation, the receiving unit 21, the display unit 22, and the speaker 23 are shown only in the receiver's video phone 1 </ b> B. The receiving unit 21 receives call data transmitted from the videophone 1A of the speaker. The display unit 22 displays an image of the speaker included in the call data received by the receiving unit 21. The speaker 23 outputs the voice of the speaker included in the call data received by the receiving unit 21.

  The videophone 1 </ b> A of the speaker and the videophone 1 </ b> B of the receiver are connected via the telephone network 2. The telephone network 2 is provided with a speech translation unit 40, for example. For example, the speech translation unit 40 is assumed to have the same configuration as the technique described in Patent Document 1, and only a portion related to the present invention will be described.

  The speaker's video phone 1 </ b> A further includes a gesture translation unit 30. The gesture translation unit 30 is realized by software (the above-described computer program). The gesture translation unit 30 is controlled by a control unit in the videophone 1A of the speaker. The gesture translation unit 30 translates the gesture of the subject (speaker) in the image taken by the camera 11. The gesture translation unit 30 includes a dictionary storage unit 31, a gesture dictionary selection unit 32, a gesture analysis unit 33, and a translation result synthesis unit 34.

  In the dictionary storage unit 31, gesture dictionaries 31A to 31D are registered for each language. Each of the gesture dictionaries 31 </ b> A to 31 </ b> D includes reference feature data (feature vector) indicating features of an operation representing a hand movement or facial expression when a user performs a gesture, and translation (meaning) of the gesture. Are registered in association with a plurality of types.

  The gesture dictionary selection unit 32 selects a gesture dictionary corresponding to the language used by the speaker as a selection gesture dictionary from the gesture dictionaries 31A to 31D registered in the dictionary storage unit 31. Examples of the gesture dictionaries 31A to 31D include Japanese, English (US), English (UK), and Chinese, but are not limited thereto.

  The gesture analysis unit 33 generates shooting feature data (feature vector) indicating the feature of the speaker's work based on the speaker's image. The gesture analysis unit 33 refers to the selection gesture dictionary selected by the gesture dictionary selection unit 32, and selects, as a translation result, translation information corresponding to the shooting feature data from a plurality of types of translation information of the selection gesture dictionary. To do.

  The translation result synthesizing unit 34 synthesizes the translation result with the speaker's image and generates a translation result synthesized image to be displayed on the display unit 22 of the receiver's videophone 1B.

  FIG. 2 is a flowchart for explaining the operation (videophone processing) of the videophone 1 according to the first embodiment of the present invention, using the configuration of FIG.

  First, in the videophone 1A of the speaker, the gesture dictionary selection unit 32 of the gesture translation unit 30 makes a gesture dictionary acquisition request (S01). Here, S01 is executed, for example, when the speaker's videophone 1A makes a call in response to the operation of the speaker. In S01, the gesture dictionary selection unit 32 selects, from the gesture dictionaries 31A to 31D registered in the dictionary storage unit 31, a gesture dictionary (for example, the gesture dictionary 31B) corresponding to the language used by the speaker as the selection gesture dictionary. (S02). The gesture dictionary selection unit 32 notifies the gesture analysis unit 33 of the gesture translation unit 30 that translation preparation is complete (S03). Although not shown, the gesture dictionary selection unit 32 notifies the translation result synthesis unit 34 of the completion of translation preparation so that the gesture analysis unit 33 and the translation result synthesis unit 34 of the gesture translation unit 30 can acquire the same image. To do.

  Next, in response to the notification of translation preparation completion, the gesture analysis unit 33 makes a speaker image acquisition request to the camera 11 (S04), and acquires a speaker image from the camera 11 (S05). The gesture analysis unit 33 generates shooting feature data indicating the feature of the speaker's work based on the speaker's image from the camera 11, and refers to the selected gesture dictionary (gesture dictionary 31B) selected in S02. Next, the gesture analysis unit 33 selects, as selection reference feature data, reference feature data that matches the shooting feature data from among a plurality of types of reference feature data in the selection gesture dictionary, and the plurality of selection gesture dictionaries. Of the types of translation information, the translation information corresponding to the selected reference feature data is output as a translation result to the translation result synthesis unit 34 (S06).

  In response to the notification of completion of translation preparation, the translation result synthesis unit 34 issues a speaker image acquisition request to the camera 11 (S07), and acquires a speaker image from the camera 11 (S08). Next, the translation result synthesizing unit 34 synthesizes the translation result from the gesture analyzing unit 33 with the image of the speaker from the camera 11 to generate a translation result synthesized image, and outputs it to the transmitting unit 12 (S09).

  The transmission unit 12 transmits call data including the translation result synthesis image from the translation result synthesis unit 34 and the voice of the speaker captured by the microphone 13 to the receiver's video phone 1B via the telephone network 2 (S10). ).

  In the telephone network 2, the speech text conversion unit of the speech translation unit 40 converts the speech of the speaker (for example, English speech) included in the call data transmitted from the speaker's video phone 1 </ b> A into text data (text data in English notation). ). The text language translation unit of the speech translation unit 40 translates the language described in the text data into the language specified by the listener (for example, Japanese). The image text synthesis unit of the speech translation unit 40 synthesizes the translation result included in the call data so that the translated text data is displayed as a caption on the end (for example, the lower end) of the display unit 22 of the receiver's videophone 1B. The image is synthesized and transmitted to the receiver's video phone 1B.

  In the videophone 1B of the receiver, the receiving unit 21 receives the call data transmitted from the videophone 1A of the speaker, outputs the translation result composite image included in the call data to the display unit 22, and is included in the call data. The speaker's voice is output to the speaker 23. The display unit 22 displays the translation result composite image from the receiving unit 21. The speaker 23 outputs sound from the receiving unit 21.

  After S10, when the call is continued, that is, when the videophone process is continued, S04 and subsequent steps are executed, and when the call is not continued, the videophone process is terminated.

  As described above, in the videophone 1 according to the first embodiment of the present invention, by translating the gesture taken by the subject (speaker) of the transmission source on the videophone 1A of the speaker, Smooth communication can be achieved without causing misunderstandings.

[Second Embodiment]
In the second embodiment, only the changes from the first embodiment will be described.

  FIG. 3 is a schematic block diagram showing the configuration of the videophone 1 according to the second embodiment of the present invention. In the first embodiment, the gesture translation unit 30 is provided in the video phone 1A of the speaker. In the second embodiment, the gesture translation unit 30 is provided in the video phone 1B of the receiver.

  FIG. 4 is a flowchart for explaining the operation (videophone processing) of the videophone 1 according to the second embodiment of the present invention, using the configuration of FIG.

  First, in the videophone 1B of the receiver, the gesture dictionary selection unit 32 of the gesture translation unit 30 makes a gesture dictionary acquisition request (S01). Here, S01 is executed according to the operation of the receiver when, for example, a call is made from the speaker's video phone 1A to the receiver's video phone 1B. In S01, the gesture dictionary selection unit 32 selects, from the gesture dictionaries 31A to 31D registered in the dictionary storage unit 31, a gesture dictionary (for example, the gesture dictionary 31B) corresponding to the language used by the speaker as the selection gesture dictionary. (S02). The gesture dictionary selection unit 32 notifies the gesture analysis unit 33 and the translation result synthesis unit 34 of the gesture translation unit 30 that translation preparation is complete (S03).

  Now, in the video phone 1A of the speaker, the transmitter 12 calls the receiver's video phone 1B with call data including the image of the speaker captured by the camera 11 and the voice of the speaker captured by the microphone 13. It transmits via the network 2.

  In the telephone network 2, the speech text conversion unit of the speech translation unit 40 converts the speech of the speaker (for example, English speech) included in the call data transmitted from the speaker's video phone 1 </ b> A into text data (text data in English notation). ). The text language translation unit of the speech translation unit 40 translates the language described in the text data into the language specified by the listener (for example, Japanese). The image text synthesizing unit of the speech translating unit 40 displays the translated text data as subtitles on the end of the display unit 22 (for example, the lower end) of the receiver's videophone 1B. The image is synthesized and transmitted to the receiver's video phone 1B.

  In the receiver's videophone 1B, the receiving unit 21 receives the call data transmitted from the speaker's videophone 1A.

  Next, the gesture analysis unit 33 of the gesture translation unit 30 makes a speaker image acquisition request to the reception unit 21 in response to the notification of completion of translation preparation (S04). In response to the speaker image acquisition request from the gesture analysis unit 33, the reception unit 21 outputs the image of the speaker included in the received call data to the gesture analysis unit 33, and the gesture analysis unit 33 receives the image. Obtained from the unit 21 (S05). The gesture analysis unit 33 generates shooting feature data indicating the features of the speaker's work based on the image of the speaker from the reception unit 21, and refers to the selected gesture dictionary (gesture dictionary 31B) selected in S02. . Next, the gesture analysis unit 33 selects, as selection reference feature data, reference feature data that matches the shooting feature data from among a plurality of types of reference feature data in the selection gesture dictionary, and the plurality of selection gesture dictionaries. Of the types of translation information, the translation information corresponding to the selected reference feature data is output as a translation result to the translation result synthesis unit 34 (S06).

  In response to the notification of completion of translation preparation, the translation result synthesis unit 34 issues a speaker image acquisition request to the reception unit 21 (S07). In response to the speaker image acquisition request from the translation result synthesis unit 34, the reception unit 21 outputs the image of the speaker included in the received call data to the translation result synthesis unit 34. The translation result synthesis unit 34 An image is acquired from the receiving unit 21 (S08). Next, the translation result synthesis unit 34 synthesizes the translation result from the gesture analysis unit 33 with the image of the speaker from the reception unit 21 to generate a translation result synthesized image, and outputs it to the display unit 22. The display unit 22 displays the translation result composite image (S21). At the same time, the speaker 23 outputs the voice of the speaker included in the call data.

  After S21, when the call is continued, that is, when the videophone process is continued, S04 and subsequent steps are executed, and when the call is not continued, the videophone process is terminated.

  As described above, in the videophone 1 according to the second embodiment of the present invention, the gesture taken by the subject (speaker) of the transmission source is translated by the receiver's videophone 1B, so that the other party (receiver) can be translated. Smooth communication can be achieved without causing misunderstandings.

[Third Embodiment]
In the third embodiment, only changes from the first or second embodiment will be described.

  FIG. 5 is a schematic block diagram showing the configuration of the gesture translation unit 30 in the videophone 1 according to the third embodiment of the present invention. The gesture translation unit 30 further includes a telephone directory storage unit 35, a determination storage unit 36, and a determination unit 37.

  In the telephone directory storage unit 35, identifiers (for example, network numbers and account names for IP telephones, etc.) used for calls and attribute information related to the language used are registered in association with each user. ing. The attribute information includes at least identification information (such as an IP address) that can identify the address of the user, the nationality of the user, and the location of the user.

  In the determination storage unit 36, attribute information 51 and dictionary information 52 indicating a gesture dictionary to be selected are registered in association with each other. The dictionary information 52 is registered in the determination storage unit 36 for each attribute information 51. FIG. 6 shows an example of the determination storage unit 36 of FIG. As shown in FIG. 6, the attribute information 51 indicates the nationality. Examples of the nationality include, but are not limited to, Japan, the United States, the United Kingdom, and China. For Japan, the United States, the United Kingdom, and China indicated by the attribute information 51 (nationality), the gesture dictionary indicated by the dictionary information 52 is a Japanese gesture dictionary, an English (United States) gesture dictionary, and an English (United Kingdom) gesture dictionary. Suppose that it is a Chinese gesture dictionary.

  When the speaker and the receiver make a call, the determination unit 37 uses the identifier of the speaker used for the call from the attribute information registered in the telephone directory storage unit 35 (here, a telephone number). Attribute information corresponding to is acquired as selection attribute information. At this time, the determination unit 37 searches the attribute information 51 registered in the determination storage unit 36 for attribute information 51 that matches the selected attribute information, and the determination unit 37 is registered in the determination storage unit 36. From the dictionary information 52, the dictionary information 52 corresponding to the searched attribute information 51 is acquired as selected dictionary information.

  As a result, the gesture dictionary selection unit 32 automatically selects the gesture dictionary indicated by the selected dictionary information from the gesture dictionaries 31A to 31D registered in the dictionary storage unit 31 as the selection gesture dictionary.

  FIG. 7 is a flowchart for explaining the operation (videophone processing) of the videophone 1 according to the third embodiment of the present invention, using the configuration of FIGS. 5 and 6.

  First, when a speaker and a receiver make a call, in the speaker's video phone 1A or the receiver's video phone 1B, the determination unit 37 of the gesture translation unit 30 makes a request for acquiring the speaker's nationality ( S31). Here, S31 is executed, for example, when the videophone 1A of the speaker makes a call in response to the operation of the speaker. Alternatively, S31 is executed according to the operation of the receiver when, for example, a call is made from the videophone 1A of the speaker to the videophone 1B of the receiver. The determination unit 37 acquires the nationality (for example, [USA]) corresponding to the telephone number of the speaker as the selected nationality from the nationalities registered in the telephone directory storage unit 35 (S32). Next, the determination unit 37 acquires the dictionary information 52 [English (US) gesture dictionary] corresponding to the selected nationality [USA] as the selected dictionary information from the dictionary information 52 registered in the determination storage unit 36. Then, it is output to the gesture dictionary selection unit 32 (S33).

  Next, the gesture dictionary selection unit 32 of the gesture translation unit 30 makes a gesture dictionary acquisition request according to the selected dictionary information [English (US) gesture dictionary] from the determination unit 37 (S34). In S <b> 34, the gesture dictionary selection unit 32 selects, from the gesture dictionaries 31 </ b> A to 31 </ b> D registered in the dictionary storage unit 31, an English (US) gesture dictionary (for example, the gesture dictionary 31 </ b> B) indicated by the selection dictionary information as the selection gesture dictionary. The selection is made automatically (S35). The gesture dictionary selection unit 32 notifies the gesture analysis unit 33 and the translation result synthesis unit 34 of the gesture translation unit 30 of the completion of translation preparation (S36).

  After S36, S04 and subsequent steps are executed.

  Thus, in the videophone 1 according to the third embodiment of the present invention, when a speaker and a receiver make a call, a gesture dictionary corresponding to the language used by the speaker is automatically selected. Thus, it is possible to save time and effort for the operation of the speaker or receiver.

[Fourth Embodiment]
In the fourth embodiment, only changes from the first to third embodiments will be described.

  FIG. 8 shows an example of a gesture dictionary in the dictionary storage unit 31 of the gesture translation unit 30 in the video phone 1 according to the fourth embodiment of the present invention. Gesture dictionaries 31A to 31D are registered in the dictionary storage unit 31 for each language. In each of the gesture dictionaries 31A to 31D, reference feature data 53 (feature vectors) indicating features of an operation when a user performs a gesture. And translation information 54 indicating translation (meaning) of the gesture are registered in association with a plurality of types. As shown in FIG. 8, the translation information 54 is a character string that represents the meaning of the gesture.

  In this case, when the translation result synthesis unit 34 of the gesture translation unit 30 synthesizes the translation result with the speaker's image, the translation representing a character string in the speaker's image, for example, in the form shown in FIG. The information 54 is synthesized. At this time, it is preferable to synthesize the translation information 54 representing a character string at a position that does not overlap with the caption to be synthesized with the speaker image in the speaker image.

[Fifth Embodiment]
In the fifth embodiment, only changes from the first to third embodiments will be described.

  FIG. 10 shows an example of a gesture dictionary in the dictionary storage unit 31 of the gesture translation unit 30 in the videophone 1 according to the fifth embodiment of the present invention. In the dictionary storage unit 31, gesture dictionaries 31A to 31D are registered for each language and for each user. In each of the gesture dictionaries 31A to 31D, reference feature data 53 indicating the features of the action when the user performs a gesture. (Feature vector) and translation information 55 indicating translation (meaning) of the gesture are registered in association with a plurality of types. The translation information 55 is an avatar image indicating a user image prepared in advance for the user gesture. The user image prepared in advance is, for example, an image file taken by the user in advance with the camera 11.

  In this case, the gesture analysis unit 33 of the gesture translation unit 30 includes the translation information 55 representing a plurality of types of avatar images in the selected gesture dictionary (for example, the gesture dictionary 31B) when generating the shooting feature data (feature vector). Then, the translation information 55 representing the avatar image corresponding to the shooting feature data of the speaker is selected as the translation result. When the translation result synthesizer 34 synthesizes the translation result with the speaker's image, for example, in the form shown in FIG. 11, an image other than the background image of the speaker's image is converted into the speaker's avatar. Replace with translation information 55 representing an image. This avatar image is preferably such that the facial expression of the speaker appears remarkably.

[Sixth Embodiment]
In the sixth embodiment, only changes from the first to fifth embodiments will be described.

  In the video phone 1 of the present invention, the gesture translation unit 30 is provided in the speaker's video phone 1A or the receiver's video phone 1B, but the present invention is not limited to this. Each video phone 1 may be provided with a plurality of gesture translating units 30, and the receiver's video phone 1B may display images from the plurality of speaker's video phones 1A on the display unit 22 in one screen. However, it may be individually displayed on the display unit 22 by switching the screen.

  For example, as shown in FIG. 12, when the first to third utterers and the receiver simultaneously make a videophone call, the display unit 22 of the receiver's videophone 1B displays the first to third translation results. A composite image is displayed. In the first translation result synthesized image, the first translation result, which is the translation result of the gesture of the first speaker, is synthesized with the image of the first speaker. In the second translation result synthesized image, the second translation result that is the translation result of the gesture of the second speaker is synthesized with the image of the second speaker. In the third translation result synthesized image, the third translation result, which is the translation result of the gesture of the third speaker, is synthesized with the image of the third speaker.

  A computer program (hereinafter referred to as a program) that operates on the video phone 1 of the present invention is a program that controls a CPU or the like (a program that causes a computer to function) so as to realize the functions of the above-described embodiments related to the present invention. It is. Information handled by these devices is temporarily accumulated in the RAM in terms of processing, then stored in various ROMs and HDDs, read out by the CPU, and corrected and written as necessary. As a recording medium for storing the program, a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical storage medium (for example, DVD, MO, MD, CD, BD, etc.), a magnetic recording medium (for example, magnetic tape, Any of a flexible disk etc. may be sufficient. Further, by executing the loaded program, not only the functions of the above-described embodiment are realized, but also by co-processing with the operating system or other application programs based on the instructions of the program, The functions of the invention may be realized.

  In addition, when distributing to the market, the program can be stored and distributed in a portable recording medium, or transferred to a server computer connected via a network such as the Internet. In this case, the recording apparatus of the server computer is also included in the present invention. In addition, a part or all of the transmitting station apparatus and the receiving station apparatus in the above-described embodiment may be individually converted into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces LSI is realized by the advancement of semiconductor technology, an integrated circuit according to the technology can be used.

1 ... Video phone,
1A ... Speaker's video phone,
1B ... The receiver's video phone,
2 ... telephone network,
11 ... Camera,
12 ... Transmitter,
13 ... Mike,
21 ... receiving part,
22 ... display part,
23… Speaker,
30 ... Gesture Translation Department,
31 ... Dictionary storage,
31A-31D ... Gesture dictionary,
32 ... Gesture dictionary selection part,
33 ... Gesture analysis section,
34 ... Translation result synthesis unit,
35 ... Phone book storage,
36 ... storage unit for discrimination,
37 ... discriminator,
40 ... speech translation department,
51 ... attribute information,
52… Dictionary information,
53 ... feature data for reference,
54 ... Translation information (character string),
55 ... Translation information (image file)

Claims (5)

A gesture dictionary associating reference feature data indicating the features of the gesture with translation information indicating the translation of the gesture;
Gesture analysis that generates shooting feature data indicating the feature of the subject gesture based on the image of the subject, and selects translation information corresponding to the shooting feature data from the translation information of the gesture dictionary as a translation result And
A translation result synthesis unit that synthesizes the translation result with the image of the subject to generate a translation result synthesized image;
A video phone comprising: A receiving unit for receiving an image of the subject;
A display unit for displaying the translation result synthesized image;
The video phone according to claim 1, further comprising: A transmission unit for transmitting the translation result composite image;
The video phone according to claim 1, further comprising: A dictionary storage unit in which a plurality of gesture dictionaries are registered;
A telephone directory storage unit in which an identifier used for a call and attribute information related to a language used are registered in association with each other;
A storage unit for determination in which selection dictionary information indicating a gesture dictionary to be selected is registered for each attribute information;
From the attribute information registered in the telephone directory storage unit, to obtain attribute information corresponding to an identifier used for a call as selection attribute information, from among the dictionary information registered in the determination storage unit, A determination unit that acquires dictionary information corresponding to the selection attribute information as selection dictionary information;
A gesture dictionary selection unit that selects a gesture dictionary indicated by the selection dictionary information as the gesture dictionary from the plurality of gesture dictionaries registered in the dictionary storage unit;
The video phone according to claim 1, further comprising: Generating feature data at the time of photographing indicating the feature of the gesture of the subject based on the image of the subject;
Referring to the gesture dictionary that associates the reference feature data indicating the feature of the gesture with the translation information indicating the translation of the gesture, the translation information corresponding to the shooting feature data is translated from the translation information of the gesture dictionary A step to select as a result;
Synthesizing the translation result with the image of the subject to generate a translation result synthesized image;
A computer program that causes a computer to execute each step of.

Images