JP2004040525A - Device and method for transmitting video signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the face of a talker from being displayed in a video telephone, or the like. <P>SOLUTION: An identifier attached to the face of the talker and for detecting the position and direction of the talker and a video camera 14 for imaging the talker are provided. A first circuit 11 for discriminating the position, size and direction of the face of the talker from image data of the identifier among video signals outputted from the video camera 14 and a second circuit 11 for forming image data of a face of a character whose size and direction are equal to those of the face of the talker discriminated by the first circuit 11 are provided. A third circuit 11 for substituting the face image data of the character formed by the second circuit 11 for the image data of the face of the talker is provided, and a video signal outputted from the third circuit 11 is transmitted. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a video signal transmitting device and a video signal transmitting method.
[0002]
[Prior art]
As communication and its environment have evolved, communication has changed from using only audio to capturing video. A so-called videophone is one of them. According to this videophone, it is possible to make a call while looking at the face of the other party shown on the display.
[0003]
Also, in the case of an IP telephone or an Internet telephone, if a broadband line such as ADSL, CATV, or FTTH is used, a videophone can be realized, and both parties can talk while looking at the other party's face.
[0004]
[Problems to be solved by the invention]
However, there is a case where the user does not necessarily want to show his / her own face even in the communication that captures the video. For example,
(1) I want to use an Internet tutor, but it's embarrassing to show my face.
(2) I want to use the convenience of the Internet to communicate widely with people in the unknown world, but it is not easy to suddenly know my face.
(3) I want to create a simulated self and enjoy the Internet community casually.
And so on.
[0005]
The present invention seeks to provide a communication device that can deal with these cases.
[0006]
[Means for Solving the Problems]
In the present invention, for example,
An identifier attached to the face of the sender to detect the position and orientation of the face of the sender;
A video camera for imaging the sender,
A first circuit for determining the position, size, and orientation of the face of the sender from the image data of the identifier in the video signal output from the video camera;
A second circuit for forming image data of a face of a character having a size and orientation equal to the size and orientation of the sender's face determined by the first circuit;
According to the position of the face of the sender determined by the first circuit, image data of the face of the sender is formed by the second circuit from the video signal output from the video camera. A third circuit for replacing image data of the character's face,
A video signal transmitting apparatus configured to transmit a video signal output from the third circuit.
Therefore, an image in which the face of the sender is replaced with the face of the character is displayed on the display of the other party.
[0007]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 shows an example in which the present invention is applied to a video telephone 10 for an IP telephone. This telephone 10 includes a control circuit 11, a transmitter 12, a receiver 13, and various operation keys (operation switches) 16. Have. In this case, although not shown, the control circuit 11 has various hardware and software for implementing a videophone, and includes, for example, a microcomputer, an encoder circuit and a decoder circuit, a network controller, and the like. The control circuit 11 controls the entire operation of the telephone 10 and executes an encoding process (data compression) and a decoding process (data decompression process) of the audio signal and the video signal. The control circuit 11 is connected to a network 20 through a network controller.
[0008]
When transmitting a call without video, an analog audio signal from the transmitter 12 is supplied to the control circuit 11 and subjected to A / D conversion and encoding, and then packetized. Will be sent to At the time of reception, a packet sent through the network 20 is supplied to the control circuit 11 and subjected to decoding processing and D / A conversion to take out the original analog audio signal. This audio signal is supplied to the receiver 13. You.
[0009]
Further, a video camera 14 and a display 15 are connected to the control circuit 11 for transmitting and receiving images. During a call with video, the sender is imaged by the video camera 14, and the video signal is supplied to the control circuit 11, where the video signal is subjected to A / D conversion and encoding, and the encoded video signal and the encoding process are performed. The converted audio signal is packetized, and the packet is sent out to the network 20. Also, a packet sent from the other party via the network 20 is supplied to the control circuit 11 where the packet is decoded and D / A converted, so that the original audio signal and video signal are taken out. 13 and the display 15.
[0010]
In this case, as shown in FIG. 2A, for example, the display on the screen 15S of the display 15 is a picture-in-picture, and the face of the other party is displayed in a large size on the main screen, and the self-image captured by the video camera 14 is displayed on the small screen. , That is, his / her face transmitted to the other party is displayed in a small size. When a predetermined key operation is performed, for example, as shown in FIG. 2B, the image of the main screen is replaced with the image of the child screen, and the face of oneself is displayed large and the face of the other party is displayed small.
[0011]
In order to satisfy the above requirements (1) to (3), the present invention replaces the face of the sender captured by the video camera 14 with the face of the character by animation and sends it to the other party. is there. In the following description, the videophone mode in which the face of the sender captured by the video camera 14 is replaced with the face of the character by animation and transmitted to the other party will be referred to as “character mode”.
[0012]
In order to realize this character mode, the videophone 10 is further configured as follows. That is, the microcomputer constituting the control circuit 11 is provided with, for example, a routine 100 shown in FIG. 3 as a part of a program executed by the microcomputer. As will be described in detail later, this routine 100 is for replacing the face of the sender captured by the video camera 14 with the face of the character by animation, and is executed, for example, at a rate of 15 times / 1 second. You. Note that, in FIG. 3, the routine 100 shows only portions related to the present invention.
[0013]
Further, for example, glasses 17 as shown in FIG. 4 are prepared. The glasses 17 are for detecting the three-dimensional position and orientation of the face of the sender. For this reason, for example, LEDs (17A to 17C) are provided as identifiers above the front of the center, below the right front, and below the front left of the lens frame of the glasses 17, and the identifiers are provided in the middle of the left vine of the glasses 17, for example. An LED (17D) is provided.
[0014]
In this case, when the transmitter faces the front with the glasses 17, LEDs (17A to 17C) are located in the same vertical plane orthogonal to the front axis, and in the same horizontal plane orthogonal to the vertical plane. It is preferable to provide the LEDs (17A to 17D) such that the LEDs (17A, 17D) are located at the same position. The LEDs (17A to 17D) are supplied with an operating voltage from the control circuit 11, for example, to emit light.
[0015]
Therefore, when the glasses 17 are imaged by the video camera 14, their positions can be detected from the light emission of the LEDs (17A to 17D) by performing predetermined image processing. The height H of the LED (17A) with respect to the LED (17B, 17C) on the screen, the distance W between the LED (17B) and the LED (17C), the depth direction (front-back direction) from the LED (17A) to the LED (17D). The distance D can be determined. The LEDs (17A to 17C) define a plane including them.
[0016]
Since the actual values H, W, and D of the glasses 17 are known, the distances of the glasses 17 to the video camera 14 are calculated based on the values H, W, and D, and the values H, W, and D in the imaging result of the glasses 17. The direction centering on the front-back direction, the left-right direction, and the up-down direction can be obtained. Further, the positions of the glasses 17 in the vertical and horizontal directions with respect to the video camera 14 can be obtained from the positions of the LEDs (17A) in the imaging screen.
[0017]
Therefore, when the speaker wears the glasses 17, it is possible to obtain the three-dimensional position and orientation of the face of the sender with respect to the video camera 14 by detecting the positions of the LEDs (17A to 17D). it can. When the character mode is used, the caller wears the glasses 17 to make a call.
[0018]
In such a configuration, for example, when the telephone 10 is set to the videophone mode by operating a predetermined key during a call, the microcomputer processing of the control circuit 11 starts from step 101 of the routine 100, and then proceeds to step 102. For example, as shown in FIG. 5A, one frame of the video signal output from the video camera 14 is taken into the control circuit 11 as image data, and in the subsequent step 103, the videophone mode is set to the normal videophone mode. Or character mode is determined.
[0019]
If the character mode has been set, the process proceeds from step 103 to step 111. In this step 111, all the LEDs (17A to 17D) are included in one frame captured in step 102. Is checked. When all the LEDs (17A to 17D) are included, the process proceeds from step 111 to step 112. In this step 112, the positions and the intervals H, W, By obtaining D, the distance from the video camera 14 to the sender, the position and size of the sender's face in the imaging screen, and the orientation of the sender's face are detected.
[0020]
Subsequently, in step 113, as shown in FIG. 5B, image data of the face of the character is formed by animation, and the size and orientation of the face are equal to those of the sender according to the data obtained in step 112. Is done. In this case, the image data of the face of the character is prepared by preparing in advance the image data of the face when the character is facing the front, and performing arithmetic processing on the image data in accordance with the data obtained in step 112. Can be formed. Further, the image of the face of the character can be a three-dimensional image.
[0021]
Then, in step 114, according to the information on the position of the face of the sender obtained in step 112, as shown in FIG. Is replaced with the image data of the face of the character generated in step 113. Thereafter, the image data processed in step 114 is subjected to encoding processing (data compression), the image data resulting from this encoding is transmitted to the network 20 in step 122, and the routine 100 ends in step 123.
[0022]
Therefore, when the character mode is set, the face of the sender imaged by the video camera 14 is replaced with the face of the character, and the replaced image of the sender is displayed on the other party's television. Will be sent to the phone.
[0023]
At this time, the routine 100 is executed, for example, at a rate of 15 times / 1 second (15 frames / 1 second), and the position and orientation of the face of the character sent to the other party are determined by the face of the sender. Since it moves according to the position and orientation, an image of the sender with a mask of the character's face is displayed as a moving image on the other party's videophone. At this time, as shown in FIGS. 2A and 2B, a predetermined key is operated to switch the magnitude relationship between the images of the sender and the receiver in the picture-in-picture, so that the sender can change the state of the face of the character. Can be confirmed, that is, the state of the mask can be confirmed.
[0024]
On the other hand, if it is determined in step 111 that any one of the LEDs (17A to 17D) is not included in one frame captured in step 102, the process proceeds from step 111 to step 119. Dummy image data indicating that the speaker's face is out of the imaging range of the video camera 14 is formed. Thereafter, the process proceeds to step 121, where the dummy data is encoded and transmitted to the network 20. . Therefore, even when the character mode is set, if the position of the face of the sender cannot be specified, the image of the sender is not sent to the other party's videophone.
[0025]
If it is determined in step 103 that the normal videophone mode has been set, the process proceeds from step 103 to step 121, where the image data captured at this time (by step 102) is encoded. It is processed and sent out to the network 20. Therefore, when the normal videophone mode is set, the image of the sender captured by the video camera 14 is sent to the other party's videophone as it is.
[0026]
Thus, according to the videophone 10, when the character mode is set, the face of the sender imaged by the video camera 14 is replaced with the face of the character and transmitted to the other party. In other words, a video of the sender wearing the mask of the character's face is displayed, so that it is possible to remove mental barriers such as embarrassing or having trouble with putting out one's own face.
[0027]
In addition, since the communication is performed with a mask attached, a new entertainment property is added even for a normal call with a friend, for example. Further, by preparing image data of several types of different character faces in the control circuit 11 and selecting and using any one of them, the entertainment property can be further enhanced.
[0028]
In the above description, the transmitter 12 and the receiver 13 can be provided on the glasses 17. Further, the control circuit 11 may be a personal computer, and the display 15 may be a television receiver or the like. Further, when the LEDs (17A to 17D) are provided on the glasses 17, the power supply thereof can be provided on the glasses 17 as a button battery or the like.
[0029]
Alternatively, image data of several types of character faces may be prepared in a server such as a manufacturer and downloaded to the telephone 10 for use. Further, in the above description, the present invention is applied to an IP telephone, but the present invention can also be applied to a mobile telephone as long as it has a video telephone function.
[0030]
[List of abbreviations used in this specification]
A / D: Analog to Digital
ADSL: Asymmetric Digital Subscriber Line
CATV: CAble Television
D / A: Digital to Analog
FTTH: Fiber To The Home
IP: Internet Protocol
LED: Light Emitting Diode
[0031]
【The invention's effect】
According to the present invention, since the face of the sender imaged by the video camera is replaced with the face of the character and transmitted to the other party, it is embarrassing to put out one's own face, and mental barriers such as being troubled are removed. be able to. In addition, it becomes a communication with a mask, so to speak, and a new entertainment property is added.
[Brief description of the drawings]
FIG. 1 is a system diagram illustrating one embodiment of the present invention.
FIG. 2 is a diagram of a display screen for explaining the present invention.
FIG. 3 is a flowchart illustrating one embodiment of the present invention.
FIG. 4 is a perspective view showing one embodiment of a part of the present invention.
FIG. 5 is a diagram of a display screen for explaining the present invention.
[Explanation of symbols]
11 control circuit, 12 transmitter, 13 receiver, 14 video camera, 15 display, 16 operation switch, 17 glasses, 17A to 17D LED, 20 network

Claims (6)

An identifier attached to the face of the sender to detect the position and orientation of the face of the sender;
A video camera for imaging the sender,
A first circuit for determining the position, size, and orientation of the face of the sender from the image data of the identifier in the video signal output from the video camera;
A second circuit for forming image data of a face of a character having a size and orientation equal to the size and orientation of the sender's face determined by the first circuit;
According to the position of the face of the sender determined by the first circuit, image data of the face of the sender is formed by the second circuit from the video signal output from the video camera. A third circuit for replacing image data of the character's face,
A video signal transmitting device configured to transmit a video signal output from the third circuit. The video signal transmitting device according to claim 1,
An encoder circuit for encoding a video signal output from the third circuit,
A video signal transmitting device which encodes the video signal output from the third circuit by the encoder circuit and then transmits the encoded video signal. The video signal transmitting device according to claim 1 or 2,
An apparatus for transmitting a video signal, wherein the identifier is a plurality of LEDs provided on glasses worn by the sender. The sender wearing the identifier for detecting the position and orientation of the face is imaged by a video camera,
Of the video signals output from the video camera, determine the position and orientation of the face of the sender from the image data of the identifier,
Image data of the face of the character is formed according to the information of the determination result,
In the video signal output from the video camera, the image data of the face of the sender is replaced with image data of the face of the character,
A video signal transmission method for transmitting the video signal after the replacement. The method for transmitting a video signal according to claim 4,
A method of transmitting a video signal, wherein the video signal after the replacement is further encoded and then transmitted. In the method for transmitting a video signal according to claim 4 or 5,
A method of transmitting a video signal, wherein the identifier is a plurality of LEDs provided on glasses worn by the sender.

Images