JPH08251562A - Video conversation system - Google Patents

Abstract

PURPOSE: To permit conversation in a natural state wherein users look at each other when the users communicate with each other while viewing mutual images of television telephone, television conference, etc., on monitors. CONSTITUTION: A memory 103 stores the image of the full face of a user 100 (image pickup device sight) and a memory 104 stores the image of a current user (who is using the system) at all times. A movement detection means 105 detects the state of the position shift of the image in the memory 103 from the image in the memory 104, and an image position moving means 106 moves the image position in the memory 103 so as to minimize the position shift and writes the moved image in the memory 103 into a memory 107. A moving detecting means 108 detects moving vectors by units of pixels or areas of the image in the memory 104 as to the image in the memory 107 and a control means 109 makes corrections on the basis of the moving vectors so that the image in the memory 104 is put composite with the image in the memory 107, and outputs the resulting image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video interactive system for communicating while watching the image of the other party through a monitor such as a videophone and a video conference.

[0002]

2. Description of the Related Art A video interactive system for bidirectionally transmitting video and audio transmits and receives video by using an image pickup device that captures an image of itself or the other party and a monitor that displays the image captured by this image pickup device. . Here, when the user has a dialogue while looking at the other party's face through the monitor, if the eyes of the users are matched, a natural dialogue without a feeling of strangeness can be performed.

A conventional image interactive system is disclosed in, for example, Japanese Utility Model Laid-Open No. 62-21687.

A conventional video dialogue system will be described below. FIG. 13 is a block diagram of this conventional video dialogue system. In FIG. 13, 1301 is a light-shielding case, 1302 is an image pickup device for photographing the user, 130
Reference numeral 3 is a monitor showing the other user, 1304 is a half mirror, and 1305 is a user.

The operation of the video dialogue system configured as described above will be described below.

First, the light shielding case 1301 blocks the external light entering the system. User 1305 is monitor 1
The other user reflected on 303 is reflected by the half mirror 1304 for viewing. At this time, since the user's eyes are directed to the half mirror 1304, this half mirror 130
The optical axis of the image pickup device 1302 is aligned with the line of sight of the user, which is separated by 4, and the user is photographed and the image is transmitted to the other user.

[0007]

However, the above-mentioned conventional configuration has a problem that the size of the monitor is limited because the half mirror is used, and the whole system has a large volume.

The present invention solves the above-mentioned conventional problems. It is compact, the size of the monitor can be freely selected, and users can talk in a natural state while looking at the monitor while looking at each other. The purpose is to provide a video dialogue system that can be used.

[0009]

In order to achieve this object, a video interactive system of the present invention is capable of storing an image pickup device for photographing a person looking at a monitor and a front image of the person's face. No. 1 memory, a second memory for storing an image of the imaging device, and a second memory for the image of the first memory.
First motion detecting means for detecting the motion vector of the image in the memory, image position moving means for moving the image position in the first memory so that the scalar amount of the motion vector is minimized, and the first position after the movement. Third storing image of memory 1
Memory, and second motion detecting means for detecting a motion vector of the image of the second memory with respect to the image of the third memory, and controlling the image of the second memory based on this motion vector to control the second Control means for outputting an image obtained by synthesizing the image in the memory and the image in the third memory.

[0010]

According to the present invention, with the above-described structure, the second image for the front-facing user image stored in the first memory can be obtained.
Detects the motion vector of the image of the current user stored in the memory of, and moves the image position of the first memory so that this motion vector becomes the smallest, and approximates it to the image of the second memory. It is stored in the third memory.

Then, the second image for the image in the third memory is
The motion vector of the image in the memory is detected, and the image in the second memory is corrected so that the motion vector becomes k times (0 <k <1). By performing the above processing, when making a call while viewing each other's images on a monitor such as a videophone or a videoconference, it is possible to make a natural call by matching the eyes.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a video interactive system according to the first embodiment of the present invention. In FIG. 1, reference numeral 100 denotes a user, 101 denotes an image pickup apparatus for photographing the user, and 1010 denotes an analog-digital conversion unit (denoted as A / D in the figure) for converting the output of the image pickup apparatus 101 into a digital signal. Yes, 102 is a monitor showing a conversation partner, 103 is a memory for storing an image of the front of the user 100 (as viewed from the imaging device), and 104 is an image of the current user (while using the system). Is a memory for storing an image, and 105 is a memory 1 for an image in the memory 103.
Reference numeral 04 is a motion detecting means for detecting the state of positional deviation of the image, and 106 is an image position moving means for moving the image position of the memory 103 so as to minimize the positional deviation.
107 is a memory for storing the moved image of the memory 103, 108 is a motion detecting means for detecting a motion vector for each pixel of the image of the memory 104 or an area of a certain unit with respect to the image of the memory 107, 109
Is a control means for correcting the image of the memory 104 based on this motion vector so as to be combined with the image of the memory 107 and outputting it.

The operation of the video interactive system of the present embodiment constructed as described above will be described below.
The user 100 is interacting while watching the video of the other party displayed on the monitor 102, and the imaging device 101 photographs the user 100. Analog-digital conversion means 101
0 converts the signal of the imaging device 101 into a digital signal.

Before the system is used, the memory 103 stores the front image of the user (the image viewed from the camera), and the memory 104 stores the image of the image pickup device 101 at any time.
The motion detection means 105 includes the image of the memory 103 and the memory 10.
The image of No. 4 is divided into a plurality of areas, the image of the memory 103 and the image of the memory 104 are compared for each area, the motion vector of the image of the memory 104 with respect to the image of the memory 103 is detected for each area, and they are calculated. The average position is outputted, and the image position moving means 106 makes the memory 10 so that the motion vector outputted from the motion detecting means 105 becomes the minimum.
Move the image of 3. As a result, the image in the memory 103 is approximated to the image in the memory 104. The memory 107 stores this moved image. The motion detection means 108 detects the motion vector of the image of the memory 104 with respect to the image of the memory 107 in pixel units, and the control means 109 corrects the image of the memory 104 so that the motion vector is reduced to a certain range. An image similar to the image of 104 and the image of the memory 107 is output.

A detailed description will be given below with reference to FIGS. 2, 3, 4, 5, and 6. FIG. 2A shows an image (a front image of the user) stored in the memory 103. FIG. 2B shows an image (image of the image pickup apparatus 101) stored in the memory 104, and since the user is looking at the monitor 102, the line of sight is not facing the image pickup apparatus 101. FIG. 3A shows an image in the memory 103 (see FIG.
FIG. 3A is a diagram schematically showing an image (FIG. 2B) in the memory 104 superimposed on each other. The positional deviation of the image in this figure is detected by the motion detecting means 105 as a motion vector.
As a detection method, each image is divided into a plurality of areas (shown in FIG. 3), one area of one image is compared with the other image for each area, and when there is a matched or approximated area, The difference between the positions of these areas is detected as a motion vector, this processing is performed for all areas, and the detected motion vector is output to the image position moving means 106. FIG. 3B shows only the motion vector. The image position moving means 106 averages the motion vectors detected by the motion detecting means 105, and moves the image in the memory 103 so as to minimize the motion vector. In the movement, the image of the memory 103 is written in the memory 107 so that the position of the head address data of the memory 103 is shifted from that position to the position of the average value of the motion vector, and the image position of the memory 107 overlaps the image position of the memory 104. Turn it on.

FIG. 3C is a schematic diagram thereof. FIG. 4 shows the detection of the motion vector for each pixel of the image in the memory 104 with respect to the image in the memory 107 by the motion detecting means 108. The detection method is the same as that of the motion detection means 105, but the size of the divided area is in pixel units. The arrows in FIG. 4 indicate motion vectors, and the number of arrows is omitted for clarity. Here, the motion detecting means 105, the image position moving means 106, and the motion detecting means 108.
Is already generally known in the field of image processing, for example, Technical Report of Television Society Vol.11 No.3, pp43-
48, PPOE'87-12 (May 1987) has a detailed report and can be realized by such a method. Therefore, further detailed description is omitted in this embodiment.

FIG. 5 shows the processing contents of the control means 109, and the arrow a in the figure represents the motion vector of the pixel in the memory 107 for a certain pixel in the memory 104 detected by the motion detecting means 108. . The processing of the control means 109 is performed by multiplying the motion vector a by k (0
By moving the pixel of the memory 104 to the position <k <1), the pixel of the memory 104 is corrected so as to be combined with the pixel of the memory 107, and then output. FIG. 6A is a block diagram of the control means. In the figure, s1 is memory 1
The image data of No. 04 is input to the control unit 109 for each pixel or each area data when the screen is divided into areas of a certain unit. s2 is the address of this data s1 in the memory 104, and s3 is the motion detection means 1
It is a motion vector corresponding to the data s1 detected in 08. Reference numeral 601 denotes a buffer for adjusting the input timing of the data s1, the address s2 and the motion vector s3, and 602 is a write address control for giving a write address based on the motion vector s3 and the address s2 when writing the data s1 in the memory 603 described later. Means and memory 6
Reference numeral 03 is a memory for writing the data s1 at the address given by the write address control means 602 and outputting it as corrected image data when all the data in the memory 104 has been written. Reference numeral 604 denotes the motion vector s3 by k.
It is a multiplier that doubles (0 <k <1). The detailed operation will be described below with reference to FIG. FIG. 6B is a schematic view showing how the data in the screen is moved by the control means 109. In the figure, A1 is the data position (address in the memory 104) of the data s1 in the memory 104, and ka
Is the motion vector s3 multiplied by k in the multiplier 604.

A2 is a position (address in the memory 603) where the data s1 is moved according to the motion vector ka, and this position is an address for writing the data s1 in the memory 603.

Write address control means 602
6A calculates the address indicating the position of A2 in FIG. 6B from the address s2 (A1 in FIG. 6B) and the motion vector s3 (ka in FIG. 6B) and outputs it to the memory 603.
Writes data s1 to this address. This causes the data s1 to move from A1 to A2. FIG. 7 shows how the control means 109 corrects the image in the memory 104. FIG. 7A is an enlarged view inside the thick circle in FIG. 4 before correction, and the arrow indicates the motion vector.
FIG. 7B shows a state in which the motion vector is multiplied by k (0 <k <1) and corrected by the control unit.
In FIG. 7B, the middle wavy line part is an image of the memory 104 and the memory 107 before correction and is shown by the solid line part after correction.

With the above configuration, when making a call while viewing each other's images on a monitor such as in a videophone or a videoconference, it is possible to make a natural call by matching the eyes.

In this embodiment, the motion detecting means 10
5. Regarding the image position moving means 106 and the motion detecting means 108, the Television Society Technical Report Vol.11 No.3, pp43-48, PPO
Although it can be realized by the method reported in E'87-12 (May 1987), it is not limited thereto.

Further, in the present embodiment, the image pickup device is described as outputting an analog video signal, but the present invention is not limited to this. For example, an image pickup device outputting a digital signal is used, and analog-digital conversion means is used. It is also possible that there is no configuration.

In the present embodiment, the output of the video interactive system is a digital signal, but the output is not limited to this, and conversion from digital to analog or image information is performed according to the communication system with the other party. It is obvious that the compression and the like can be performed.

Further, although two motion detecting means are provided in the present embodiment, it is obvious that a configuration in which they are combined is also conceivable.

FIG. 8 is a block diagram of a video interactive system showing a second embodiment of the present invention. In the figure, 100 is a user, 102 is a monitor, 101 is a user 1.
Reference numeral 1010 denotes the image pickup apparatus 1
Reference numeral 103 is an analog-to-digital conversion means (indicated as A / D in the figure) for converting the output of 01 into a digital signal, and 103 is a memory for storing the front image of the user (camera line-of-sight image) before using the system. , 104 to the imaging device 101
105 is a memory for storing the image of
Image of the memory 104 (image pickup device 10
1) is a motion detecting means for detecting a position deviation state, and 106 is an image position moving means for moving the image in the memory 103 so as to minimize the position deviation.
7 is a memory for storing the moved image of the memory 103, 108 is a motion detecting means for detecting a motion vector of the image of the memory 104 with respect to the image of the memory 107, and 109 is k times (0 <K <
This is a control means for correcting the image of the memory 104 so as to be combined with the image of the memory 107 by controlling the image of the memory 104 so as to be 1), and the above is the same as the configuration of the first embodiment. It is something. The difference from the first embodiment is that an 801 selector is provided. The selector 801 judges that the magnitude of the motion vector detected by the motion detecting means 105 is out of the correction range when it is very large, and judges that correction is unnecessary when it is very small, and the image in the memory 104 is passed through. Is output by.

As described above, according to the present embodiment, the correction image and the through image are selected and output by the selector according to the magnitude of the motion vector of the image of the user who is using the system with respect to the front image of the user. Unnatural image output due to excessive correction can be reduced, and when making a call while viewing each other's image on a monitor such as a videophone or a videoconference, it is possible to make a natural call by aligning the eyes.

FIG. 9 is a block diagram of the third embodiment of the present invention. In the figure, 100 is a user of the video interactive system, 102 is a monitor for displaying the video of the other party, 901,
Reference numeral 902 denotes an image pickup device installed on the side surface of the monitor 102 for taking an image of the user 100, and 1010 and 1011 denote analog-to-digital conversion means (A / D and A / D in the figure) for converting outputs of the image pickup devices 901 and 902 into digital signals. Note), 9
04 is analog-digital conversion means 1010, 101
It is an image synthesizing means for synthesizing, by image processing, a video similar to that when the user 100 is photographed from the front, from the video signal of the user 100 converted into a digital signal by 1.

FIG. 10 is a block diagram showing a concrete configuration example of the image synthesizing means shown in FIG. In FIG. 10, reference numeral 1000 denotes a feature extracting means for extracting constituent elements such as eyes, nose, and mouth of a face from the images of the user 100 captured by the image capturing devices 901 and 902, and 1001 denotes a feature extracting means 1000. A three-dimensional position estimating unit 1002 that estimates the three-dimensional position of each part of the face, eye, nose, mouth, etc. of the extracted user 100, and three-dimensional position estimating unit 100.
It is a front image synthesizing means for synthesizing the front image of the face of the user 100 based on the estimation result of 1.

The operation of the video interactive system of the present embodiment having the above configuration will be described below.
The image synthesizing means 904 is generally known in the field of image processing, and is described in, for example, IEICE Technical Research Report HC91-43, p. 9-1
6 or National Technical Report Vol. 40, No. 6, p. There are detailed reports in 106-113, etc. Therefore, detailed description is omitted in this embodiment.

Two images of the user 100 photographed by the image pickup devices 901 and 902 are sent to the feature extraction means 1000. The feature extracting unit 1000 first extracts the features of the face portion from the two images, for example, face constituent parts such as eyes, nose, and mouth. Then, based on this extraction result, the three-dimensional position estimating means 1001 estimates the three-dimensional positions of the face components such as the eyes, nose, mouth, etc., and based on this estimation result, the front image synthesizing means. 1002, user 1
The front image of 00 is synthesized. Then, the synthesized front image of the user 100 is transmitted to the other party.

As described above, according to this embodiment, the front images of the users are combined by using the images captured by the two image pickup devices, so that the images of each other such as a videophone and a video conference can be displayed. When making a call while looking at a monitor, it is possible to make a call in a natural state by aligning the eyes of each other.

Further, as compared with the video interactive system having the configuration using the half mirror shown as the conventional example, in the configuration of the present embodiment, the image synthesizing means 904 can be constructed by electric parts, for example, IC, etc. It can be made smaller and lighter.

In this embodiment, the configuration using two image pickup devices has been described, but the present invention is not limited to this. For example, a configuration using one image pickup device can be considered.
In this case, the entire system configuration is simplified, and the size and weight can be further reduced. In addition, for example, if three or more image pickup devices are used, it is possible to more accurately combine the front images of the user.

Further, in the present embodiment, the image pickup device is described as outputting an analog video signal, but the present invention is not limited to this. For example, an image pickup device outputting a digital signal is used, and the image synthesizing means 904 is a digital interface. It is also conceivable that the configuration receives a video signal.

Further, in the present embodiment, the output of the image synthesizing means 904 is a digital signal, but the output is not limited to this, and conversion from digital to analog or according to the communication system with the other party, or It is obvious that image information can be compressed.

Further, although the present embodiment has been described by using the configuration in which the image pickup device is arranged on both sides of the monitor, the present invention is not limited to this. For example, the image pickup device is arranged above and below the monitor or on a diagonal line of the monitor. A configuration such as is also conceivable.

Further, in the present embodiment, the image synthesizing means is constituted by three means of the feature extracting means 1000, the three-dimensional position estimating means 1001 and the front image synthesizing means 1002, but the invention is not limited to this.

FIG. 11 is a block diagram of the fourth embodiment of the present invention. In the present embodiment, based on the result of detecting the area of the flesh-colored portion in the video signal of the user 100 converted into a digital signal by the analog-digital conversion means (denoted as A / D in the figure) 1010, 1011, 2 An image determination unit 1101 which sends one of the two video signals to the selector 1102 described later and sends the two video signals to the image synthesis unit 904 as they are by another signal output system,
Only a portion provided with a selector 1102 for selecting whether to output the final output image from the image synthesizing means 904 or the image determining means 1101 by a control signal from the image determining means 1101 (C1). Different from the third embodiment.

Therefore, only the parts different from the third embodiment will be described. FIG. 12 is a block diagram showing a specific configuration example of the image determination means shown in FIG. 12
In 1202 and 1203, flesh color detection for detecting a flesh color part from a video imaged by the imaging devices 901 and 902 (i1 is a video imaged by the imaging device 901 and i2 is a video imaged by the imaging device 902) A means 1201 compares the detection results obtained by the skin color detecting means 1202, 1203, and a comparison means for controlling the selector 1204 and the selector 1102 based on the comparison result.
1204 is a control signal from the comparison means 1201
It is a selector that selects and outputs either one of the two images i1 and i2 based on the (c). The skin color detecting means 1202 and 1203 are generally known, and there are detailed reports in prior patents (for example, Japanese Patent Laid-Open No. 6-105323). Therefore, detailed description is omitted in this embodiment.

The operation of the video interactive system of the present embodiment having the above configuration will be described below.

The flesh color detecting means 1202 detects the flesh color portion from the image i1 and counts the number of pixels (the counting result is sa). Similarly, the skin color detecting means 120
3 counts the number of pixels of the flesh-colored portion from the image i2 (the count result is sb).

The reason why the flesh-colored portion is counted here is to detect the area of the face portion of the user 100 in the image. The count results sa and sb are sent to the comparing means 1201, where the ratio of sa and sb is calculated. Then, the comparison means 1201 controls the selector 1204 to select and output either one of the images i1 and i2 according to the value of this ratio. Specifically, thresholds k1 (k1> 1) and k2
When (k2 <1) is set and sa / sb> k1, that is, when the face portion of the user 100 is displayed more in the image i1 than in the image i2, the selector 1204 selects i.
1 is selected and sa / sb <k2, that is, the image i1
When more of the face portion of the user 100 is displayed in the image i2 as compared with, the selector 1204 controls to select and output i2. Similarly, the comparing means 120
1 is in the case of k1 ≦ sa / sb ≦ k2, that is, the image i
When there is no large difference in the areas of the skin-colored portions of 1 and i2 (this can be judged to be the user 100 gazing at the monitor), the selector 1102 outputs the final output to the image synthesizing means 9
04 / sa / sb> k1 or sa / sb <
In the case of k2, that is, when there is a large difference in the areas of the flesh-colored portions between the images i1 and i2 (this can be judged to be that the user 100 faces his / her imaging device rather than the monitor), the selector 1102 makes a final output. Is output as the output of the image determination means 1101. In this way, by selecting the image with the selector according to the result of the skin color detection, for example, the image i
If there is a large difference in the area of the flesh-colored portion between 1 and i2, that is, if it can be determined that the user 100 faces his or her imaging device rather than the monitor, it is not a composite image of the two images, but You can send the captured image with and to the other party as it is, compared to the case where only the composite image is continuously sent
A more realistic video dialogue can be realized.

As described above, according to the present embodiment, the images captured by the two image pickup devices are used to synthesize the front images of the users so that images of each other such as a videophone and a video conference can be displayed. When making a call while looking at a monitor, it is possible to make a call in a natural state by aligning the eyes of each other.

Further, as compared with the video interactive system having the structure using the half mirror shown as the conventional example, in the structure of the present embodiment, the image synthesizing means 904 can be composed of electric parts, for example, IC, etc. It can be made smaller and lighter.

If it is possible to determine that the user 100 faces his / her face toward the image pickup device rather than the monitor, it is not a composite image of the two images, but an image of the one in which the user 100 faces his / her face. The captured image of the device can be directly transmitted to the other party, and a more realistic video dialogue can be realized as compared with the case where only the composite image is constantly transmitted.

In the present embodiment, the configuration using two image pickup devices has been described, but the present invention is not limited to this. For example, a configuration using one image pickup device may be considered.
In this case, the entire system configuration is simplified, and the size and weight can be further reduced. In addition, for example, if three or more image pickup devices are used, it is possible to more accurately combine the front images of the user.

In the present embodiment, the image pickup device is described as outputting an analog video signal, but the present invention is not limited to this. For example, an image pickup device outputting a digital signal is used, and the image synthesizing means 904 is a digital interface. It is also conceivable that the configuration receives a video signal.

Further, in the present embodiment, the output of the image synthesizing means 904 is a digital signal, but the output is not limited to this, and conversion from digital to analog or according to the communication system with the other party, or It is obvious that image information can be compressed.

Further, although the present embodiment has been described by using the configuration in which the image pickup device is arranged on both side surfaces of the monitor, the present invention is not limited to this. For example, the image pickup device is arranged above and below the monitor or on a diagonal line of the monitor. A configuration such as is also conceivable.

In this embodiment, two skin color detecting means are used. However, the present invention is not limited to this, and a structure in which the skin color detecting means is integrated is also conceivable.

[0052]

As described above, according to the present invention, an image pickup apparatus for photographing a person looking at a monitor, a first memory capable of storing an image of the front of the face of this person, and an image of the image pickup apparatus are stored. And a first motion detecting means for detecting the motion vector of the image of the second memory with respect to the image of the first memory, and the first memory for minimizing the scalar amount of this motion vector. Image position moving means for moving the image position of the second memory, a third memory for storing the image of the first memory after the movement, and a motion vector of the image of the second memory with respect to the image of the third memory. And a control means for controlling the image in the second memory based on this motion vector and outputting an image obtained by combining the image in the second memory and the image in the third memory. To be Ri, if you want to call while watching a monitor each other's image, such as a television telephone and videoconferencing, it is possible to call the natural state in accordance with the eyes each other.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a video dialogue system according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram of images in memories 103 and 104 of the video interactive system according to the first embodiment of the present invention.

FIG. 3 is a schematic diagram of the processing contents of the motion detecting means 105 of the video interactive system according to the first embodiment of the present invention.

FIG. 4 is a schematic diagram of processing contents of the motion detecting means 108 of the video interactive system according to the first embodiment of the present invention.

FIG. 5 is a schematic diagram of the processing contents of the control means of the video interactive system according to the first embodiment of the present invention.

FIG. 6A is a block diagram of the control means of the video interactive system according to the first embodiment of the present invention. FIG. 6B is a diagram showing the movement of data in the screen by the control means of the video interactive system according to the first embodiment of the present invention. Schematic diagram showing the situation of

FIG. 7 is a schematic diagram of the processing contents of the control means of the video interactive system according to the first embodiment of the present invention.

FIG. 8 is a configuration diagram of a video interactive system according to a second embodiment of the present invention.

FIG. 9 is a configuration diagram of a video dialogue system according to a third embodiment of the present invention.

FIG. 10 is a block diagram of an image synthesizing means of a video interactive system according to a third embodiment of the present invention.

FIG. 11 is a configuration diagram of a video dialogue system according to a fourth embodiment of the present invention.

FIG. 12 is a block diagram of image determination means of a video interactive system according to a fourth embodiment of the present invention.

FIG. 13 is a block diagram of a conventional video dialogue system.

[Explanation of symbols]

 100 system user 101, 901, 902 imaging device 102 monitor 103, 104, 107 memory 105 motion detection means 106 image position moving means 108 motion detection means 109 control means 801, 1102 selector 904 image synthesizing means 1101 image judging means

Claims (11)

[Claims] 1. A monitor for displaying an image of the other party, an image pickup device for photographing a person talking with the other party while looking at the monitor, and a front image of a face of the person can be stored.
Memory, a second memory for storing an image of the imaging device, a first motion detection unit for detecting a motion vector of the image of the second memory with respect to the image of the first memory, and the first memory. Image position moving means for moving the image position of the first memory based on the motion vector detected by the motion detecting means, and storing the moved image of the first memory moved by the image position moving means. And a second motion detecting means for detecting a motion vector of the image of the third memory with respect to the image of the second memory, and a motion vector detected by the second motion detecting means. A video interactive system provided with control means for controlling the image in the second memory and outputting an image obtained by combining the image in the second memory and the image in the third memory. Beam. 2. A monitor displaying an image of the other party, an imaging device for photographing a person talking with the other party while looking at the monitor, and a front image of a face of the person can be stored.
Memory, a second memory for storing an image of the imaging device, a first motion detection unit for detecting a motion vector of the image of the second memory with respect to the image of the first memory, and the first memory. Image position moving means for moving the image position of the first memory based on the motion vector detected by the motion detecting means, and storing the image of the moved first memory moved by the image position moving means. And a second motion detecting means for detecting a motion vector of the image of the third memory with respect to the image of the second memory, and a motion vector detected by the first motion detecting means. Control means for controlling the image of the second memory to output an image obtained by combining the image of the second memory and the image of the third memory, and the first motion If the value of the motion vector obtained by the detection means is outside the certain range, the through image of the second memory is output, and if the value of the motion vector is within the certain range, the image output from the control means is output. Video dialogue system with selector. 3. A monitor displaying an image of the other party, one or more imaging devices for photographing a person talking with the other party while looking at the monitor, and the person based on the image signal of the person obtained from the imaging device. A video interactive system including image synthesizing means for synthesizing an image similar to that when a front is photographed by image synthesizing processing. 4. A feature of a monitor displaying an image of the other party, one or more imaging devices for photographing a person talking with the other party while looking at the monitor, and an image signal of the person output from the imaging device. An image determination unit that selects one image signal based on the amount, and an image that is similar to an image obtained when the person is photographed from the front based on the image signal of the person that is output from the image pickup apparatus and that is combined by image combining processing. A video interactive system comprising a synthesizing means, an image selected by the image determining means, and a selector for selecting one of the images synthesized by the image synthesizing means. 5. The first motion detecting means and the second motion detecting means divide each input image into each pixel or into each certain area, and respectively divide one image into one area and the other image. 3. The video interactive system according to claim 1, wherein the areas are compared with each other, and if there is a matching or approximate area, the difference in the position of each area is detected and output as a motion vector. 6. The image position moving means averages the motion vectors obtained by the first motion detecting means, and converts the address of the data in the first memory into a position according to the averaged motion vector, 3. The image of the first memory is moved by writing the data of the first memory to the converted address of the third memory.
The described video dialogue system. 7. The control means comprises a buffer for adjusting the timing of the image in the second memory and the motion vector output from the second motion detecting means, and a write address for setting a write address when writing data in the memory. The write address control means includes a control means and a fourth memory capable of writing an image of the second memory output from the buffer, and the write address control means is based on the motion vector output from the second motion detection means. 3. The video interactive system according to claim 1, wherein an address for writing the image of the second memory output from the buffer into the fourth memory is set. 8. The image synthesizing means comprises a feature extracting means for extracting facial components such as eyes, nose and mouth from the image of the person photographed by the image pickup device. The described video dialogue system. 9. The image synthesizing means comprises a three-dimensional position estimating means for estimating a three-dimensional position of each part of the person, particularly eyes, nose, mouth, etc., from the image of the person photographed by the imaging device. The video interactive system according to claim 3 or 4, characterized in that 10. An image synthesizing means, a feature extracting means for extracting facial constituent elements such as eyes, nose, mouth, etc., from an image of a person photographed by an imaging device, and the person extracted by the feature extracting means. Eyes, nose,
A three-dimensional position estimating means for estimating a three-dimensional position of each part such as a mouth, and a front image synthesizing means for synthesizing a front image of the person based on an estimation result by the three-dimensional position estimating means. The video dialogue system according to claim 3 or 4. 11. The image determining means includes a skin color detecting means for detecting a skin color portion of each image, a comparing means for comparing the sizes of the skin color portions of the images detected by the skin color detecting means, and a comparing means of the comparing means. It is provided with a selector controlled by a comparison result, and an image from each image pickup device is input to the selector, and an image from the image pickup device having the most skin color part is selected by the selector based on the result of the comparison means. The video dialogue system according to claim 4, characterized in that