JPH11196329A - Superimposing device and display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a superimposed image from an image of an object and its background image in a state that the object exists under a background to be virtual reality. SOLUTION: The superimposing device to be disclosed consists of a video monitor 1 for displaying an input image signal, a video camera 2 for photographing an image displayed on the monitor 1 together with an object 100 arranged in compositing front of the monitor 1 and a video image compositing 3. The device 3 consists of a difference image generating means 13 for finding out the difference between an output image signal and an input image signal from/to the camera 2, a contour extraction means 14 for extracting a black area in the difference signal as a mark image and an image superimposing means 15 for superimposing an image obtained by masking the output image from the camera 2 by the mask image to an image corresponding to the mask image for an image formed from the input image signal and outputting the superposed image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a superimpose apparatus and a video display system, and more particularly, to a superimpose apparatus which cuts out a part of a video image and inserts another image, and a superimpose apparatus. The present invention relates to a video display system used.

[0002]

2. Description of the Related Art Conventionally, as this kind of superimposing technique, a technique using chroma key synthesis is known.
In chroma keying, after creating an image with a specific color as the background, the background color is made transparent, and another image is superimposed on this part to superimpose the first image and another image. The method is general (Television Image Information Engineering Handbook, edited by The Institute of Television Engineers of Japan, pp.704-705).

There is also known a method of eliminating a background using a polarizing filter. In this case, a polarizing filter is attached to a portion of the monitor screen to be erased, and a polarizing filter having a polarizing plane perpendicular to the polarizing filter is attached to a camera, and the monitor screen is photographed. Erase the pasted area (Videodraw: A video interface for collaborative
drawing, John C. Tangand Scott L. Minneman, CHI'90 P
roceedings, 1990 pp.313-320).

[0004]

However, in the above-described conventional superimposition technique using chroma key synthesis, for example, a person who gives an instruction (instructor) points to a certain object and displays an image of a scene in which the instruction is given by the instructor. When the image of the pointer and the background image including the target object are created by chroma-key synthesis, the pointing image is performed in the state where the pointing performance is performed with the screen of a specific color (usually blue) as a background. You need to create an image. In this case, there is a problem that it is difficult to perform the performance because the background is different from the actual situation, and therefore the operation tends to be unnatural. Also,
In the case of synthesis using a polarizing filter, the image is transmitted through the polarizing filter and the image is taken.
There is also a problem that the image quality tends to deteriorate.

The present invention has been made in view of the above circumstances, and enables an actor to be a subject to perform with a natural motion, and furthermore, the screen is not darkened, and the image quality is improved. It is an object of the present invention to provide a superimposing device with little deterioration and a video display system using such a superimposing device.

[0006]

According to a first aspect of the present invention, there is provided a superimposing apparatus comprising: a monitor for displaying an input image signal; and a display screen of the monitor for displaying an input image signal in front of the monitor. Image capturing means for capturing an image of the subject together with the subject placed in the camera, and video combining means for combining an image of the subject captured by the image capturing means into an image based on the input image signal and combining the images. .

According to a second aspect of the present invention, there is provided the superimposing apparatus according to the first aspect, wherein the video synthesizing means calculates a difference between an output image signal of the imaging means and the input image signal. A mask image generating means for extracting a black area in the difference signal as a mask image; an image obtained by masking an output image of the imaging means with the mask image; and an image corresponding to the mask image in the image based on the input image signal And an image superimposing means for superimposing and outputting the images.

According to a third aspect of the present invention, there is provided the superimposing apparatus according to the second aspect, wherein the arithmetic means comprises:
The output image signal of the imaging means, the difference between the input image signal and the image signal corrected according to the image distortion and color tone change in the monitor means and the imaging means, the image superimposing means, An image obtained by masking the output image of the imaging unit with the mask image and an image corresponding to the mask image in an image obtained by correcting the input image signal according to the distortion of the image in the monitor unit and the imaging unit. It is characterized by being superimposed and output.

[0009] The invention described in claim 4 is the first invention.
4. The superimposing apparatus according to item 2 or 3, wherein the monitor means is a video monitor that displays an input image signal on a screen of a display.

[0010] The invention according to claim 5 is based on claim 1,
4. The superimposing apparatus according to item 2 or 3, wherein the monitor means is a video projector that projects an input image signal on a screen.

Further, the video display system according to the invention of claim 6 is a first imaging means for photographing a first subject, and a first monitor means for displaying an image of the first imaging means. Second imaging means for photographing the screen of the first monitoring means together with a second object placed in front thereof; output image signals of the second imaging means; and the first imaging means And a black area in the difference signal is extracted as a mask image, and an image obtained by masking the output image of the second imaging unit with the mask image and the output of the first imaging unit Video synthesizing means for superimposing and outputting an image of an area corresponding to the mask image in an image based on an image signal, and second monitoring means for displaying an output image signal of the video synthesizing means on the side of the first subject With It is characterized by a door.

According to a seventh aspect of the present invention, there is provided the image display system according to the sixth aspect, wherein a third imaging means for photographing a third subject to which the second subject belongs, and And a third monitor for displaying the output image signal of the imaging means on the side of the first subject.

[0013]

According to the structure of the present invention, an input image signal to be used as a background is displayed on a monitor, and the display screen of the monitor is obtained from an image taken together with a subject placed in front of the monitor.
By subtracting an image similar to the image displayed on the monitor created separately, a portion other than the subject becomes a black region. Therefore, a mask image is generated using the black region. Then, using the mask image, an image of the subject is extracted from the image captured by the monitor, and a portion other than the subject is extracted from the separately created image and superimposed by superimposing. A superimposed image can be obtained in a state where the image should be under the background.

Further, in another configuration of the present invention, an image of the situation of the instructed person (student) is displayed on a monitor of the instructor (teacher), and the subject is positioned in front of the image of the monitor. The image is taken, a superimposed image is created by the above-described method, and sent to the instructed person for display. Can be reliably recognized, and the instructor can also confirm the situation of the instructed person.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. The description will be made specifically using an embodiment. First Embodiment FIG. 1 is a block diagram showing an electric configuration of a superimposing apparatus according to a first embodiment of the present invention, FIG. 2 is an explanatory diagram of a method of using the apparatus, and FIG. It is a flowchart which shows the operation | movement of the same apparatus. As shown in FIG. 1, the superimposing apparatus of this example includes a video monitor 1 for displaying an input video signal as an image, a video camera 2 for photographing a subject, and a subject 100 And a video image synthesizing device 3 for generating an output video signal obtained by superimposing the above image. In this example, the video camera 2 is used to capture the subject 100 and the screen of the video monitor 1 placed as a background behind the subject 100.

The video image synthesizing device 3 includes a distortion correction circuit 11, a color tone correction circuit 12, and a difference image generation circuit 1.
3, an outline extraction circuit 14, and an image superposition circuit 15. The distortion correction circuit 11 outputs a signal obtained by performing distortion correction on an image based on distortion information obtained in advance with respect to a video signal input. The color tone correction circuit 12 outputs a signal obtained by performing color tone correction on an image based on color tone change information obtained in advance in response to a video signal input. The difference image generating circuit 13 generates an output signal by subtracting one video signal input from the other video signal input. The contour extraction circuit 14 extracts a black area in the video signal input where no image exists, and outputs mask image data corresponding to the black area. The image superimposing circuit 15 superimposes the output of the unmasked area when one video signal input is masked with the mask image and the output of the area corresponding to the mask image in the other video signal input, and outputs the resultant. .

Next, the operation of the superimposing apparatus of this embodiment will be described with reference to FIGS. First, as a preliminary process, a screen displaying the test pattern as an input signal on the video monitor 1 is photographed by the video camera 2 with the same equipment arrangement and lighting as those actually used. Then, distortion information and color tone change information when the obtained image is compared with the original test pattern are obtained (step S1). When the apparatus is actually used, first, the input video signal is displayed on the video monitor 1 (step S3). Then, in this state, as shown in FIG.
00 is captured by the video camera 2, and the image data is taken into the video image synthesizing device 3 (step S4). The subject 100 is, for example, a hand (finger) of an actor who points to a certain part of the screen of the video monitor 1.

On the other hand, the same input video signal is captured by the video image synthesizing device 3 for one screen (step S5), and the distortion correction circuit 11 corrects the video image by using the distortion information obtained in advance in step S1. (Step S6), and the color tone correction circuit 12 performs color tone correction on the video image subjected to the distortion correction by using the color tone change information obtained in advance in Step S1 (Step S7). Thereafter, the video camera 2 calculated in step S4 by the difference image generation circuit 13
The difference image is generated by subtracting the distortion and color tone correction image obtained in step S7 from the photographed image (step S7).
8). In the difference image, the background portion of the screen of the video monitor 1 in the image captured by the video camera 2 is blackened out,
Since only the subject portion is displayed, the contour extracting circuit 14 extracts the black area of the image and generates a mask image showing the contour of the subject (step S).
9).

Next, the image superimposing circuit 15
The image corresponding to only the subject 100 obtained by masking the background of the video image from the video camera 2 with the mask image from the contour extraction circuit 14 and the input video signal from the distortion correction circuit 11 are distorted. An output video signal is generated by superimposing an image corresponding to only the background obtained by extracting a region corresponding to the mask image from the corrected image (step S10).
The above operation is repeated until there is an end command (step S2).

As described above, according to the configuration of this example, since the image serving as the background is displayed on the screen of the video monitor 1, the instructor can use the hand (finger) as the subject on the background image. Is performed while observing the behavior of the video camera 2, and a superimposition process of fitting the image of the subject from the video camera 2 to the background image obtained by cutting out the subject from the image based on the input video signal at that time is performed. Thus, a superimposed image signal can be generated.

Second Embodiment Next, a second embodiment of the present invention will be described. FIG.
FIG. 7 is an explanatory diagram for explaining a method of using a superimposing apparatus according to a second embodiment of the present invention. The superimposing device of this example has substantially the same configuration as the superimposing device (FIG. 1) of the first embodiment, except that a video projector 4 is provided instead of the video monitor 1. This is different from the first embodiment. The video projector 4 has a large screen, and can project an input video signal onto the screen and display it. Therefore, the instructor who is the subject 200 stands while standing in front of the screen of the video projector 4 and performs an action of pointing a part of the background image on the screen with, for example, a hand (finger). The background image displayed on the projector 4 and the subject 200 are
By photographing with A, a superimposed image similar to that of the first embodiment can be generated.

As described above, according to the configuration of this example, since the image serving as the background is displayed on the screen of the video projector 4, the instructor can use the hand (finger) as the subject on the background image. While observing the behavior of (1), an act of moving this can be performed, and a superimposed image signal similar to that of the first embodiment can be generated. In the second embodiment, since the video projector is used to display the background image, the instructor can perform on a larger background image and generate a superimposed image signal.

Third Embodiment Next, a third embodiment of the present invention will be described. FIG.
FIG. 9 is an explanatory diagram of a configuration of a video display system according to a third embodiment of the present invention. The image display system of this example is shown in FIG.
As shown in the figure, a remote control video camera 21, an instruction screen monitor 22, a pointing and shooting video camera 23, a video image synthesizing device 24, and a pointing screen display monitor 25
, A video camera 26 for photographing a face, and a monitor 27 for displaying a face screen. 300 is a teacher,
Reference numeral 400 denotes a hand (finger) of the teacher 300 as a subject, 500 denotes a student, and 600 denotes a teaching material placed on the student side. On this occasion,
Instruction screen monitor 22 and pointing camera 2
3 and the face video camera 26 are on the teacher side, and the remote control video camera 21 and the pointing screen display monitor 2
5 and the face screen display monitor 27 are on the student side, and the video image synthesizing device 24 and the teacher side and the student side are mutually connected by a communication line (not shown).

The remote control video camera 21 captures a student's situation by remote control from the teacher. The instruction screen monitor 22 displays a background indicating the student's situation obtained by the remote control video camera 21. The pointing and shooting video camera 23 shoots the teacher's hand (finger) as the subject 400 on the background displayed on the instruction screen monitor 22. The video image synthesizing device 24 has a function similar to that of the video image synthesizing device 3 shown in FIG. Generate a superimposed image in which is embedded. The pointing screen display monitor 25 displays the superimposed image generated by the video image synthesizing device 24. The face photographing video camera 26 photographs the face of the teacher 300. The face screen display monitor 27 displays the face of the teacher 300 photographed by the face photographing video camera 26.

Next, the operation of the video display system of this example will be described with reference to FIG. The teacher 300 operates the remote control video camera 21 to remotely control the student 50 by remote control.
By photographing the 0 side, the situation on the student side including the teaching material 600 is photographed and displayed on the screen of the instruction screen monitor 22. Further, the teacher 300 indicates, for example, the teaching material 600 on the screen of the instruction screen monitor 22 with the hand (finger) as the subject 400. This situation is photographed by the pointing and photographing video camera 23, and the image of the subject 400 with the screen of the instruction screen monitor 22 as a background is sent to the video image synthesizing device 24. The video image synthesizing device 24 is a superimposed image in which the image of the subject 400 captured by the pointing and shooting video camera 23 is fitted to the background image that displays the student based on the signal from the remote operation video camera 21. This image is displayed on the pointing screen display monitor 25 on the student side. Therefore, the student can confirm the behavior (instruction state) of the hand (finger) of the teacher 300 with respect to the teaching material 600 by looking at the pointing screen display monitor 25.

The pointing screen display monitor 25
Is in the field of view of the remote-controlled video camera 21, so that the teacher can remotely confirm the situation of the student who is looking at the screen of the pointing screen display monitor 25 by remotely operating the teacher. On the other hand, the face photographing video camera 26
The image of the face of the teacher 300 taken by the teacher is displayed on the face screen display monitor 27 on the student side, so that the student can see the teacher's facial expression on the image on the face screen display monitor 27.

As described above, according to the configuration of this example, the student can use the pointing screen display monitor 25 to display the teaching material 600.
Of the teacher's hand (finger) can be seen, so that learning can be performed while checking the teacher's instruction to the teaching material 600, and the teacher's understanding of the explanation can be ensured. On the other hand, the teacher can see the situation in which the student learns while checking the instruction of the teacher, so that it is easy to grasp the understanding state of the student.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and there are design changes and the like without departing from the gist of the present invention. Even this is included in the present invention. For example, the present invention can be applied not only to the case where a superimposed image is generated by a video image signal composed of a moving image but also to the case where a superimposed image is generated by a still image signal of a digital camera or the like. The display in various monitors may use a cathode ray tube, or
Liquid crystal displays and EL (electro luminescence)
A display may be used.

The superimpose apparatus of the present invention may be entirely or partially of a hardware configuration, and may be of a software configuration. For example, the video image synthesizing device constituting the superimposing device of the present invention may be of an analog processing type or of a digital processing type, and the input video signal may be used as it is according to either of them. Alternatively, it can be handled by converting the input video signal into an analog signal or a digital signal and inputting it, and further converting the output video signal as it is or into an analog signal or a digital signal according to the processing method on the output side.

When the video image synthesizing apparatus is a digital processing system, a CPU (central processing unit), a ROM and
Internal storage such as AM, FDD (flexible disk driver), HDD (hard disk driver), C
It is preferable that the computer is configured by a computer having an external storage device such as a D-ROM driver, an input device such as a keyboard and a mouse, and a display device.
It is preferable that a recording medium, such as an FDD (flexible disk), an HDD (hard disk), or a CD-ROM, record a video image synthesizing program that causes a computer to execute processing for realizing the above-described functions.
The recording medium may be a semiconductor memory or another recording medium.
In such a configuration, the video image synthesizing program
It is read by a computer from a recording medium and controls the operation of the computer. When the video image synthesizing program runs, the computer realizes the functions of the video image synthesizing apparatus described above.

[0031]

As described above, according to the superimposing apparatus of the present invention, the image is synthesized in a state where the subject is under the background which should be the virtual reality.
The superimposition can be performed while capturing the situation of the subject while maintaining the relationship with the background. Therefore, the performance of the subject does not fall into an unnatural state as in the case of the conventional chroma key system using a single color background. In addition, since the input video signal is used as it is for the background to be superimposed, the image quality does not deteriorate. Further, unlike the case where a conventional polarizing filter is used, there is no possibility that the screen becomes dark. Also, according to the video display system of the present invention, the instructed person can see, for example, a situation in which the instructor points to an object on the instructed person side, so that the instructor can surely understand the instruction of the instructor. Become. Further, since the instructor can confirm the situation where the instructed person is watching the instructor's instruction, the instructor can easily grasp the instructed state of the instructed person.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an electrical configuration of a superimposing apparatus according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining a method of using the superimpose apparatus.

FIG. 3 is a flowchart showing an operation processing procedure of the superimposing apparatus.

FIG. 4 is an explanatory diagram for explaining a method of using a superimposing apparatus according to a second embodiment of the present invention.

FIG. 5 is an explanatory diagram of a configuration of a video display system according to a third embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 1 video monitor (monitor means) 2, 2A video camera (imaging means) 3, 24 video image synthesizing device (video synthesizing means) 4 video projector 11 distortion correcting circuit (distortion correcting means) 12 color tone correcting circuit (color tone correcting means) 13 Difference image generation circuit (arithmetic means) 14 Contour extraction circuit (mask image generation means) 15 Image superposition circuit (Image superposition means) 21 Remotely operated video camera (First imaging means) 22 Monitor for instruction screen (First) Monitor means 23 pointing video camera (second imaging means) 25 pointing screen display monitor (second monitoring means) 26 face video camera (third imaging means) 27 face screen display monitor (second 3 monitor means) 100, 200, 400 Subject 300 Teacher 500 Student 600 Teaching material

Claims (7)

[Claims] 1. A monitor for displaying an input image signal, an imager for photographing a display screen of the monitor together with a subject placed in front of the monitor, and an image taken by the imager on the image based on the input image signal. A video synthesizing means for inserting and synthesizing the image of the subject to be synthesized. 2. The image synthesizing means, an arithmetic means for calculating a difference between an output image signal of the imaging means and the input image signal, a mask image generating means for extracting a black area in the difference signal as a mask image, An image superimposing means for superimposing and outputting an image obtained by masking an output image of the imaging means with the mask image and an image corresponding to the mask image in an image based on the input image signal. Item 2. A superimposing apparatus according to Item 1. 3. A difference between an output image signal of the imaging unit and an image signal obtained by correcting the input image signal in accordance with image distortion and a change in color tone between the monitor unit and the imaging unit. Wherein the image superimposing means masks an output image of the imaging means with the mask image, and an image based on a signal obtained by correcting the input image signal according to the distortion of the image in the monitor means and the imaging means. 3. The superimposing apparatus according to claim 2, wherein an image corresponding to the mask image is superimposed and output. 4. The superimposing apparatus according to claim 1, wherein said monitor means is a video monitor for displaying an input image signal on a screen of a display device. 5. The superimposing apparatus according to claim 1, wherein said monitor means is a video projector for projecting an input image signal on a screen. 6. A first imaging means for photographing a first subject, a first monitoring means for displaying an image of the first imaging means, and a screen of the first monitoring means in front of the first imaging means. Second imaging means for photographing together with a second subject placed in the camera; calculating a difference between an output image signal of the second imaging means and an output image signal of the first imaging means; Is extracted as a mask image, and an image obtained by masking the output image of the second imaging unit with the mask image and a region corresponding to the mask image in the image based on the output image signal of the first imaging unit A video display system, comprising: video synthesizing means for superimposing and outputting an image; and second monitor means for displaying an output image signal of the video synthesizing means on the side of the first subject. 7. A third image pickup means for photographing a third object to which the second object belongs, and a third image display means for displaying an output image signal of the third image pickup means on the side of the first object. 7. The video display system according to claim 6, further comprising: a third monitor.