JPH0481088A - Coincident glance type figure picture display image pickup device - Google Patents

Abstract

PURPOSE:To miniaturize an image pickup device itself without lowering presence by moving a camera to a range, where the camera does not interrupt a party figure image displayed on a display device, and displaying the range to be interrupted by the camera corresponding to the setting position of the camera. CONSTITUTION:A CCD supercompact camera having a supercompact camera head part 8 is used, the image of the observer is picked up while arranging the supercompact camera on the front face of a display device 1, and a display face hidden by the supercompact camera is masked by a window called camera window 15. Then, the display position of the camera window 15 is moved corresponding to the setting position of the supercompact camera by a transparent supercompact camera supporting board 9 and by displaying the camera window 15 at a position where the window does not interrupt the figure image of the communicating party, the image is picked up with the aligned glance. Thus, the device is miniaturized, and the presence or the expression can be satisfactorily read.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a line-of-sight matching type person image display/imaging device that captures an image by aligning the line of sight of conference participants in a video conference, video telephone, or the like.

(Prior Art) With the development of image communication technology, video conferencing systems and video telephone systems have been put into practical use that allow conferences and meetings to be held while viewing the face image of the other party in a remote location. Videoconferencing systems and videophone systems require imaging and display methods that provide a natural, realistic feel that does not make users aware of a teleconference, and that can effectively read facial expressions.

A method of capturing images by aligning each other's lines of sight via an image display device such as a CRT is an effective means for realizing a system that is natural and realistic, and that can effectively read facial expressions.

However, to achieve line-of-sight alignment, the camera is
If the camera is installed in the front of a video display device such as an RT, the camera will block the display screen of the display device, so in these current systems, the camera is installed above or beside the video display device. As a result, a facial image viewed diagonally from above or from the side is sent to the communication partner, and because both parties' eyes are facing a video display device such as a CRT, it is not possible to have meetings or meetings with the same line of sight. There wasn't.

In order to solve the above problems, a line-of-sight display/imaging device using a half mirror has been conventionally used as a method for installing a camera in the line of sight of the subject without blocking the display screen of the video display device. Are known.

FIG. 12 is a diagram showing an example of the configuration of a display/imaging device using a half mirror, where 1 is a display device and 2 is a half mirror.
13 is an imaging device, and 4 is an observer.

An observer 4 is observing the display device 1 through the half mirror 2. At the same time, the face of the observer 4 is imaged by the imaging device 3, so the line of sight of the observer 4 is directed toward the imaging device 3, and a face image with a matching line of sight is transmitted to the communication partner.

(Problems to be Solved by the Invention) However, in the conventional line-of-sight type display/imaging device described above, ■ In order to make the imaging device 3 invisible to the observer 4, it is placed in a box to block light from the outside. It must be stored and fixed, making the device large.

■ Observer 4 always views display device 1 through half mirror 2.
As a result, the brightness of the display surface and the amount of light from the imaging device are reduced, resulting in a decrease in the sense of realism.

(2) As mentioned above, this device is large-sized and the brightness of the display surface is reduced, so it is difficult to use it for other purposes, such as for combined use as a television display or computer display.

- This device is quite troublesome due to its configuration in order to capture and display images on multiple screens using multi-windows.

There is a good luck point.

(Objective of the Invention) The invention described in claim (1) of the present invention is directed to increasing the size of the conventional line-of-sight type person image display/imaging device, reducing the sense of realism due to the reduction in display brightness and imaging light amount, etc. This solves the difficulty of converting it into a display device, and provides a natural,
The purpose of the present invention is to provide a line-of-sight matching type person video display/imaging device that gives a sense of realism and can read facial expressions etc. well.

In addition to the above object, the invention as set forth in claims (2) and (3) provides a line-of-sight type human video display/imaging device that can easily capture and display images on multiple screens using multi-windows. With the goal.

(Means for Solving the Problems) In order to solve the above problems and achieve the objects, the invention as set forth in claim (1) provides a video input device or a part of the video input device on the front part of the display screen of the video display device. a device for installing a video input device or a part of the video input device, and a means for detecting the horizontal and vertical positions of the installed video input device or a part of the installation; The apparatus is characterized by comprising means for generating a window on the display screen for a portion hidden by a part of the apparatus, and means for causing the window to follow according to the movement of the video input apparatus or the part of the apparatus. The invention described in claim (2) provides a video input device or a video input device on the front part of the display screen of a video display device that can display one or more images such as moving images and still images received on multiple screens using multi-windows. In the apparatus in which a part of the video input device is installed, means for detecting the horizontal and vertical positions of the installed video input device or the installed part, and a means for detecting the position of the video input device based on the position information obtained by the detection means or means for generating a window on the display screen for a portion hidden by a portion of the device; means for causing the window to follow as the video input device or a portion of the device moves; The invention as set forth in claim (3) further comprises means for setting a window size and display position according to a display position and the number of windows of the received image. and a display position of the window relative to the portion hidden by the video input device or the portion of the device so as to maximize the size of the window of the received image according to the number of windows of the received image. The present invention is characterized by having means for changing the size and display position of the window.

(Function) The invention described in claim (1) of the present invention solves the problems of conventional line-of-sight-dispersed display/imaging devices using half mirrors, such as increase in device size, reduction in display brightness, and imaging light amount. In order to solve the problem of the difficulty of using it together with other display devices, we used a CCD ultra-compact camera with an ultra-compact camera head, and placed the ultra-compact camera in front of the display device so that the viewer could not see it. The display surface that is captured and hidden by the ultra-compact camera is masked by a window called a camera window, and the display position of the camera window is moved depending on the installation location of the ultra-compact camera to a position that does not interfere with the image of the person being communicated with. By displaying a camera window, images are taken with the line of sight aligned.

In addition to the effects of the invention as claimed in claim (1), the inventions described in claims (2) and (3) of the present invention also have the advantage that, in addition to the effects of the invention described in claim (1), the received images are placed in a position that is not obstructed by the camera window as much as possible, depending on the number of received images. Enlarge to display size.

By displaying the image in a reduced size, images with the same line of sight are captured.

(Embodiment) FIGS. 1 to 5 show an embodiment of the invention according to claim (1) of the present invention, and FIG. 1 shows a configuration diagram of the first embodiment. In the figure, ■ indicates a display device such as a CRT, 5-1
is an ultra-compact camera support, 6 is a horizontal camera movement unit and position detection unit, 7 is a vertical camera movement unit and position detection unit, 8
9 is an ultra-small camera head, 9 is an ultra-small camera transparent support plate, IO is a camera connection cable, 11 is a camera control unit (CCU), 12 is a window generation control unit,
13 is a side part on the net side, and 14 is a display screen composition part.

Next, to explain the operation, the human image captured by the ultra-compact camera head section 8 is sent to the CCU via the camera connection cable 10.
II. The person image sent to the communication partner from the CCU II is transferred to the network side section 13, and the image is transferred to the network side section I3.
is sent out to the network.

The ultra-compact camera head section 8 is supported by an ultra-compact camera transparent support plate 9, and this ultra-compact camera transparent support plate 9 serves as a horizontal and vertical moving unit and position detection unit attached to the ultra-compact camera support 5-1. 6 and 7 can move left and right and up and down, respectively. A filter or the like for eliminating reflection of external light onto the display device 1 is attached within the frame of the ultra-small camera support 5-1.

The horizontal and vertical position information detected by the horizontal and vertical movement units and position detection units 6 and 7 is transferred to the window generation control unit 12. This window generation control section 12 controls the ultra-compact camera head section 8 based on the position information.
Camera window 15 (of the size of the display part hidden by
(see Figure 2). The camera window is synthesized with the other person's image by the display screen synthesis section 14 and displayed on the display device l.

The communication partner person image received by the network side section 13 is sent to the display device 1 and displayed. The observer moves the micro-compact camera head section 8 to a position where it does not interfere with the image of the other person displayed on the display device 1, and the micro-compact camera head section 8 is displayed in the camera window 15 at the moved position.

FIG. 2 shows an example of a camera window display device displayed in the display screen composition unit 14 of FIG. 1, and FIG.
(2) shows a front view, and (2) shows a side view.

In the figure, 15 is a camera window, and in this display example, a portion of the camera connection cable 10 attached to the ultra-small camera head section 8 is also displayed as a window.

FIG. 3 shows a support having an arm-shaped structure instead of the camera support 5- shown in the first embodiment (FIG. 1). 5-2 is an arm-type ultra-compact camera support. Here, O is the angle between the ground and the horizontal line, and r is the distance from the support point of the arm to the ultra-compact camera head section 8.

This ultra-compact camera head section 8 is attached to the tip of an arm-type ultra-compact camera support 5-2. The position of the ultra-compact camera head 8 can be detected from the angle O of the arm and the distance r to the ultra-compact camera head 8.

FIG. 4 is a diagram showing the configuration of a second embodiment of the present invention,
Horizontal camera moving unit and position detecting unit 6, (vertical camera moving unit and position detecting unit 7 is one set), ultra-compact camera head unit 8, ultra-compact camera transparent support plate 9, camera connection cable lO, Camera control unit (CCUHI)
This is the case where multiple (3 types) are installed.

This is equipped with an output screen compositing section 16 for compositing the screens from three types of ultra-compact camera head sections 8, and in the first embodiment, basically one person to one person is used; In this embodiment, communication between three parties is realized.

The images of the three observers captured by the three cameras mentioned above are combined into one screen image by the sending screen combining section 16, transferred to the network control section 13, and sent from the network control section 13 to the network. Ru. In this embodiment, since the three observers are imaged by three cameras each, the three observers are always imaged by the camera in front of them, so communication can be performed with their lines of sight always aligned.

FIG. 5 shows an example of the camera window display position displayed by the display screen synthesis section 14 of FIG. 4.

This display example shows a display example of the camera window 15 when there are three observers and images of each are captured using a micro-compact camera.

Next, FIGS. 6 to 6 show claims (2) and (2) of the present invention.
3) An embodiment diagram of the invention according to the description is shown, and FIG. 6 shows a configuration diagram of the third embodiment. In the figure, the output screen composition section 1
1.5 to 15 excluding 6 are in FIG. 1 above.

The components are the same as those shown in FIG. 4, including a received video enlargement/reduction section 17 and a display screen control section 18, each of which is controlled by a rope control section I3.

Next, to explain the operation, the human image captured by the ultra-compact camera head section 8 is sent to the CCU via the camera connection cable lO.
II. From this CCUII, the person image to be sent to the communication partner is transferred to the network control section 13, and sent from the network control section 13 to the network.

Images received from the network are input to the network control unit 13, and the number of received images is ascertained. The received images from the network control section 13 are transferred to the received video enlargement/reduction section 17, and the number of received images is transferred to the display screen control section 18.

The ultra-compact camera head section 8 is supported by an ultra-compact camera transparent support plate 9, and this ultra-compact camera transparent support plate 9 includes a horizontal and vertical movement section/position detection section 6 attached to the ultra-compact camera support body 5, 7, it can be moved left and right and up and down. A filter or the like is attached within the frame of the ultra-small camera support 5 to eliminate reflection of external light onto the display device 1. The position of the ultra-compact camera detected by the horizontal and vertical movement units/position detection units 6 and 7 is transferred to the display screen control unit 18.

Here, a method for determining received image window enlargement/reduction information and received image window position information in the display screen control section 18 will be described below. First, the range in which the camera window is displayed is determined based on the ultra-compact camera position information.

Next, based on the number of received images transferred from the network control unit 13, the received image display position and display size are determined so that the received images can be displayed at the maximum on the display screen excluding the camera window. Finally, the received image display position and enlargement/reduction ratio are transferred to the window generation control section 12 and the received image enlargement/reduction zJX section 17, respectively.

The window generation control unit 12 generates a camera window 15 having a size that hides the ultra-compact camera head unit 8 in accordance with the ultra-compact camera horizontal and vertical position information, and transfers it to the display screen composition unit 14.

Further, the received image transferred from the network control section 13 to the received video enlargement/reduction section 17 is enlarged/reduced to the size instructed by the display screen control section I8, and then transferred to the window generation control section 12. The window generation control section 12 determines a window display position based on the received image display position information instructed by the display screen control section 18, and transfers the received image window to the display screen synthesis section 14. The camera window 15 and the received image window transferred to the display screen composition section 14 are subjected to composition processing, and then sent to the display device 1 for display.

FIG. 7 shows the window generation control section 12 and received video enlargement/reduction section 17 based on instructions from the display screen control section 18 shown in FIG.
Examples of the received image window display position, the received image window display size, and the camera window display position controlled by are shown below.

(a)-1 is a display example when the number of stations is one, that is, one-to-one communication. (a)-2 is a display example when the position of the ultra-compact camera in (a)-1 is changed.

(b)-1 is a display example when the number of stations is 2, that is, three-party communication. (b)-2 is a display example when the position of the ultra-compact camera in (b)-1 is changed. (C) ■ is a display example when the number of stations is 3, that is, 4-party communication. (c)-
2 is an example of a display when the position of the ultra-compact camera in (c)-1 is changed. (d)-1 is a display example when the number of destinations is 4, that is, 5-party communication.

(d)-2 is a display example when the position of the ultra-compact camera in (d)-1 is changed. For a larger number of locations, the display screen control unit I8 determines the optimal size and display position of the received image according to the position of the ultra-compact camera.

FIG. 8 shows a configuration diagram of a fourth embodiment of the present invention. In the figure, 19 is a horizontal and vertical drive control section, and 20 and 21 are horizontal and vertical drive sections, respectively. In the first embodiment, the size and display position of the received image window were changed according to the position of the ultra-compact camera head section 8 and the number of received images, but in this embodiment, the display screen control section 18 changes the size and display position of the received image window. The difference is that, depending on the number, the enlargement/reduction ratio and display position of the received image window that can be displayed at maximum, and the position of the ultra-compact camera within a range that does not interfere with line-of-sight alignment are determined.

Here, only the differences will be explained. The position information of the ultra-compact camera head section 8 determined by the display screen control section 18 is transferred to the horizontal and vertical drive control section 19,
The horizontal and vertical drive sections 20 and 21 are driven to change the position of the ultra-compact camera head section 8.

Also, the display screen control unit 18 calculates the ultra-compact camera 8.
The enlargement/reduction ratio of the received image of the maximum size that can be displayed in the remaining display area that is not obstructed by the position of is transferred to the received image enlargement/reduction unit 17, where it is enlarged/reduced.

FIG. 9 shows an example of the movable range of the ultra-compact camera head, and shows the range within which a person's image can be captured satisfactorily without interfering with line-of-sight coincidence. 22 is a movable range of the camera head.

FIG. 10 shows an example of the received image window display position, the camera window display position, and the enlargement of the received image, which are controlled by the window generation control unit 12 according to instructions from the display screen control unit 18.

FIG. 10(a) is a display example when the number of stations is one, that is, one-to-one communication. A description will be given below according to a specific display example. Since the number of ground points is 1, the display screen control unit 18 determines the position of the micro-compact camera to be, for example, the top center, and the received image magnification rate as the maximum range that can be displayed below the micro-compact camera. The determined position information of the ultra-compact camera is transferred to the horizontal and vertical drive controller 19, and the horizontal and vertical drive controller 2
The ultra-compact camera head section 8 is driven at 0.21. At the same time, the position information of the ultra-compact camera is transmitted to the window generation control unit 1.
Transferred to 2.

Further, the received image enlargement rate is transferred to the received image enlargement/reduction section 17, subjected to enlargement/reduction processing, and then transferred to the window generation control section 12. The camera window and the received image window transferred to the window generation control section 12 are transferred to the display screen synthesis section 14, synthesized, and sent to the display device 1 for display.

FIG. 10(b) is a display example when the number of destinations is two, that is, three-party communication. (c) is a display example when the number of stations is 3, that is, 4-party communication. (d) is a display example when the number of stations is 4, that is, 5-party communication. For ten more points, the display screen control unit 18 determines the optimal raised camera position, the size of the received image, and the display position of the received image.

In the first and second embodiments, the received image window is not displayed in the area where the camera window is displayed, but a display window for documents or other received images may be superimposed and displayed in this area. It is also possible to do so. When necessary, the window can be displayed and observed in a range that is not obstructed by the camera window.

FIG. 11 is a diagram showing an example in which a display window for displaying documents or other received images is displayed in a superimposed manner on the area in which the camera window is displayed. 23 is a received image window, 24 is a dead area, and 25 is an image displayed in the dead area.

(a) shows a display example when the number of ground points is 2. dead area 2
4, a display window for documents or other received images is superimposed and displayed.

(b) shows a display example when the number of ground points is 3 - an example in which the camera window 15 is displayed from the upper right; a display window for documents or other received images is superimposed on the dead area 24; it's shown.

In each of the embodiments of the invention of claims (1) to (3) above, the display screen has been described assuming a human image.
At meetings, documents and the like may be displayed on the screen. According to the present invention, even when a document or the like is displayed, it is possible to capture an image in the same way as a human image.

Even in this case, since the line of sight of the person observing the document displayed on the display device is directed toward the display surface, it is possible to hold a meeting with the same line of sight on the communication partner side.

(Effects of the Invention) As explained above, in the invention described in claims (1) to (3') of the present invention, an ultra-compact camera is used and the camera is installed on an ultra-compact camera support (9, the ultra-compact camera support). Install the camera support in front of the display device, move the camera to a range where it does not interfere with the other person's image displayed on the display device, and move the camera to the area that will be obstructed by the camera depending on the installation position of the camera. Since this is a line-of-sight type person display/imaging device that is displayed in a window, the imaging device itself can be miniaturized.

Furthermore, since the display brightness of the display device and the amount of imaging light of the imaging device do not decrease, the sense of realism does not deteriorate.

In addition, when in use, only a small camera support is installed in front of a normal display device, so the display device can be used for other displays.
It is easy to use it together as f.

There is an advantage.

Furthermore, the inventions described in claims (2) and (3) are convenient for use because the multi-window allows the optimal size and display position of the received image to be displayed even in a large number of locations.

[Brief explanation of the drawing]

1 to 5 show an embodiment of the invention according to claim (1) of the present invention, FIG. 1 is a configuration diagram of the first embodiment, and FIG. 2 is a camera window display of the first embodiment. 3 is an example of an arm-type support instead of the camera support in FIG. 1, FIG. 4 is a configuration diagram of the second embodiment, and FIG. It is a figure which shows the example of a display of the camera window display device of a figure. 6 to 11 show embodiments of the invention according to claims (2) and (3) of the present invention, and FIG.
7 is a diagram showing a display example of the camera window display device in FIG. 6, FIG. 8 is a diagram showing the configuration of the fourth embodiment, and FIG. 9 is a diagram showing a display example of the camera window display device in FIG. 6. FIG. 10 is a diagram showing the movable range of the camera head section. Image window display position, camera window display position,
FIG. 11 is a diagram showing an example of enlarging a received image, and FIG. 11 is a diagram showing an example of superimposed display of a document and an image. FIG. 12 is a diagram showing an example of the configuration of a conventional line-of-sight matching type imaging device. 1...Display device, 2...Half mirror 3...
- Imaging device, 4... Observer, 5-1... Ultra-compact camera support, 5-2... Arm-type ultra-compact camera support, 6... Horizontal camera moving unit and position detection unit,
7... Vertical camera moving unit and position detection unit, 8
・・Ultra small camera head part, 9 ・・Ultra small camera transparent support plate, lO...Camera connection cable, 11...
・Camera control unit (CCU), 12・
Window generation control unit, 13...Network control unit, 14
...display screen composition section, 15...camera window,
16... Sending screen composition section, 17... Received video enlargement/reduction section, 18... Display screen control section, 19...
Horizontal/vertical drive control unit, 20... horizontal drive unit,
21... Vertical direction drive unit, 22... Camera head movable range, 23... Reception image window, 24
... Dead area, 25... Image displayed in the dead area. Patent Applicant Nippon Telegraph and Telephone Corporation Figure Figure Figure Figure 2277 Merah... Reminder Diagram 2 Half-Five End

Claims (3)

[Claims] (1) In a device in which a video input device or a part of the device is installed in front of the display screen of a video display device, detecting the horizontal and vertical positions of the installed video input device or a part of the device. means for generating a window on the display screen for a portion hidden by the video input device or a part of the device based on the position information obtained by the detection device; and the video input device or the device. A line-of-sight matching type person video display/imaging device, comprising: means for causing the window to follow according to the movement of a part of the window. (2) Videos received on multiple screens using multi-windows
In a device in which a video input device or a part of the device is installed in the front part of the display screen of a video display device capable of displaying one or more images such as still images, the installed video input device or a part of the device is means for detecting a position in the horizontal and vertical directions; and means for generating a window on the display screen for a portion hidden by the video input device or a part of the device based on the position information obtained by the detection means. a means for following the window as the video input device or a part of the device is moved; and a means for adjusting the size and display position of the window according to the display position of the window and the number of windows of the received image. What is claimed is: 1. A line-of-sight matching type human image display/imaging device, characterized in that it has a setting means. (3) means for driving a video input device or a part of the device, and a means for driving the video input device or a part of the device so as to maximize the size of the window of the received image according to the number of windows of the received image; 3. The line-of-sight type person video display/imaging device according to claim 2, further comprising means for changing the display position, window size, and display position of the window with respect to the portion hidden by the screen.