JP2007251778A - Image input-output apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image input-output apparatus used for a television telephone, a television conference system or the like to display an image in which eyes of interlocutors are reconciled always and perfectly, to a partner apparatus. <P>SOLUTION: An imaging display apparatus 20 is provided with a liquid crystal display apparatus 21, an image sensor 24 set up behind its back face, and a LED 23. The liquid crystal display apparatus 21 switches an image display and a formation of a LCD transmitting portion 22 to become a pinhole in a period of approximately 1/30 seconds and images the image of a caller by means of the image sensor 24 through the LCD transmitting portion 22. The LCD transmitting portion 22 moves the LCD transmitting portion 22 according to a position of eyes of a partner caller. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video input / output device that can be used in a video phone, a video conference system, and the like.

A so-called video phone that displays a call partner on the display screen of the telephone and makes a call includes a display device such as an LCD that displays the other party together with a call device that makes a call with the other party, and an imaging device that images the caller.
In videophones currently in use, the image pickup device is often provided at the top of the display device, and the caller who speaks while looking at the display device on which the other party is shown is picked up from above, and the picked-up image Are transmitted to the other party, the faces projected on the display screens are slightly downwards, and if you talk while looking at the display image, the eyes will not see each other, resulting in an unnatural image.
Therefore, an image input / output device has been developed in which an imaging means is provided in the display screen, the face of the caller who is looking at the display image is imaged from the surface, and the conventional one with eyes downward is improved (patent) Reference 1).

FIG. 7 is a cross-sectional view of the video input / output unit of the video input / output device described in Patent Document 1.
The video input / output device 100 includes a liquid crystal panel 110 and an imaging panel 120. The liquid crystal panel 110 includes a translucent first substrate 111 and a second substrate 112, and a plurality of TFT transistors arranged in a matrix on the surface of the first substrate 111 facing the second substrate 112 side. A driving element array 113 is formed, and a counter electrode 114 is formed on the surface of the second substrate 112 facing the first substrate 111 side. A liquid crystal layer 115 is sandwiched between the first substrate 111 and the second substrate 112, and a pixel is formed in the liquid crystal layer between each pixel driving element and the counter electrode. In addition, the imaging panel 120 includes a translucent third substrate 121, and an imaging element array 122 including light receiving elements is formed on a surface opposite to the liquid crystal panel 110. Further, a light guide member 123 is provided on the front surface of the image sensor array 122, and a light incident angle limiting member 124 for regulating the incident angle of light imaged on the image sensor array 122 is formed on the front surface of the light guide member 123. The liquid crystal panel 110 and the imaging panel 120 are integrated by sticking with the adhesive layer 130. Further, the pixel drive element array 113 of the liquid crystal panel 110 and the image pickup element array 122 of the image pickup panel 120 are connected to the control circuit 150 by connection wirings 140, respectively, and image pickup and display are controlled. As described above, the image input / output device 100 has a configuration in which the imaging direction and the display direction are opposite to each other. When the image input / output device 100 is used for a video display device such as a videophone, the caller can When a call is made while looking at the face of the other party displayed on the display screen, the line of sight of both faces toward the front, so that the call can be made with a natural appearance (see Patent Document 1).

If the video of the caller captured by the video input / output device 100 having a configuration in which the imaging direction and the display direction are opposite to each other is used for a videophone, the direction of the line of sight is the previous videophone. It is considered that they do not become obliquely downwards that occurred in, and are directed in the same direction.
However, the image input / output device described in Patent Document 1 is basically a system that captures and captures an image with a compound eye, and regulates the incident angle of light imaged on the image sensor array and guides the light guide plate. Since the light that is incident at an angle is limited so as not to reach the image sensor array, the captured image is an image captured from infinity perpendicular to the display / imaging screen. For this reason, only the front area having the same size as the screen can be imaged. For example, the entire upper body cannot be imaged even if the screen is separated from the screen. Although the lines of sight roughly match, an image that completely matches is not obtained, and it is equivalent to the half mirror method only by taking an image from the focal point. Therefore, it was far from a sense of realism in which conversation was carried out with actual eyes, and was not satisfactory.
JP-A-8-102924

  The present invention has been made in view of such a situation, and an object of the present invention is to provide an image input / output display device used in a videophone, a videoconferencing system, etc., in which an image in which the lines of sight of the interlocutors always coincide. It is to be able to display on the other party's device.

The invention of claim 1 is a video input / output device comprising means for transmitting and receiving video, means for displaying the received video, and means for imaging through a translucent area formed on the display surface of the display means. Means for detecting the eye position of the person image on the display screen, means for forming the translucent area at the eye position of the display screen based on the detected eye position, and And a means for alternately switching between a display means and a means for forming the light-transmitting region.
According to a second aspect of the present invention, in the video input / output device according to the first aspect, the display means is a liquid crystal display means, and the translucent area forming means includes a liquid crystal display pixel corresponding to the detected eye position. Control is performed so as to be translucent and make other pixel regions opaque.
According to a third aspect of the present invention, in the video input / output device according to the second aspect, the liquid crystal display device includes backlight means and means for alternately turning on or off the backlight, and the backlight is turned off. The translucent region forming means is actuated.

  According to the present invention, in a video input / output device used in a videophone, a video conference system, or the like, a video in which the lines of sight of the talkers always match can be displayed on the talker's device.

An outline of a video input / output device according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram schematically showing a cross-sectional structure of a central portion of a video input / output device according to an embodiment of the present invention. In the figure, the video input / output device 1 includes a main body device 10 and an imaging display device 20 installed in the device 10. The main unit 10 incorporates a control device 11 for controlling the main functions of the video input / output device, a microphone 12 for inputting the voice of the caller, and a speaker 13 for outputting the voice of the other party, and a keyboard not shown in the figure. An input device such as a panel is provided. Note that the microphone 12 and the speaker 13 are not built-in, but may be an external portable handset.

  The imaging display device 20 includes, for example, an active matrix drive type liquid crystal display device 21. Each pixel of the liquid crystal display device 21 has position information (address). By designating an address from the control device 11, for example, an image driving unit composed of a thin film transistor (TFT) is controlled to form an LCD transmission unit 22 that transmits only the pixel at the designated address. At the rear of the liquid crystal display device 21, a backlight 23 such as an LED is provided as a light source. Further, an imaging device such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor 24 is provided in the vicinity of the center of the rear portion of the liquid crystal display device 21, and an LCD transmission portion having a designated address. The subject located in front of the imaging display device 20 can be imaged by the camera system using 22 as a pinhole.

  FIG. 2 is a cross-sectional configuration diagram of the liquid crystal display device 21. In the drawing, the liquid crystal display device 21 includes a transparent first substrate 211 and a second substrate 212 made of glass, plastic, or the like. An RGB color filter 213 is provided on the inner portion of the first substrate 211, and a common transparent electrode 214 is provided on the inner portion thereof. A transparent electrode 215 for each pixel is provided on the inner portion of the second substrate 212, and a light distribution film 216 is provided on the inner portion thereof. A liquid crystal layer 217 is provided between the first substrate 211 and the second substrate 212 including these components. The liquid crystal display device 21 is configured to be sandwiched. The common electrode 214, the pixel electrode 215, the TFT attached to each pixel (not shown), and the LED 23 that is the light source of the backlight are connected to the control device 11.

When the video input / output device 1 receives the voice and video information of the other party, the video input / output device 1 outputs the voice information from the speaker 13 and displays the video information on the liquid crystal display device 21. The device 11 turns the backlight 23 on and off at a predetermined cycle, and when the backlight 23 is turned off, the LCD 11 does not transmit light, and is incorporated into a part of the LED backlight on the back side of the liquid crystal substrate. The LCD portion corresponding to the image sensor is in a light transmissive state, and light incident through the light transmissive portion is imaged, that is, detected by the image sensor, converted into an electric signal, and recorded in the memory. When the backlight is turned on, the light transmission control of the LCD is performed according to the image information, and an image is displayed.
Here, by setting the repetition cycle of the backlight on / off to, for example, 1/30 seconds, the caller can perform display and imaging in parallel without making the caller aware of this on / off at all. it can.

  The imaging display device 20 detects the position of the other party's eye on the screen of the liquid crystal display device 21, dynamically changes the LCD transmission unit 22 according to the position, and passes through the LCD transmission unit 22. The caller is imaged. Thus, when the caller speaks by looking at the other party image, the LCD transmission unit 22 is imaged at the eye position of the other party image. Can display an image in line of sight with the other party. For this purpose, it is desirable that the image sensor is large even if it does not coincide with the size of the LCD display unit. Thereby, even if the eye position on the screen of the caller moves, it is possible to take an image. In this way, by detecting the position of the eyes of the person displayed on the screen and capturing an image at that position and exchanging images with each other, an image with the same line of sight as when facing each other and speaking is matched. Can be sent to the other party.

  FIG. 3 is a block diagram illustrating a hardware configuration of the video input / output device. In the figure, components having the same reference numerals as those in FIG. 1 operate in the same manner as in FIG.

  In FIG. 3, the control device 11 is a DSP (Digital Signal Processor), a ROM that records a startup program of the control device 11, a RAM that temporarily records information such as the position of the caller's eyes, and audio and video information. It consists of transmission / reception means. A CODEC (COmpression / DECompression) 14 is a compression / decompression unit that compresses and decompresses audio information and video information, and a keyboard 15 is an input device configured on the front surface of the main body device 10. Is an interface with the video input / output device of the other party in order to transmit and receive audio and video signals through a communication line such as a LAN or broadband line, and performs D / A conversion. The memory 17 is, for example, a hard disk type external recording device that records received audio information, video information, and the like as necessary.

  FIG. 4 is a diagram for explaining the relationship between the display of the other party and the screen change at the time of imaging of the other party. FIG. 4A shows the screen when displaying the other party. ) Shows the screen of the other party's eye position a at the time of display of the other party's caller, and FIG. 4C shows the screen showing the pinhole 22 corresponding to the eye position a when imaging the caller. Show.

3 and 4, when the caller inputs a telephone number from the keyboard 15 to call the other party and the other party answers the phone, voice and video are returned from the other party. The control device 11 controls the LED 23 and the pixel driver (TFT) of the liquid crystal display device 21 to display the other party in color on the liquid crystal display device 21 (FIG. 4A). The caller exchanges a conversation with the microphone 12 and the speaker 13 while looking at the other party's eyes a displayed on the screen (FIG. 4B). When this display is performed, the position (or region) a of the eyes (which may be left or right) of the display image is detected by means known per se or known means (for example, see Japanese Patent Application Laid-Open No. 11-250267). The control device 11 acquires the detected position as position information (address) of the display pixel and stores it in the RAM. Next, the control device 11 turns off the backlight LED 23 and controls the pixel driving unit of the liquid crystal display device 21, and controls the pixel driving unit based on the position information stored in the RAM, and transmits only the pixel at this address. Form in part 22. Then, the CCD or CMOS sensor 24 takes in external light through the transmission unit 22 and images the caller (FIG. 4C). This display and imaging are alternately switched periodically.
The captured image of the caller is transmitted to the other video input / output device through the interface 16 and displayed on the screen of the liquid crystal display device.

  FIG. 5 is a diagram illustrating the relationship between the display period and the imaging period. In the figure, when the LED 23 is on, the display period starts and the partner image is displayed on the liquid crystal display device 21. During this time, the image sensor 24 stands by, and when the LED backlight 23 is off, the image sensor 24 takes an image. As described above, the display and imaging periods are repeated at a period of 1/30 seconds, for example.

  As described above, according to the present embodiment, since the image pickup position is the position of the displayed partner's eye, images in which the line of sight completely matches between the caller and the other party's caller can be displayed. Therefore, when a conversation is made on a video phone, it is possible to present a sense of realism in which conversations are actually made face to face.

The video input / output device according to the embodiment of the present invention is not only used as a one-to-one video phone, but is also used for a video conference video input / output device when a conference is held between a plurality of bases such as a company head office and a branch office. Can also be used. Below, the utilization form is demonstrated.
Here, in order to simplify the explanation, a case will be described in which a meeting is held by three persons, that is, one person (participant A) and two other persons (participant B and participant C).

  FIG. 6 is a diagram showing the relationship between attendee display and imaging when a video conference is held between three locations. FIG. 6A is a display screen in the display period of the liquid crystal display device 21 and has a two-screen configuration in which participant B is displayed on the left side and participant C is displayed on the right side. FIG. 6B shows an imaging position b of an image to be transmitted to the participant A of the participant A, and FIG. 6C shows an imaging position of an image to be transmitted to the participant C of the participant A. c.

Now, when the screen of FIG. 6 (A) is displayed, if participant A exchanges opinions while watching participant B, participant A sees the image of participant B's image on the screen of FIG. 6 (B). The pixel portion corresponding to the position is the translucent part 22, and an image is taken through the translucent part 22. Therefore, the imaged image of the participant A is transmitted and displayed to the image input / output device of the participant B under the control of the control device 11 as an image in which the eyes of the participant A and the eyes of the participant B are completely matched. The In that case, since the image transmitted to the participant C is captured by the eye part of the image of the participant C, the line of sight of the participant A in the captured image does not match the participant C. The person C can easily understand that the participant A is not speaking to himself / herself. When participant A exchanges opinions while watching participant C, participant A's video is similarly a video of participant C as a video in which participant A's eyes and participant C's eyes completely coincide. It is sent to the input / output device and displayed.
Thus, when a large number of people hold a video conference, it is possible to clearly recognize who the speaker is speaking to.

It is a figure which shows roughly the structure of the center part cross section of the video input / output device which concerns on embodiment of this invention. It is a cross-sectional block diagram of one pixel part of a liquid crystal display device. It is a block diagram which shows the hardware constitutions of the video input / output apparatus which concerns on embodiment of this invention. It is a figure explaining the relationship of the screen change at the time of an other party's caller's display, and the said caller's imaging. It is a figure which shows the relationship between a display period and an imaging period. It is a figure which shows the relationship between the display of an attendant at the time of having a video conference between several bases, and imaging. It is sectional drawing of the video input / output part of the conventional video input / output device.

Explanation of symbols

  1. Video input / output device, 10. Main unit, 11. Control device, 12. Microphone, 13. Speaker, 14. Compression / decompressor, 15. Keyboard, 16. Network interface, 17. · Memory · 20 · · Imaging display device · 21 · · Liquid crystal display device · 22 · · LCD transmissive part · 23 · · LED backlight · 24 · · CMOS sensor.

Claims (3)

A video input / output device comprising: means for transmitting and receiving video; means for displaying received video; and means for imaging through a translucent area formed on a display surface of the display means,
Means for detecting the position of the eyes of the human figure on the display screen;
Means for forming the translucent region at the position of the eye on the display screen based on the detected eye position;
Means for alternately switching the display means of the received image on the display screen and the means for forming the light-transmitting region;
A video input / output device comprising: The video input / output device according to claim 1.
The display means is a liquid crystal display means, and the translucent area forming means controls the liquid crystal display pixels corresponding to the detected eye position to be translucent and the other pixel areas to be opaque. A video input / output device characterized by performing. The video input / output device according to claim 2.
The liquid crystal display device includes a backlight unit and a unit that alternately turns on or off the backlight, and operates the translucent region forming unit when the backlight is turned off.

Images