JPH06253295A - Space scanning image pickup display device - Google Patents

Abstract

PURPOSE:To allow the display device to be compatible with a common work environment such as an office. CONSTITUTION:An operation display section 100 is operated and a video image to be observed is photographed sequentially. The image pickup position direction data representing a position direction of the photographed video are detected by an image pickup position direction detector. The photographed video is stored in a picture memory corresponding to the image pickup position direction data. An operation display section 101 is operated and the video photographed by the operation display section 101 is displayed on a display space 130. In this case, the display position direction data of the operation section display section 101 are detected by a display position direction detector. A video control device 150 detects image pickup position direction data corresponding to the display position direction data and selects a video in a picture memory corresponding to the detected image pickup position direction data. The selected video is set to the operation display section 101, in which the video is displayed. The other party's video display device displays a direction of the video image in existing and displayed on the other party side onto an operation display section respectively.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a spatial scanning image pickup display device having a spatial image pickup function and an excellent face-to-face interactive function, which is used in a videophone device or the like.

[0002]

2. Description of the Related Art A conventional videophone device comprises a video display unit for displaying a TV image, a video input unit for inputting a TV image, a videophone signal communication control unit, a voice transmission / reception unit, etc. It is a device that interacts with each other while transmitting and receiving images, and a video display unit and a video input unit are fixedly arranged so as to be suitable for capturing and displaying a face image.

[0003]

When such a conventional videophone device is installed in a work environment where a plurality of people coexist, such as an office, and is applied to a remote conference, remote education, remote collaborative work, etc. Has the following human interface problems.

(1) Fixed camera inconvenience When video communication is to be used for transmitting coastal information of objects other than face images such as documents, articles, and scenery, the image input section is fixedly arranged. Therefore, it is very inconvenient to properly photograph an object. (2) Inability to display the image on the side where there is no object in front of the eye Using video communication, the other side sends information about the object such as documents, goods, and scenery that is not in front of me, and displays and observes it In order to see the place I wanted to see, I had to ask the other party to input the video, and I could not see what I wanted to see freely. (3) Grasping the viewpoint In a normal face-to-face dialogue, one person presents topic information (image information), and the other person proceeds while talking while viewing the information. The person presenting the topic information proceeds while confirming that "the other party certainly saw it." Alternatively, proceed while confirming which part the other person pays attention to and interest in. In other words, in an actual face-to-face dialogue, what the other party is looking at is grasped and confirmed by looking ahead of the line of sight of the other party to proceed with the conversation. Through such detection of the line-of-sight directions of each other, smooth dialogue and communication are performed. However, the conventional videophone device cannot see what is ahead of the line of sight of the other party. Even if various image information is provided to the other party, it is not clear if the other person really saw it, nor what part the other person paid attention to. Therefore, in the videophone device, the dialogue does not proceed smoothly as compared with the actual face-to-face dialogue. (4) Small screen size In an actual face-to-face dialogue, a knowledge table or desk is usually prepared, and topic information is spread to that table and the conversation proceeds while comparing various things. There is a large space for gathering topic information. However, in the conventional videophone device, the size of the video display unit is the same as the size of the video display unit of the normal TV reception inspection.
It is not practical to prepare a larger image display unit or a plurality of image display units because the cost and size of the device increase. For this reason, there is a problem in that it is difficult to spread the topic information and proceed with the conversation with only one video display unit.

In view of the above problems, it is an object of the present invention to provide a spatial scanning image pickup display device that meets the requirements of a collaborative work environment such as office equipment and is applicable to a videophone device or the like.

[0006]

In order to achieve the above-mentioned object, a spatial scanning image pickup display device of the present invention is a spatial scanning image pickup display device which scans a space and displays an image. And an image pickup position direction detecting device for detecting an image pickup position and an image pickup direction of an image shot by the image input device, and a first image display device for displaying a sent image. A first terminal device including: an imaging position / direction data memory for accumulating imaging position / direction data relating to an imaging position and an imaging direction detected by an imaging position / direction detecting device of the first terminal device; The image captured by the image input device of the device is made to correspond to the imaged position direction data detected by the imaged position direction detecting device with respect to the image, and the image memory for accumulation and the display position of the image can be freely set. The is possible to kinematic 2
Terminal device including a display position and direction detection device for detecting display position and direction data relating to the display position and display direction of the second video display device and the display position of the second terminal device When the display position / direction data is input from the direction detection device, one of the imaging position / direction data stored in the imaging position / direction data memory is detected that corresponds to the display position / direction data detected by the display position / direction detection device.
A video selection device for selecting a video in the image memory corresponding to the detected imaging position / direction data; a transmission device for transmitting the video selected by the video selection device to the second terminal device and displaying the video on the video display device; The first and second video display devices each have a partner video position display device that displays the direction in which the video displayed on the partner side exists.

[0007]

On the first terminal side, the first operator moves the first terminal device or rotates the first terminal device while fixing the first terminal device, and sequentially captures images to be observed. Imaging position / direction data indicating the position and direction of the captured image is detected by the imaging position / direction detecting device. The video shot is
The data is stored in the image memory in association with the image pickup position / direction data.

On the second terminal side, the second operator rotates the second terminal device while moving or fixing it. At this time, the display position / direction detecting device detects the display position / direction data indicating the position and direction of the image to be displayed by the second terminal device. The image selection device detects the image pickup position / direction data corresponding to the detected display position / direction data, and selects the image in the image memory corresponding to the detected image pickup position / direction data. The selected video is transmitted to the second terminal device by the transmission device and displayed on the second video display device. At this time, the partner video position display device displays the direction in which the video displayed on the partner side exists in the first and second directions.
Are displayed on the respective image display devices.

Therefore, the present invention has the following features. (1) Instead of transmitting the image taken by the video input device to the other party as it is, the data of the detected imaging position and imaging direction and the taken image are temporarily stored in the memory. The other party can freely look around the image in accordance with the display position and the display direction by changing the viewing position from the accumulated images. (2) By displaying the portion displayed on the video display device of the other party on his / her own video display device using a mark such as an arrow or a circle, it is possible to grasp the field of view of the other party.

[0010]

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the spatial scanning image pickup display device of the present invention. The person A uses the operation display unit lOO including the image input unit and the position sensor unit to input an image in the imaging space 120. The range in which the image can be input at one time is determined by the operation / display unit lOO, and the wide imaging space 1
20, the shooting area l21, the shooting area l22, and the shooting area l
23, the image capturing area 124, and the image is input in several times. The captured images 170 captured respectively are sent to and accumulated in the video control device 150. At the same time, the image-capturing position / imaging angle data 160 that has been input as an image is sent to and accumulated in the image control device 150.

On the other hand, the person B uses the operation display unit 101 including the image display unit and the position sensor unit and uses the display space 130.
Then, the image is displayed as if the imaging space 120 were present. The display direction of the operation display unit 10 is sent to the video control device 150 as display position / display angle data 161.
Based on the sent data 161, the video control device 15
In the case of 0, the optimum one is selected from the stored images and sent as a display image 171 to the operation display unit lO1 for video display. The image display is, for example, the display area 131, which is the imaging area 122 captured by the person A,
The imaging space 120 not in front of the person B is displayed on the operation display unit 10 of the person B.

FIG. 2 is a diagram for explaining the function of the video control apparatus of the embodiment shown in FIG. The picked-up image 27 image-inputted on the operation display unit 200 including the image input unit and the position sensor unit
0 is stored in the video control device 210. At the same time, the image pickup position and the image pickup angle at which the image is input are combined with the image as the maximum image position / image pickup angle data 260, and the image control device 2
Accumulate to 10. The captured image 221 is position / angle data 2
31 and the captured image 222 are position / angle data 232.
The captured image 223, the captured image 224, the position / angle data 234, and the captured image 22N, the position / angle data 23N, form a set and are stored in the video control device 210.

The image pickup image 270 and the image pickup position / image pickup angle data 260 are automatically stored at fixed time intervals, or when a fixed movement is in the data from the position sensor section of the operation display section. , It is done by the human being who pushes the button artificially. The display position / display direction of the operation display unit 201 is sent to the video control device 210 as display position / display angle data 261. Position / angle data selection unit 24
In 0 (hereinafter, abbreviated as the data selection unit 240), the optimum one, for example, the position / angle data 234 is found from the accumulated position / angle data 231 to 23N, and it is paired with that data. The captured image 224 is sent as a display image 271 to the operation display unit 201 and displayed as an image.

FIG. 3 is a diagram for specifically explaining the method of selecting a display image in the embodiment of FIG. The direction vector of the display position / angle data is OV. The vector direction of the imaging position / angle data 231 is OVa, the vector direction of the imaging position / angle data 232 is OVb, and the vector direction of the imaging position / angle data 23N is OVn. OV x
The angles about the axes, y-axis, and z-axis are (α,
Similarly, OVa is (αa, βa, γ)
a), OVb is (αb, βb, γb), and OVn is
(Αn, βn, γn).

Here, a method for selecting the optimum image pickup position / angle data from the display position / angle data will be described. The distance between the two vectors is set to di, i is calculated from a to n, and image pickup position / angle data that minimizes the distance is found. The distance di is expressed by the following equation (1).

Di ² = (cos α-cos αi) ² + (cos β-cos βi) ² + (cos β-cos βi) ² (1) The imaging position found as the one at the minimum distance.
The captured image that is paired with the angle data is selected and used as the display image. Here, although the display image is selected using only the angle data, it is obvious that the display image can be selected using both the position data and the angle data.

FIG. 4 is a block diagram showing a spatial scanning image pickup display device in which the embodiment of FIG. 1 is further embodied. The image input by the image input unit 420 of the operation display unit 400 is sent to the image input unit 431 of the image control device 450, processed by the image / position / angle data control unit 430, and the image storage unit 453.
Accumulated in. At the same time, the image sensor position angle data of the image input unit 420 is detected from the position sensor 410, sent from the position sensor control unit 412 to the image / position angle data control unit 455, processed, and stored in the position angle data storage unit 456.

In the video display unit 431 of the operation display unit 401, the display position angle data is detected by the position sensor 411 and sent from the position sensor control unit 412 to the video / position angle data control unit 455. Based on the sent data, the optimum image is selected from the image storage unit 453 by referring to the data stored in the position / angle data storage unit 456. Position sensor 410 for the selected image
And the position angle data of the position sensor 411 are compared with each other, an image (for example, an arrow) indicating the position of the image displayed on the other side is added, and the image is sent to the image display control unit 452 together with the selected image to display the image. The image is displayed on the section 431. The image on the operation display unit 401 changes as the display position / angle changes.

In the operation display section 400, the video input section 420
The image input from the image control unit 450 is sent to the image / position angle data control unit 455 through the image input unit 454 of the image control device 450, and the data of the position sensor 410 and the position sensor 411 is compared to determine the position of the other party. Is added to the video,
It is sent to the video display control unit 451 and displayed on the operation display unit 400. In this way, it becomes possible for each other to select an image in the desired direction while informing the other party of the direction of the image viewed by the other party with respect to the image viewed by the other party.

FIG. 5 is a diagram for specifically explaining a state in which an image is observed according to the concrete example of FIG. The person A is in the space 520, and the imaging space 522 is in front of it.
The person B is in the space 521, and there is nothing in front of him. An image capturing space 522 in the space 520 is image-inputted by a terminal device 540 having an image input unit, an image display unit, and a position sensor unit, and a spatial scanning image capturing display control unit and a communication network 500 (hereinafter abbreviated as an image capturing network 500). Do)
In addition, a plurality of captured images are accumulated. The person A uses the imaging network 50 based on the position sensor data.
The optimum display image 530 stored in 0 and the position display arrow 530 indicating the position of the display image displayed on the person B side are viewed. Similarly for the person B, a position display arrow indicating the position of the optimum display image 551 stored in the imaging network 500 and the position of the display image displayed on the person A side based on the data of the position sensor. 531
It is viewed by the terminal device 541 including the image display unit and the position sensor unit. The other party's position display arrows displayed on the terminal devices 540 and 541 respectively make it possible to understand where the other party is looking, and to display the desired video. The situation is shown in
The display area 523 displayed on the person A side and the display area 524 displayed on the person B side are
Sees different images belonging to the same imaging space 522 while confirming the direction of the line of sight of the other person.

In the above-mentioned embodiment, for the sake of simplicity of explanation, the operation display unit is changed only in the angle at a certain place, but the position of the operation display unit can be changed freely. In this case, the photographer (person A) operates the operation display unit 100.
Move to change the image pickup position and the image pickup direction to take an image. The display person (person B) moves the operation display unit 101 to change the display position and the display direction and display an image. For example, this is a case where the imager uses the operation display unit 101 to introduce a large product from various directions. In this case, it is preferable to select the image as described below.

Letting α, β, γ be the angles in the image direction with respect to the x, y, z axes, P be the current position, and V be the vector in the line-of-sight direction, P = (x, y, z), V = ( cos α, cos β, cos
γ), where the i-th position is Pi and the imaging direction vector is Vi, Pi = (xi, yi, zi), Vi = (cosαi,
cosβi, cosγi).

A set B smaller than a constant distance A between P and Pi
Is selected from the stored images. In the set B, the distance Di between V and Vi is obtained, and the i-th image corresponding to the minimum distance Di is selected and displayed.

[0025]

As has been discussed above, the present invention is compact, has good operability and workability, and has both a large screen presentation and a partner's visual field grasp in a collective collaborative work environment such as an office. Can realize a videophone.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a spatial scanning imaging display device of the present invention.

FIG. 2 is a diagram illustrating functions of the video control device according to the embodiment of FIG.

FIG. 3 is a diagram specifically illustrating a display image selection method in the embodiment of FIG.

FIG. 4 is a block diagram showing a spatial scanning imaging display device which is a more specific version of the embodiment of FIG.

FIG. 5 is a diagram specifically illustrating a state in which an image is observed according to the specific example of FIG.

[Explanation of symbols]

100, 101, 200, 201, 400, 401
Operation display unit 120,522 Imaging space 121,122,123,124 Imaging area 130 Display space 131 Display area 150 Video control device 160 Imaging position / imaging angle data 161 Display position / display angle data 170 Imaging image 171 display Image 210 Video control device 240 Position / angle data selection unit 410, 411 Position sensor 412 Position sensor control unit 420 Video input unit 430, 431 Video display unit 450 Video control device 451, 452 Video display control unit 453 Image storage unit 454 Image input unit 455 Image / position angle data control unit 456 Position angle data storage unit 500 Network 520, 521 Space 522 Imaging space 523 Display area 530, 531 Position display arrow 540, 541 Terminal device 550, 551 display image

Claims (1)

[Claims] 1. A spatial scanning image pickup display device for scanning a space to display an image, wherein an image pickup position capable of freely moving an image pickup position and an image pickup position of an image picked up by the image input device. An imaging position / direction detecting device for detecting an image pickup direction and a first image display device for displaying a transmitted image; and an imaging position / direction detecting device for the first terminal device. The image pickup position / direction data memory that stores the image pickup position / direction data relating to the detected image pickup position and the image pickup direction, and the image picked up by the image input device of the first terminal device are detected by the image pickup position / direction detector with respect to the image. An image memory for accumulating and a second video display device and a second video display device capable of freely moving the video display position in correspondence with the imaging position direction data. When the display position / direction data is input from the second terminal device including the display position / direction detection device that detects the display position / direction data related to the display position and the display direction of the display device and the display position / direction data of the second terminal device. Among the image pickup position / direction data stored in the image pickup position / direction data memory, the one corresponding to the display position / direction data detected by the display position / direction detecting device is detected, and the image memory corresponding to the detected image pickup position / direction data is stored. A video selection device for selecting the video of the above, a transmission device for transmitting the video selected by the video selection device to the second terminal device and displaying it on the second video display device, and a first and second video display device. A spatial scanning imaging display device, comprising: a partner video position display device that displays the direction in which the video displayed on the partner side exists.