KR20010096556A - 3D imaging equipment and method - Google Patents

Abstract

PURPOSE: A three dimensional image input and output method and apparatus are provided to efficiently reproduce an optical effect generated based on both eyes when a human observes an object. CONSTITUTION: A photographing device controls a vertical direction posture and an up-and-down motion of a camera and photographs various screens generated when a human observes an object. The photographing device and the object are loaded on a turntable. A motor controller(24) controls operations of a photographing device and a turntable. A camera interface device(26) controls a focus distance and a zoom of a camera. The camera interface device(26) inputs image data from the camera. An image output device(30) includes a computer(22), a three dimensional image display section(31), a distance sensor(32), and a three dimensional display interface device.

Description

3D imaging equipment and method

The present invention relates to a stereoscopic image input and output device and a method for enabling bidirectional binocular vision for an object through virtual reality.

The use of electronic commerce, virtual museums, digital encyclopedias and catalogs using digital media such as the Internet and CD-ROMs is increasing. In this case, in order to make the visual access to the object through virtual reality more effective, not only the planar image like the photograph but also the sufficient stereoscopic sense by the binocular stereoscopic view as if the real object is in front of the eyes and the direction in which the user views the object It is necessary to provide bidirectionality such as freely selecting posture, posture, and the like.

In order to achieve this purpose, the three-dimensional shape of an object is mainly restored from image information, and thus, an image pair necessary for binocular stereoscopic image has been obtained. However, three-dimensional shape restoration is one of the important tasks in the field of computer vision, but has various problems in its own way despite long active research.

One method of using real image data rather than a restored image is to use a stereo camera that shows two images taken from two positions left and right with respect to an object to each of the user's left and right eyes to give a visual three-dimensional impression. . This method uses two images obtained from a stereo camera with a fixed structure regardless of the user's visual system and observation conditions, thus bringing discomfort, fatigue and shape distortion to the user, especially for objects close to the eyes. Note that there is a problem, and in particular, it is impractical because a huge amount of data is required to obtain the entire set of image pairs that meet various conditions.

The present invention was devised to solve the above-mentioned problems, paying attention to the importance of the role of both eyes in the human visual system, and efficiently and faithfully the optical effects generated around both eyes when a human observes an object. By reproducing, we propose an efficient and practical three-dimensional input / output method and apparatus that enable a user to realistically binocular stereoscopic vision of an object through a two-way virtual reality without discomfort or fatigue.

Since this method basically uses a real image as it is, it is possible to show a three-dimensional image of an object closer to reality, and in particular, to enhance the reliability of the visual approach to the object required in electronic commerce and the like. However, unlike the above method of using a stereo camera, the present method provides an appropriate stereoscopic image by fully considering the visual system and observation conditions related to the binocular stereogram of the user, thereby minimizing discomfort, fatigue and shape distortion. In particular, it is excellent in practicality because it requires a relatively small amount of data by efficiently acquiring, managing, and outputting an entire set of image pairs suitable for various visual conditions.

1 is a block diagram of a stereoscopic image input apparatus for obtaining image data;

2 is a configuration diagram of a control device for an image data input / output device;

3 is a use example of an image output apparatus;

4 is a simple camera model, used for explanation

5 is an explanatory diagram of the use of the image input device of the present invention;

6 is a flowchart of image data acquisition.

7 shows the structure of the stored image data;

8 is a control explanatory diagram of both eyes corresponding to a change in distance from an object to be observed;

9 is an explanatory diagram illustrating a relationship between binocular control and stored image data;

10 is an explanatory diagram of use of the three-dimensional image output device according to the present invention;

11 is a flowchart of a stereoscopic image output;

12 is an explanatory diagram of selection of an output image pair;

13 is an explanatory diagram of an optical relationship between binocular stereoscopic vision and an output image pair;

* Explanation of symbols for the main parts of the drawings

1. Shooting object 10. Object rotating device

12. Camera 14. Shooting device composed of camera and exercise equipment

16. Turntable 20. Image input / output controller

22. Computer 24. Motor Controller

26. Camera Interface 30. Image Output Device

31. Stereoscopic display device 32. Distance sensor

The present invention relates to a stereoscopic image input and output method and apparatus for binocular stereoscopic vision.

The present invention efficiently reproduces the optical effects that occur around both eyes when a human observes an object, thereby enabling an interactive binocular vision without discomfort or fatigue to a stereoscopic observer through a virtual space. The method according to the present invention comprises: first means for compressing and storing a continuous image sequence for a rotating object obtained by photographing from a single camera in a storage medium as image data; Second means for selecting from the image data stored by the first means an image pair suitable for observing an object in binocular stereoscopic manner; And third means for decompressing the image pair selected by the second means so as to be displayed on each of the left and right eyes of the observer.

EMBODIMENT OF THE INVENTION Hereinafter, one Embodiment of this invention is described with reference to drawings. In addition, the same code | symbol is attached | subjected and used for the common part in each drawing.

The basic concept of the present invention is to store and store various scenes generated when a human observes an object in front of him in the form of a planar image, and to view the object in a virtual space, and the characteristics of the user's visual system, binocular and display. The most suitable image pair is selected from the archive image data according to the relationship between the surfaces, and decompressed to be presented to the left and right of the user. At this time, the viewing direction, posture, and image magnification of the object for both eyes are supported by the user.

1 is a configuration diagram of a stereoscopic image input device for obtaining image data. As shown in this figure, the image input device 10 is composed of a photographing device 14 equipped with one camera 12 capable of adjusting the focal length and zoom, and a turntable 16 carrying the photographing object 1. . The photographing apparatus 14 controls the vertical posture of the camera 12, the shanghai dong, and the like, to photograph various scenes that occur when a human observes an object, and to compress and store them as image data in a continuous image format. .

2 is a configuration diagram of a control device for controlling the image input device 10 and the output device 30. In this figure, the control device controls the focal length and zoom of the camera 12 and the motor controller 24 for controlling the operation of each axis of the imaging device 14 and the turntable 16 connected to the computer 22 and it. And a camera interface 26 or the like that is responsible for inputting image data from the camera 12. The image output device 30 is composed of a computer 22, a stereoscopic image display 31, a distance sensor 32 and a stereoscopic image display interface 33. 3 is a configuration example of the image output device 30 shown in FIG.

(Image data acquisition and storage)

4 shows a model of the simple camera 12 for explaining the shooting control of the camera. The camera's focal length and zoom are controlled so that the object is clearly projected onto its image plane at the desired size. In this case, the relationship of Equation 1 holds.

Where f is the distance from the projection center point c of the lens to the image plane, i.e. the focal length, and d is the distance from c to the object. And Y and y respectively indicate the size of the object and the size of the projected image on its image plane.

5 is a side view of the image input device 10 of the present invention, and FIG. 6 is a flowchart of image data acquisition. Acquisition of image data for the object 1 mounted on the center of the turntable 16 of FIG. 5 is performed by the user in the following order for each object by the user according to the flowchart of FIG.

First, the photographing center point o of the object placed through the rotational center line of the turntable 16 is determined, and the photographing apparatus 14 is operated so that the optical axis of the camera is in a horizontal state with the plane (x axis) of the image input apparatus 10. Point the object at the focal point o. At this time, the position of the projection center point c of the camera lens is set to c0, and the y-axis coordinate value with respect to c0 in the corresponding image input device 10 is set to y0. Next, the turntable 16 is rotated while the camera is fixed to c0, and the focal length f of the lens is adjusted so that even the largest projection image of the object enters the image plane. This value is referred to as f0. In this case, the distance between c0 and the object shooting center o is called L.

Then, after determining the y-axis coordinate value yj (j = 1, 2, ..., J) of point c when shooting the object toward the object shooting center point o, the value of the corresponding c point is cj (j = 1). , 2 ,, ..., J) and this is called the shooting position in yj. For convenience To be From the above configuration, the clockwise rotation angle with respect to the horizontal x axis of the image input device 10 at each photographing position cj. Is given by Equation 2, where yj at each cj is And the like are used for controlling the photographing apparatus 14.

Distance between camera's center of rotation rc and projection center c Locates the camera so that it is negligibly small compared to the distance d between c and the object. Then, the distance dj between each camera position cj and the object shooting center point o becomes as shown in Equation (3).

Based on the projection image size of the camera at the focal length f0 at the c0 position, the camera parameter f is used to compensate for the change in the image size caused by the change of the distance dj from each shooting position cj to the object shooting center point o. It is adjusted as in Equation 4.

here, Is the image magnification compared with the size of the projection image determined by c0. Image magnification For a large value of, the image is enlarged in detail to be projected onto the image plane. For convenience, let κ 1 = 1 here, It is determined by the user under the condition of (i = 1,2, ..., I).

On the basis of the above, acquisition of image data for an object is performed by i = 1 to I, as shown in the flowchart of FIG. This is done by repeating the following. That is, at the shooting position cj of the camera from j = 1 to J, the turntable 16 which divided one rotation equally into N steps is rotated by N steps. The image p '(i, j, n) obtained by compressing the image p (i, j, n = 1,2, ..., N) obtained by photographing at each rotation step is stored in the storage device in the structure shown in FIG. Save it. By this method, a plurality of image data can be efficiently stored in digital storage media in a compressed form.

(Stereo image output)

The importance of the role of binocular vision, in particular, is well known when humans observe objects in front of them. The image of the object is projected onto the retina of both eyes according to the distance, posture, and eye characteristics of the object to both eyes, from which the human being can recognize or observe the object. 8 is a plan view showing the relationship between the binocular and the object occurring when a human observes the object in front of both eyes, the distance T between both eyes and the object such that the projection image of the visible part of the object is at the center of each retina of both eyes. The angle between the eyes of both eyes by rotating each eye according to the change of It shows changing the.

As can be seen from this figure, the angle between the eyes of both eyes Is influenced not only by the distance T between both eyes and the object, but also by the distance between the eyes. Even for the same object at the same distance and at the same distance, there is a small, but small difference in the retina on both eyes, depending on the difference in docl between users. This individual difference is a factor that cannot be ignored for the very sensitive human eye.

9 is an image magnification ratio And the camera at the photographing position cj, the relationship between the N image strings (corresponding to the outer large circle) corresponding to one turntable turntable, the distance T between the two eyes and the object, and the distance docl between the two eyes is shown. From this, the angle between the eyes of both eyes Can be expressed as (Equation 5).

Therefore, the image p '(i, j, nl) before l from the current data pointer n on the image sequence is shown in the left eye, and the image p' (i, j, n + r) after r is shown in the right eye. Corresponds to the image. The values of the image data pointers l and r for the left and right eyes, respectively, can be calculated as shown in Equation (6).

Here, the distance between the two eyes, docl and N are already known, and T corresponding to the distance between both eyes and the object is determined according to the display device used, but here, the distance to the position where the user's eyes and the stereoscopic image should be seen.

That is, in the case of using a stereoscopic screen such as a stereoscopic television as the output device of the stereoscopic image of the present invention, a distance sensor for measuring the distance between the display surface and the user and providing the stereoscopic display device is used. For example, in the stereoscopic television of Fig. 3, information about the distance T between both eyes and the display surface is obtained from a distance sensor that measures the distance from the display surface to the user determined as the position where the stereoscopic image is viewed.

(3D observation in virtual space)

As an example of stereoscopic observation based on the method in the present invention, a system operation for a user watching stereoscopic television as in FIG. 10 will be described. It is assumed that two images respectively shown on the left and right sides are superimposed on the display surface of the stereoscopic television, and each image separated by any function of the stereoscopic television is only shown in the corresponding eyes. The distance information T between both eyes and the display surface is constantly sent from the distance sensor to the image output device.

11 is a flowchart of stereoscopic image output. By using the image magnification i, which is sequentially sent from the user through the user interface such as a mouse, a keyboard, or a remote controller, and the values of j and n, which are image data pointers related to the posture and position of the object, The location is determined as shown at the intersection of two vertical and horizontal dotted lines in FIG. Then, the left and right image data pointers l and r calculated by Equation (6) according to the distance docl between the binocular distance of the current user input in advance and the value T of the distance between the user and the image display surface constantly sent from the distance sensor. 3D image display apparatus after selecting the right and left image pairs among the stored image data by using the values, converting them to the decompressed forms p (i, j, nl) and p (i, j, n + r) Output to

If the user wants to observe the stereoscopic image while continuously changing the posture, direction, etc. of the object, the corresponding image pair is sent to the stereoscopic image display according to the values of the image data pointers j and n which are continuously sent from the user. The image magnification ratio sent from the user when the magnification ratio of the arbitrary image data pointers j and n for the corresponding stereoscopic image is necessary. According to the change of the pointer i, the corresponding image pair is sent to the stereoscopic image display.

13 is a plan view showing the relationship between the binocular and the image when a human is viewing a stereoscopic image in various visual environments. Where da represents the distance, or visual distance, from each eye to the position where the plane is visible (with APPEARANCE REAL IMAGE), and is related to the adjustment of the focal length for each eye. dv is the distance from the binocular to the position where the stereoscopic image occurs (stereoscopic distance). Is directly related to

FIG. 13 (a) shows a case where a human is observing an actual object through both eyes, and it is known that both eyes are autonomously controlled so that the stereoscopic distance dv and the visual distance da always coincide. This is extremely natural for humans. 13 (b) and 13 (c) show a case where a stereoscopic display device is used. Unlike humans seeing a real object due to a mismatch between dv and da, not only a faithful virtual reality can be obtained, It is a major factor in fatigue.

In general, a three-dimensional display device has a function of separating both left and right images so that each is visible only to each corresponding eye, and controlling each visual distance da by moving each separated image from the display surface in the forward and backward directions. This can be classified by differences in the structure of the device affecting the dv distance. A human who sees a stereo image with both eyes through such a device performs stereo matching between the visible images of each angle of the image pair to see a stereoscopic image. And the stereoscopic distance dv is determined by the structure of the device and the relationship between the two images.

In the method proposed by the present invention, a binocular stereoscopic image using a three-dimensional display device always satisfies the condition of dv = da, thereby providing a user with a near-real binocular stereotype without any discomfort, distortion, or fatigue to the object. Provides virtual reality by the city. Hereinafter, specific exemplary embodiments of two representative stereoscopic display apparatuses will be described.

Example 1

As shown in FIG. 13 (b), when the binocular and the display device have a fixed relationship, the stereoscopic distance dv is set to a desired value by appropriately disposing the display surface and the optical device in accordance with the eye distance docl. Then, adjust the lens or the like so that the visual distance da between the eye and the actual visible plane matches the dv. Substituting the values of T and docl with the same value as dv or da into (Equation 6), select the right and left image pairs to display the stereoscopic image at the position of dv = da, and output them to each corresponding display.

Example 2

FIG. 13 (c) shows the situation in the binocular stereoscopic view of a human watching stereoscopic television and the like as in the case of FIG. In this case, since the actual visual plane image is the same as the display surface and the distance information T from the distance sensor measuring the distance between the display surface and the user is equal to the visual distance da, the user's eye distance docl and T (Equation 6) When applied to, an appropriate left and right image pairs can be calculated, and a three-dimensional image is formed on the display surface and satisfies the condition of dv = da.

Even for a moving user in use of the system, it is possible to receive a variable value of T constantly sent from the distance sensor and to correctly observe the stereoscopic image to be viewed at each point in time. The left and right image pointers l (t1) and r (t1) are calculated from Equation 6 for user A of the binocular distance docl (A) at a time point t = t1 at the distance T (t1) from the image plane. At this point, the most suitable left and right image pairs p (i, j, nl (t1)) and p (i, j, n + r (t1)) for the image data pointers i, j, n selected by the user A are It is displayed on the three-dimensional image plane. This image pair shows the stereoscopic image positioned on the display surface to the user A at the point in time, and meets the condition of dv = da so that the object placed on the actual display surface despite the change of distance from the display surface due to the user's movement. As you can see, it provides a more effective virtual reality for objects without discomfort or fatigue.

The present invention regarding image acquisition and its use which enables more realistic object observation in virtual reality has the following effects due to its configuration characteristics.

For virtual reality users, it is difficult to realize the real-time image by using computer vision or graphics technology, which is very difficult to realize and does not require expensive computer vision or graphics technology. In addition to observation, more realistic observations of objects are possible. Therefore, it improves the reliability of visual access to the object, especially required in electronic commerce. This is because the photographed image is used as it is, without being processed, so the subtle texture of the object as well as the defects are realistically shown.

It uses stereoscopic vision by binocular vision, so it provides visual realism that cannot be compared with planar images. Since the system operates in consideration of the individual differences existing in each user's environment and users, the visual distortion of the object is greatly reduced along with the discomfort and fatigue of the use, which was a major obstacle in the conventional binocular stereoscopic vision. The viewing direction, posture, and stereoscopic display magnification of the object with respect to the eyes can be selected through interactive dialogue between the user and the system, thereby enhancing the realism for the user in virtual reality.

Image data acquisition For the user, unlike the method of stereo cameras using two different cameras in image acquisition, one camera is used, so that difficult calibration between two cameras is not necessary. On the other hand, only a very small amount of image data is required when acquiring the entire image pair for various binocular visual conditions. In addition, since a large amount of image data is basically required, a relatively large amount of memory is required. However, the present invention can be sufficiently put to practical use by the use of image data compression technology, which has been widely used in recent years, and the rapidly developing mass storage memory. Particularly, the highly compressible moving picture compression technique is applicable because the method in the present invention uses continuous image trains for the object cluster. In addition to the high speed of the Internet, which is actively progressing in recent years, a wide range of applications of the present invention means that require a large amount of image data is expected.

Finally, this image acquisition method records and stores the captured image data in the form of general image strings rather than a special format called stereo images, so that faithful recording is possible within the range of the object, so it can be used for other applications. High utilization.

Claims (7)

In the stereoscopic image input and output device for binocular stereoscopic, First means for compressing and storing a sequence of images of a rotating object obtained by photographing from a single camera in a storage medium as image data; Second means for selecting an image pair suitable for observing an object from the image data stored by the first means in a binocular stereoscopic manner; And third means for decompressing the image pair selected by the second means to display the left and right eyes of the viewer. In the stereoscopic image input and output method for binocular stereoscopic, Compressing and storing a sequence image of a rotating object obtained by photographing from one camera as a image data in a storage medium; Selecting an image pair suitable for observing an object from the stored image data in binocular stereoscopic manner; And decompressing the selected image pair and displaying the extracted image on the left and right eyes of the viewer. The method of claim 2, wherein the storing step of the image data, Shooting, acquiring and storing various scenes of an object using a target photographing device comprising a camera capable of adjusting focus and zoom, a device for controlling the direction and movement of the camera, and a turntable for loading and rotating the object Stereoscopic image input method characterized in that. The method of claim 3, wherein the acquisition of the image data for the object, For each image magnification κ i from i = 1 to I and by each camera direction C j from j = 1 to J obtained by control of the camera focal length and zoom, the movement of the turntable evenly divided by one revolution in N steps Is rotated N steps to input the image p '(i, j, n) obtained by compressing the image string p (i, j, n = 1,2, ..., N) taken at each rotation step into the storage device. Image input method characterized in that. The method of claim 2, wherein the step of selecting an image pair suitable for observing an object from the stored image data in binocular stereoscopic concept, The image magnification i, which is sequentially sent from the user through a user interface such as a mouse, a keyboard, or a remote controller, the values of j and n, which are image data pointers related to the posture and position of the object, and the user who has been input in advance The left and right image pair p '(i, j, nl calculated using the values of the left and right image data pointers l and r calculated according to the distance between the binocular distance docl and the value T of the distance between the user and the image display surface sent from the distance sensor. ) And p '(i, j, n + r). The method of claim 2, wherein the step of selecting an image pair suitable for observing an object from the stored image data in binocular stereoscopic concept, The image magnification i, which is sequentially sent from the user through a user interface such as a mouse, a keyboard, or a remote controller, the values of j and n, which are image data pointers related to the posture and position of the object, and the user who has been input in advance The left and right image pair p '(i, calculated using the values of the left and right image data pointers l and r, which are calculated according to the distance between the two eyes of docl and the value T of the distance between the eye and the image display surface, which is preset according to the system. j, nl) and p '(i, j, n + r). 3. The method of claim 2, wherein the displaying of the selected image pairs to each eye of the observer. The decompressed forms p (i, j, nl) and p (i, j, n + r) for the selected picture pairs p '(i, j, nl) and p' (i, j, n + r) And outputting each of them to the left and right display surfaces of the stereoscopic image display.