KR100893381B1 - Methods generating real-time stereo images - Google Patents

Abstract

The present invention relates to a method for generating a stereoscopic image in real time according to a change in the viewing direction of an object having a three-dimensional spatial coordinates, more specifically, the object is placed in the center of the virtual sphere and the observer (ie, the camera) The vertical observation position in which the observer moved vertically from the surface of the sphere of the data of the observer's observation position that changes in real time under the condition that the object is observed while moving along the surface of the sphere Is obtained by the position coordinates in which the observer moves in the vertical direction, and the horizontal observation position in which the observer moves in the horizontal direction on the surface of the sphere is rotated and the object is acquired in the spatial coordinates of the rotated object, leaving the observer intact. Principle of binocular disparity based on observer's position coordinate and spatial coordinate of object Relates to a method of photographing an object in the observation position by the collinearity condition equation to each produce a right image and left image of which projected onto a two-dimensional, and using the produced right image and left image generating a stereoscopic image of the object in real time.

In the method for generating a real-time three-dimensional image of the present invention in a method of generating a three-dimensional image in real time viewed from the target in any direction, (S10) any one point of the object is located at the virtual sphere 3 Obtaining dimensional spatial coordinates; (S20) An external expression element (that is, a photographing position) of a camera photographing the object is acquired in real time with the center of the sphere as the projection center at an arbitrary position of the virtual sphere surface, wherein the camera is vertical to the sphere among the external expression elements. The position moved in the direction (i.e., the sphere's XZ plane or YZ plane) is acquired by the camera's position coordinates, and the position in which the camera moves in the sphere's horizontal direction (i.e., the sphere's XY plane) rotates the object around its Z axis. Acquiring the spatial coordinates of the changed object; (S30) separating the position coordinates of the camera moved in the vertical direction of the sphere into left and right coordinates by the principle of binocular disparity; (S40) real-time production of the left image and the right image of the object by substituting the spatial coordinates of the object and the left coordinates of the camera, the spatial coordinates of the object and the right coordinate of the camera, respectively, in a collinear conditional expression; It is done by

Object, stereoscopic image, shooting position, left image, right image, collinear conditional expression

Description

Real-time stereoscopic image generation method {METHODS GENERATING REAL-TIME STEREO IMAGES}

The present invention relates to a method for generating a stereoscopic image in real time according to a change in the viewing direction of an object having a three-dimensional spatial coordinates, more specifically, the object is placed in the center of the virtual sphere and the observer (ie, the camera) The vertical observation position in which the observer moved vertically from the surface of the sphere of the data of the observer's observation position that changes in real time under the condition that the object is observed while moving along the surface of the sphere Is obtained by the position coordinates in which the observer moves in the vertical direction, and the horizontal observation position in which the observer moves in the horizontal direction on the surface of the sphere is rotated and the object is acquired in the spatial coordinates of the rotated object, leaving the observer intact. Principle of binocular disparity based on observer's position coordinate and spatial coordinate of object Relates to a method of photographing an object in the observation position by the collinearity condition equation to each produce a right image and left image of which projected onto a two-dimensional, and using the produced right image and left image generating a stereoscopic image of the object in real time.

The shape that we actually see the object in reality is three-dimensional, but when the object is viewed on the screen through a TV, a movie, or a computer, it is usually two-dimensional (or 2.5-dimensional with perspective) due to the flatness of the screen. Recently, however, attempts have been actively made to view an object in three-dimensional (3D) mode even through a screen of a flat (2D) surface.

In general, three-dimensional images representing three-dimensional images depend on the principle of stereo vision through two eyes, and the parallax of two eyes, ie, the parallax due to the distance between two eyes that are about 65mm apart, is the most important factor of stereoscopic feeling. can do. That is, when the left and right eyes of the human body each see a 2D image associated with each other, when these two images are delivered to the brain through the retina, the brain fuses with each other to reproduce the depth and reality of the original 3D image.

As described above, in order to view an object in 3D on a 2D screen, the left image of the 2D viewed from the left eye and the right image of the 2D viewed from the right eye are simultaneously displayed on the screen according to the principle of binocular disparity. The eye of the observer should be deceived through the use of polarization, interlace, and anaglyph.

The invention relates to a method and apparatus for displaying a left image and a right image of an object simultaneously on a screen and displaying the same as a 3D stereoscopic image. 78051 'stereoscopic image display apparatus and stereoscopic image display method', Patent Publication No. 2007-2503 'stereoscopic image display device' and the like have been published.

These publications focus on how realistically a three-dimensional image of an object can be created, provided that the left and right images of the object are already secured. Of course, this is important, but it is also important how to obtain the left and right images of the object that is the basis for the generation of the three-dimensional image.

The present invention relates to a method of generating a stereoscopic image of an object viewed from a viewer in a desired direction in real time, and requires a left image and a right image of the object viewed from all directions.

One of the methods of acquiring the left image and the right image of the object is a method of directly photographing the object from all directions with two cameras spaced a certain distance apart. This method makes it easy to acquire the left and right images, but it is cumbersome for the object to exist in real life and the camera has to shoot the object directly. It needs to be large, and complex calculation process is required to bring up the left and right images according to the observer's changing direction.

According to the present invention, in order to acquire a left image and a right image of an object in real time to generate a stereoscopic image of the object viewed from an arbitrary direction, the real object of the camera and the object are necessary to acquire the left image and the right image of the object. Only the spatial coordinates of the object and the virtual camera (more precisely, the internal and external expression elements of the camera) are needed, and the left and right images of the object are generated in real time according to the observer's movement (ie, observation direction) by the collinear condition equation. The object is to provide a three-dimensional image is generated, a three-dimensional image of the real-time three-dimensional image generation method that can be generated more accurately and quickly.

Real-time stereoscopic image generation method of the present invention for achieving the above object

In the method for generating a three-dimensional image of the object viewed from any direction in real time,

(S10) acquiring three-dimensional spatial coordinates of the object at which one point of the object is located at the center of a virtual sphere;

(S20) Acquire an external expression element (that is, a photographing position) of a camera photographing the object in real time with the center of the sphere as the projection center at an arbitrary position of the virtual sphere,

The position in which the camera is moved in the vertical direction of the sphere (ie, the sphere's XZ plane or YZ plane) among the external expression elements is obtained by the position coordinates of the camera, and the position in which the camera moves in the horizontal direction of the sphere (ie, the sphere's XY plane). The method may include: obtaining the object as spatial coordinates of the object by rotating the object about a Z axis;

(S30) separating the position coordinates of the camera moved in the vertical direction of the sphere into left and right coordinates by the principle of binocular disparity;

(S40) real-time production of the left image and the right image of the object by substituting the spatial coordinates of the object and the left coordinates of the camera, the spatial coordinates of the object and the right coordinate of the camera, respectively, in a collinear conditional expression; It is done by

And the three-dimensional image generation of the object is performed on a computer, the position coordinates of the camera moved in the vertical direction of the sphere in the step (S20) by the distance the operator moved the mouse from side to side (or up and down) on the monitor of the computer The spatial coordinates of the object to which the camera is moved in the horizontal direction of the sphere are determined by the distance the operator moves the mouse up and down (or left and right) on the monitor.

The present invention having the configuration as described above requires only the spatial coordinates of the object to be observed, and the left and right images of the object viewed from any direction are produced in real time according to the principle of binocular disparity and the collinear condition equation, and thus, the spatial coordinates of the object. And the capacity of the memory for storing the data of the produced left and right images can be reduced, the stereoscopic image of the object is generated immediately according to the observer's eye movement,

The observation direction of the observer, that is, the vertical observation position where the camera is moved vertically from the virtual sphere surface as the external facing element of the camera is composed of the position coordinates of the camera, and the horizontal observation position where the camera is moved horizontally from the virtual sphere surface. Since the camera is not moved and the object is rotated and composed of the spatial coordinates of the changed object, the computational speed for producing the left and right images is increased, and the vanishing point of the stereoscopic image is prevented from occurring in a different direction from the actual one.

Also, the stereoscopic image of the object is generated in real time according to the change of observation direction, so it can be widely used for stereoscopic image data processor, stereoscopic digital image and digital map, stereoscopic city modeling, stereoscopic advertisement, stereoscopic game, stereoscopic simulation, stereoscopic navigation, etc. Do.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a flowchart of a method for generating a real-time stereoscopic image according to the present invention. As shown in the drawing, acquiring a spatial coordinate of an object (S10) and an external expression of a camera photographing an object from a virtual sphere surface Acquiring the element (S20), acquiring the position coordinates of the camera moved in the vertical direction of the sphere (S21), and acquiring the spatial coordinates of the object rotated on the Z axis (S23); Separating the position coordinates into left and right coordinates (S30), producing a 2D left image and a right image of the object by a collinear conditional expression (S40), and generating a stereoscopic image of the object from the left image and the right image A step S50 is made.

In step S10, the spatial coordinates (X, Y, Z) of the object 10 are acquired. Here, the object may be a terrain, a feature of a certain surface area, a virtual building designed by an architect, or a vehicle, which may be a vehicle.

And the spatial coordinates (X, Y, Z) of these objects 10 can be obtained through the digital map data if the object is the terrain, features of the actual surface of the earth, and if the object is a virtual building or a car, the worker through the design, etc. Can give spatial coordinates. Since the method of giving the spatial coordinates to the object 10 is common, a detailed description thereof will be omitted.

In order to generate a stereoscopic image according to the present invention, one point of the object 10 starts on the condition that it is located at the center of the virtual sphere (0, 0, 0). Therefore, the space coordinate of the acquired object 10 is (0, 0). , 0) is not included, the spatial coordinate (X, Y, Z) of the object 10 is updated to linearly move the object so that any point of the object 10 is located at the center of the sphere.

등이 있다. Next, in step S20, an external expression element of the camera 20 is acquired. Here, the camera is not a real camera but a virtual camera, and what we need is an internal expression element and an external expression element of the camera 20 under the assumption that the camera 20 photographs the object 10 at a certain position. The internal focusing elements of the camera include the focal length (f) of the lens and the resolution. The external focusing elements include the positional coordinates (x, y, z) of the camera and the shooting angle of the camera.

Etc.

Figure 2 shows a camera 20 for photographing the object 10 in the assumption (condition) applied in the present invention, as shown in the figure any one point of the object 10 is located in the center of the virtual sphere, the camera (20) is located on the surface of the imaginary sphere whose radius is r.

The radius r of the sphere determines the accumulation of stereoscopic images. That is, when the radius r is large, the object generated in the stereoscopic image is expressed small, and when the radius r is small, the object is expressed large. The radius r is your choice.

The camera 20 moves on the surface of the virtual sphere to photograph the object 10, and the projection center is directed toward the center of the sphere.

와 카메라가 구의 수평방향(즉, 구의 XY면)으로 이동한 각도

로 표현 가능하고, 아래의 수학식1과 같다. All shooting positions (x, y, z) of the camera 20 are angles at which the camera moves in the vertical direction of the sphere (ie, the sphere's XZ plane or YZ plane) as shown in the figure.

And the angle the camera has moved in the horizontal direction of the sphere (ie, the XY plane of the sphere).

Can be expressed as

x = r(sincos)

x = r (sincos)

sin

)y = r (sin

sin

)

z = rcos

In determining the photographing position (x, y, z) of the camera 20, Equation 1 is used to determine the vertical observation position (S21) in which the camera 20 moved in the vertical direction of the sphere,

는 일정각도를 갖고,

는 0도 이다. The horizontal observation position moved by the camera in the horizontal direction of the sphere is indirectly determined by changing the spatial coordinates (X, Y, Z) of the object 10 by rotating the object 10 about the Z axis as described below. S23). In other words,

Has a certain angle,

Is 0 degrees.

In this way, only the position where the camera 20 is moved in the vertical direction of the sphere as the photographing position of the camera is to increase the calculation speed for producing the left image and the right image of the object 10 by the collinear conditional expression.

또는

와 카메라의 촬영각도

가 필요하다. 여기서

는 카메라가 각각 X축, Y축, Z축으로 회전한 각도를 나타내는 것으로서 카메라의 촬영위치를 나타내는 각도 (

,

)와 관계는 있지만 같은 값을 갖지는 않는다. In more detail, as shown in equations (4) and (5), the positional coordinates of the camera are included in the parameter applied to the collinear conditional expression.

or

Angle of camera and camera

Is needed. here

Is the angle that the camera rotates on the X, Y, and Z axes, respectively.

,

) But not the same value.

는 (

,

)로부터 직접적으로 구할 수 없고 카메라의 투영중심이 구의 중심을 향한다는 조건을 부가시켜 복잡한 수식을 통해 구해진다. Therefore, when the camera 20 uses both the vertical and horizontal positions of the sphere as the left position of the camera, the shooting angle of the camera

Is (

,

It can not be obtained directly from) but is obtained through a complex equation with the condition that the projection center of the camera is towards the center of the sphere.

=0) 하고, 카메라가 구의 수평방향으로 이동한 위치는 대상물(10)을 회전시켜 변경된 공간좌표를 사용하는 경우에는 카메라의 촬영각도는 (

=-

,

=0,

=0)이 된다. 즉, 이때는 카메라의 촬영각도 셋 중 하나는 카메라 가 구 표면에서 수직으로 이동한 각도와 같아지고 나머지 둘은 0도가 됨으로서 카메라의 촬영각도

를 별도로 구할 필요가 없다. However, only the position where the camera 20 moves in the vertical direction of the sphere is used as the camera's shooting position coordinates (that is,

= 0), and the camera moves in the horizontal direction of the sphere. When the object 10 is used to change the spatial coordinates, the camera's shooting angle is (

=-

,

= 0,

= 0). In other words, one of the camera's three shooting angles is equal to the angle the camera moves vertically from the sphere surface, and the other two are zero degrees.

There is no need to obtain

가 모두 상수화 될 것이다.) 대상물(10)을 Z축 및 X축(또는 Y축)으로 회전시켜 대상물의 공간좌표만을 변경하여 대상물의 좌측영상 및 우측영상을 제작하는 것을 생각할 수 있다. 그러나 이 방식으로 생성되는 대상물의 입체영상은 그 소실점이 엉뚱한 방향에서 생성될 위험이 높다. And on the other hand, the camera 20 is fixed to a specific position of the sphere surface (in this case, the camera shooting position (x, y, z) and the shooting angle)

Will be constant.) It is conceivable to produce the left and right images of the object by changing only the spatial coordinates of the object by rotating the object 10 along the Z axis and the X axis (or Y axis). However, stereoscopic images of objects created in this way have a high risk of their vanishing points being generated in the wrong direction.

Vanishing point means that when looking at an object, the far point of the object converges at a certain point. When we see a certain road, the road ahead is wide, but the farther it is, the narrower the road is.

,

)에 상응)과 카메라의 시선방향(

에 상응)이 일치하지 않는 경우가 많아 소실점이 엉뚱한 방향에 생길 수 있다. When looking down from the top of the object 10, the vanishing point should appear at the bottom, and from the front, the vanishing point should be at the rear, which is normal. In the case of generating a stereoscopic image, the viewing direction of the object ((

,

) And the camera's line of sight (

Are often inconsistent, resulting in vanishing points in the wrong direction.

As described above, the horizontal observation position where the camera 20 moves in the horizontal direction of the sphere is determined indirectly by changing the spatial coordinates of the object by rotating the object 10 about the Z axis. The spatial coordinate change of the object 10 is performed by Equation 2 below.

는 회전된 후 대상물(10)의 공간좌표이고,

는 회전되기 전 대상물(10)의 공간좌표이고,

는 Z축으로의 회전각도로서

=-

이다. here

Is the spatial coordinate of the object 10 after being rotated,

Is the spatial coordinate of the object 10 before being rotated,

Is the angle of rotation in the Z axis

=-

to be.

와 우측좌표

로 분리한다. Next, in step S30, the left coordinate of the position coordinates (x, y, z) in which the camera 20 is moved in the vertical direction of the sphere by the principle of binocular disparity.

And right coordinate

To separate.

는 아래의 수학식3으로부터 구해진다. The binocular parallax principle applies that the left and right eyes of a person are separated by a certain distance on a straight line, and some of the areas seen by the left and right eyes are overlapped (shared), and the initial position (x, In y, z), the x-coordinate of the camera changes by moving the camera to the left and right a certain distance. Spacing of two cameras moved left and right, respectively

Is obtained from Equation 3 below.

는 카메라에서 영상의 바닥면까지의 수직거리로서 통상 가상의 구 반지름 r에 상응한다. 그리고 R은 영상의 바닥면에서 돌출된 대상물의 최고높이 이고, P는 작업자(관측자)가 임의로 선택하는 좌측영상과 우측영상의 중복도로서 그 값은 60~90% 인 것이 바람직하다. Here, b is the length of one side of the region (that is, the image) containing the object to be photographed by the camera, f is the focal length of the camera,

Is the vertical distance from the camera to the bottom of the image and usually corresponds to the virtual sphere radius r. And R is the highest height of the object protruding from the bottom of the image, P is the degree of redundancy of the left image and the right image arbitrarily selected by the operator (observer), the value is preferably 60 ~ 90%.

로부터 좌측좌표는

=(x-

/2, y, z), 우측좌표는

=(x+

/2, y, z)로 결정된다. 물론, 좌측좌표는

=(x, y, z), 우측좌표는

=(x+

, y, z)로 결정해도 된다. The initial position coordinates (x, y, z) that the camera moved in the vertical direction of the sphere and the distance between the two cameras that are moved left and right, respectively

The left coordinate from

= (x-

/ 2, y, z), right coordinate

= (x +

/ 2, y, z). Of course, the left coordinate

= (x, y, z), right coordinate

= (x +

, y, z).

및 우측좌표

, 그리고 대상물의 변경된 공간좌표

가 모두 취득된 이후에는, 이들을 아래의 수학식4(공선조건식)에 적용하여 좌측영상과 우측영상을 제작한다.(S40) 이에 관한 일례가 도3에 도시되어 있다. Through the above process, the left coordinate of the camera according to the change of the observation position of the observer (camera)

And right coordinate

, And the modified spatial coordinates of the object

After all are obtained, the left and right images are produced by applying them to Equation 4 (collinear condition equation) below. (S40) An example thereof is shown in FIG.

,

)은 좌측영상의 픽셀이고, (

,

)은 우측영상의 픽셀이다. 그리고

~

은 카메라의 촬영각도에 관련된 것으로서 아래의 수학식5와 같다. here, (

,

) Is the pixel of the left image, (

,

) Is a pixel of the right image. And

To

Is related to the photographing angle of the camera as shown in Equation 5 below.

만큼 이동한 카메라의 위치의 좌표로 구하고, 구의 표면에서 수평방향을 이동한 수평관측위치는 대상물(10)을 Z축을 축으로 각도

(=-

)만큼 회전시켜 변경된 대상물의 공간좌표로 구하므로, 수식5의

에 각각 (

, 0, 0)을 적용하면 r11=r22=r33=1이고 나머지는 0이 된다. As described above, the vertical observation position moved vertically from the surface of the sphere with respect to any observation position is determined by the angle of the camera 20.

The horizontal observation position obtained by the coordinates of the position of the camera moved by as much as possible and the horizontal direction moved from the surface of the sphere is determined by the angle of the object 10 to the Z axis.

(=-

Rotate by) to obtain the spatial coordinates of the changed object.

On each (

, 0, 0), r11 = r22 = r33 = 1 and the rest is 0.

Therefore, Equation 4 is simplified as Equation 6 below.

After the left image and the right image are produced, the left image and the right image of the object 10 are applied to any one of Side-by-Side, Interlace, and Anaglyph methods to generate a stereoscopic image of the object. S50)

The side-by-side method is a so-called auto glasses-free method, and the left image and the right image are separated from each other by a certain distance, and the operator directly controls the distance between the left and right images.

The interlace method is a method of overlapping the left image and the right image in a so-called polarization stereoscopic method, and the operator wears polarized glasses or arranges and views a polarizing panel on the image.

The Anaglyph method is a so called stereoscopic method, and the colors of the left image and the right image are expressed in complementary colors (eg, the left image is red, the right image is blue), and the glasses having complementary colors after the two projections are superimposed. (Yes, left glasses are red, right glasses are blue).

The left and right images of the object are manufactured after newly updating the spatial coordinates of the object so that any one point of the object 10 having the three-dimensional spatial coordinates is located at the center of the virtual sphere. In addition, the process of disposing the left and right images at a predetermined distance or overlapping the images is performed on a computer.

The radius r of the sphere with constant characteristics, the focal point f of the camera, the degree of redundancy P of the left and right images, and the length b of one side of the left and right images are determined by the operator (observer) and input into the computer.

와, 카메라가 구의 수평방향으로 이동한 수평관측위치

는 작업자가 컴퓨터의 모니터에서 이동시키는 마우스의 이동거리에 의해 실시간으로 연산된다. Vertical observation position in which the camera moves in the vertical direction of the sphere to determine the observation position of the object with variable characteristics

And the horizontal observation position where the camera moved in the horizontal direction of the sphere.

Is calculated in real time by the movement distance of the mouse that the operator moves on the monitor of the computer.

의 각도가 변하여 카메라의 위치좌표가 실시간으로 연산되고 , 작업자가 마우스를 모니터에서 드래그 하여 상하로 이동시키면 그 이동거리에 비례하여 수평관측위치를 나타내는

의 각도가 변하여 대상물의 회전된 공간좌표가 실시간으로 연산된다. For example, if a worker drags the mouse on the monitor and moves it to the left or right, it indicates the vertical observation position in proportion to the movement distance.

The position coordinates of the camera are calculated in real time, and the operator drags the mouse on the monitor and moves it up and down to indicate the horizontal observation position in proportion to the movement distance.

The angle of is changed so that the rotated spatial coordinates of the object are calculated in real time.

When the position coordinates of the camera and the spatial coordinates of the object are calculated according to the moving direction of the mouse, the left image and the right image of the object are produced at the same time. The image is generated.

In the above description of the present invention, a method of generating a real-time stereoscopic image made of a specific process is described with reference to the accompanying drawings, but the present invention can be variously modified and changed by those skilled in the art. Should be interpreted as being within the scope.

1 is a flowchart of a method for generating a real-time stereoscopic image according to the present invention.

2 shows an object and a camera located on a virtual sphere.

3 is a diagram illustrating an example of producing a left image and a right image of an object;

4 is an example of a stereoscopic image to which the autostereoscopic method is applied.

5 is an example of a stereoscopic image to which a polarized stereo system is applied.

6 is an example of a three-dimensional image to which the color scheme is applied.

<Description of the symbols for the main parts of the drawings>

10: object 20: camera

Claims (2)

In the method for generating a three-dimensional image of the object viewed from any direction in real time, (S10) acquiring three-dimensional spatial coordinates of the object at which one point of the object is located at the center of a virtual sphere; (S20) Acquire an external expression element (that is, a photographing position) of a camera photographing the object in real time with the center of the sphere as the projection center at an arbitrary position of the virtual sphere, As shown in FIG. 2 of the drawing, the vertical observation position of the camera in the vertical direction of the sphere (i.e., the XZ plane or the YZ plane of the sphere) is represented by the Equation of [Detailed Invention]. [1], and the horizontal observation position where the camera is moved in the horizontal direction of the sphere (i.e., the XY plane of the sphere) as shown in [Fig. 3] of [Drawing]. Acquiring the object in spatial coordinates of the object by rotating the object in the Z axis using Equation 2; (S30) separating the position coordinates of the camera moved in the vertical direction of the sphere into left and right coordinates by the principle of binocular disparity; (S40) Substituting the spatial coordinates of the changed object, the left coordinates of the camera, the spatial coordinates of the object and the right coordinates of the camera, respectively, into collinear conditional expressions (see [Equation 6] in [Detailed Description of the Invention]). Real-time stereoscopic image generation method comprising a; producing the left image and the right image of the object in real time. The method of claim 1, Stereoscopic image generation of the object is performed on a computer, The position coordinates of the camera moved by the camera in the vertical direction of the sphere in step S20 are determined by the distance the operator moves the mouse left and right (or up and down) on the monitor of the computer, and the object moved by the camera in the horizontal direction of the sphere. The spatial coordinate of the real-time stereoscopic image generation method, characterized in that determined by the distance the operator moves the mouse up and down (or left and right) on the monitor.

Images