KR19980023727A - Virtual Reality System for Displaying Panoramic Images - Google Patents

Abstract

The present invention relates to a virtual reality system for displaying a panoramic image, from the image data storage means 71, the panoramic image storage means 73, the image data storage means 71 Panoramic image generating means 72 for receiving the image data divided into a plurality of frames and storing the image data in row order in the panoramic image storage means 73 to generate the panoramic image, according to the user's viewpoint. 3D graphics processing units 75 and 76 for reading RGB image signals by reading corresponding frame data from the panoramic image storage unit 73 and rendering and texture mapping, and RGB output of 3D graphics processing unit. Signal converters 12-1 and 12-2 for converting the video signal to a TV video signal, and the HMD 23 and HMD for detecting the user's movement while receiving and displaying the output of the signal converter. A position and direction tracker 11 that receives a motion signal and calculates a position and direction of the user, and calculates a view volume of the panoramic image according to the user's movement input from the position and direction tracker 11 to the 3D The viewing angle calculation unit 78 outputs to the graphic processing units 75 and 76. Therefore, in the virtual reality system, a panoramic image can be displayed naturally according to the user's view point.

Description

A virtual reality system for displaying panoramic images

The present invention relates to a virtual reality system (VR), and more particularly to a virtual reality system for displaying a panoramic image (panoramic image).

The virtual reality system vividly shows the unfinished or non-existent virtual situation as if it is a real situation, and gives users a lot of knowledge and experience by freeing the desired action in it. It is implemented based on computer technology and image technology. Unlike the existing computer that provides a two-dimensional environment of use in the form of a window and treats the user as an observer or a foreigner, VR provides a three-dimensional environment of use (virtual world) to the user. It feels like you are in this virtual world and allows you to directly manipulate objects that appear in virtual reality. Because of these differences, the use of virtual reality (VR) technology enables vivid knowledge and experiences that are difficult to provide by traditional information provision and interaction methods, so education / training, entertainment / culture, science / medicine, design, etc. It is expected to create a huge new market in various fields such as manufacturing, telecommunications, and defense.

Such VRs are used interchangeably with terms such as virtual environment, virtual presence, artificial world, virtual world, cyberspace, depending on the point of view. However, in general, a sensor provides a three-dimensional virtual world similar to the real world created by a computer, an input means that can be freely manipulated in real time with the virtual world, and a real sense in response to the user's manipulation. It is a technology that enables artificial experience and experience by providing a means of sensory feedback.

As an example of such a virtual reality (VR) system, a system using a PC-based platform is composed of a reality engine and an effector, as shown in FIG. 1. In addition, the reality engine is a VGA graphics board, a first 3D graphics board, a second 3D graphics board, a 3D sound board, an interface board, a communication port, a hard disk, a CPU, a RAM, etc., implemented on a PC computer main body, and a first 3D graphics board. First signal converter 12-1 for converting the RGB signal into an NTSC TV signal, a second signal converter 12-2 for converting the RGB signal of the second 3D graphics board into an NTSC method TV signal, VGA And a VGA monitor 14 for displaying an image input from the graphic board, an NTSC monitor 13 for displaying an image input from the first signal converter 12-1, a position and direction tracker 11, and the like.

The effect generator is composed of a head mounted display (HMD) 23, a transmitter 22, a joystick 21, etc. for the user to write and experience the world of virtual reality in an audiovisual manner, and the HMD 23 is shown in FIG. As shown in the drawing, a sensor 24 for detecting a user's head direction, a left and right speaker 25 for providing stereo sound, a left and right LCD 27 for displaying an image, an optical lens 26 and the like are included. have.

In such a system, the user's head movement is tracked by the transmitter 21, the sensor 24, and the position tracker 11 and input to the reality engine, and the reality engine displays an image at an appropriate point in time according to the user's head movement. To provide. In addition, the user may interface with the virtual world using an input device such as a joystick.

However, the image provided by the virtual reality system is a level of reconstructing the image of the entire virtual reality already determined according to the user's point of view, and there is a problem in that the panoramic image such as a wide beach cannot be properly represented.

In other words, display devices such as LCD and CRT, which are used as the display means of the virtual reality, are limited to the size and operation speed of the video RAM, and thus the size of the image is limited to 680 x 480. The screen cannot be displayed all at once. Therefore, in order to provide a panoramic image in a conventional virtual reality system, as shown in FIGS. 3A and 3B, since a panoramic image composed of a plurality of frames is sequentially displayed, the movement of the screen is abrupt. There is a problem in that the contents of the moving and video screens do not match.

Accordingly, the present invention provides a virtual reality system that can generate a single panoramic image by synthesizing an image divided into a plurality of frames in order to solve the above problems, and display it naturally as the viewpoint moves. Its purpose is to.

In order to achieve the above object, the present invention, the panoramic image generating means for receiving the image data divided into a plurality of frames to generate a panoramic image; Panoramic storage means for storing the generated panoramic image; A three-dimensional graphic processing unit which reads out image data of one frame corresponding to the viewpoint of a user from the panoramic image storage unit, renders and texture-maps and outputs an RGB image signal; A signal converter for converting the RGB output of the 3D graphic processor into a TV image signal; Image display and motion detection means for receiving and displaying the output of the signal converter and detecting a user's movement; A position and direction tracker which receives a user's motion signal from the image display and motion detection means and calculates the position and direction of the user; And a viewing angle calculator for calculating a view volume of the panoramic image according to the movement of the user input from the position and direction tracker and outputting the view volume to the 3D graphic board.

1 is a block diagram showing a typical virtual reality system,

2 is a detailed configuration diagram of an exemplary HMD used in the virtual reality system of FIG.

3A and 3B are conceptual views of implementing a panoramic display in a conventional manner in a virtual reality system;

4A and 4B illustrate a concept of generating a panoramic image according to the present invention in a virtual reality system;

5 is a flowchart illustrating a method of generating a panoramic image according to the present invention;

6 is a conceptual diagram for displaying a panoramic image according to the present invention;

7 is a block diagram of a virtual reality system for displaying a panoramic image according to the present invention,

FIG. 8 is a functional block diagram of the viewing angle calculator shown in FIG. 7.

* Explanation of symbols for main parts of the drawings

11: Position and direction tracker 12-1, 12-2: Signal converter

21: input device 22: transmitter

23: HMD

71: image data storage unit 72: panoramic image generator

73: panoramic image storage unit 74: CPU

75, 76: 3D graphics board 77: 3D sound processor

78: viewing angle calculation unit

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

3A and 3B are conceptual views of implementing a panoramic display in a conventional manner in a virtual reality system, and FIGS. 4A and 4B are views illustrating a concept of generating a panoramic image according to the present invention in a virtual reality system. 5 is a flowchart illustrating a method of generating a panoramic image according to the present invention.

Referring to FIG. 3, conventionally, as shown in 3a, if the panoramic image is composed of three frames Sa, Sb, and Sc (assuming that each frame is composed of 640 x 480 pixels), image data of these frames is provided. Are stored separately on the video RAM as shown in 3b.

That is, the Sa frame consists of row data from a1 to a480, these data are stored sequentially from a1 to a480 on the video RAM, and the Sb frame consists of row data from b1 to b480, and these data are video The RAM frames are sequentially stored from b1 to b480, and the Sc frame is made up of row data from c1 to c480, and these data are stored from c1 to c480 sequentially on the video RAM.

When such a panoramic image is displayed in a conventional virtual reality system, since only one frame screen is sequentially displayed in order of a Sa screen, an Sb screen, and an Sc screen, the screen moves rapidly. For example, when the user's viewpoint gradually moves from the Sa screen to the Sb screen, the right part of the Sa screen and the left part of the Sb screen should be visible and gradually moved to the Sb screen. And screen do not match.

Therefore, in the present invention, a screen bent in three frames as shown in FIG. 3A is composed of one wideband frame (ie, a panoramic image Sp) as shown in FIG. 4A, and then a corresponding image is displayed according to a user's view point. By extracting data to form and display image data of one frame, a panoramic screen is naturally provided according to a viewpoint.

To this end, in the present invention, image data divided and stored in units of frames as shown in FIG. 3B are continuously stored for each row as shown in FIG. 4B to form one huge frame.

That is, when three frames make one panoramic image as shown in Fig. 3, a1, a2, ... a480, b1, b2. Video data stored in the order of ..., b480, c1, c2, ... c480 is stored in the order of a1, b1, c1, a2, b2, c2, ... a480, b480, c480.

Even if one frame consists of n rows and one panoramic image consists of k frames, the image data divided and stored for each frame is rearranged and stored for each row in the same manner as described above. You can create a panoramic image.

As described above, the method according to the present invention for generating one panoramic image with k frames having n rows includes a first memory, which is divided into frames from the first frame to the k-th frame, as shown in FIG. 5. From the first row of the first frame to the nth row of the kth frame are sequentially read in row number order and stored in the second memory to generate a panoramic image.

In FIG. 5, in the initialization step 100, the frame variable i and the row variable j are initialized to 1, in the reading step 101, the j row data of the i frame is read from the first memory, and in the storage step 102, the row variable j is read. i frame j row data is sequentially stored on the second memory.

Subsequently, after the frame variable i is increased by 1 (103) and the increased frame variable i is not greater than the last frame number k, the reading step 101 and the storing step 102 are repeated for the same row number of another frame. If the increased frame variable i is greater than the last frame number k, the process is repeated for the next row.

That is, if the increased frame variable i is greater than the last frame number k, the row variable j is increased by one (105). If the increased row variable j is not larger than the last row number n after initializing the frame variable i to 1, another row is increased. The reading step 101 and the storing step 102 are repeated for the number, and the process ends when the increased row variable j is greater than the last row number k (107).

When such a process is performed, one panoramic image is stored in the second memory.

Meanwhile, FIG. 6 is a conceptual diagram for displaying a panoramic image according to the present invention, FIG. 7 is a configuration diagram of a virtual reality system for displaying a panoramic image according to the present invention, and FIG. 8 is shown in FIG. Is a functional block diagram of a calculated viewing angle calculator.

As described above, the present invention forms one panoramic image and then extracts screen data according to the user's viewpoint to create and display one frame image so that the user can enjoy a wide range of images naturally. It would be.

Referring to FIG. 6, a plurality of frames (... Sn-1, Sn, Sn + 1 ...) are generated as one panoramic image by texture mapping on the inner side of the cylindrical coordinates to the user's viewpoint. Therefore, the corresponding image data is extracted and displayed. In other words, the image consisting of .... Sn-1, Sn, Sn + 1 ..... appears on the inner surface of the cylindrical coordinates away from the user by r, when the user turns the head using the HMD (23) accordingly Panoramic images are displayed naturally.

As described above, the virtual reality system according to the present invention for displaying a panoramic image includes a CPU 74, an image data storage unit 71, a panoramic image generator 72, and a panorama. MIC image storage unit 73, the first 3D graphics board 75, the second 3D graphics board 76, the 3D sound processor 77, the viewing angle calculator 78, the position and direction tracker 11, the first And a signal converter 12-1, a second signal converter 12-2, an HMD 23, a transmitter 22, an input device 21, and the like.

The CPU 74, which is the core of the reality engine, controls various components of the system by executing a predetermined program, and the image data storage 71 stores image data divided into several frames. As the image data storage 71, a hard disk drive (HDD) or other auxiliary storage devices may be used.

The panoramic image generator 72 performs a sequence as shown in FIG. 5, reads images stored separately by frames in the image data storage 71, and stores the images sequentially in the panoramic image storage 73. To create one panoramic image. Therefore, the panoramic image storage unit 73 stores the panoramic image data.

The first 3D graphics board 75 reads image data corresponding to a viewpoint according to a user's view point signal input from the viewing angle calculator 78 among data stored in the panoramic image storage 73. And a frame to generate the left RGB image signal, and the second 3D graphic board 76 is a view point of the user input from the viewing angle calculator 78 among the data stored in the panoramic image storage 73. ), The image data corresponding to the viewpoint is read to form a frame to generate the right RGB image signal.

In this case, in order to correspond to the change of viewpoint of the panoramic image in the 3D graphic, the texture mapping of the panoramic image inside the cylindrical coordinate axis is performed, and the view volume is adjusted according to the movement of the user head. Calculate (view volume) to extract / display only the relevant data.

For this purpose, the head direction of the user is detected by a sensor in the HMD 23 and the direction is calculated in the position and direction tracker 11.

As shown in FIG. 8, the viewing angle calculator 78 calculates the angle conversion amount of the user head according to the user's motion signal input from the position and direction tracker 11, and then calculates the pixel change amount on the display accordingly. Then, the new view point is set by adding the pixel change amount to the current view point, and the view volume is calculated accordingly.

That is, in FIG. 8, the head angle change calculation unit 81 analyzes the direction signal received from the HMD 23 to calculate the angle conversion amount of the user head, and the pixel change amount calculation unit 82 displays the display according to the angle change amount. Compute the amount of change in the image pixels

The adder 84 then adds the current view point 83 and the pixel change amount, and the view point setting unit 85 sets a new view point according to the output of the adder 84, and calculates the view volume. The unit 86 calculates a view volume according to the newly set view point.

The input device 21 is required for the user to manipulate the virtual reality, and a joystick, a mouse, a data globe, and the like may be used.

On the other hand, the operation of detecting the direction of the user in the HMD 23, when the transmitter 22 transmits a radio signal under the control of the position and direction tracker 11, the receiver (sensor: Figure 2 of the inside of the HMD) 24 receives this and forwards the received signal to the position and direction tracker 11. The position and direction tracker 11 analyzes the received signal to detect the position and direction of the user.

As described above, according to the present invention, after rearranging and storing image data divided into a plurality of frames to generate a panoramic image, only the corresponding data is extracted according to a viewpoint, and a frame is formed and then displayed. In the real system, the panoramic image can be displayed naturally according to the user's view point.

Claims (4)

Image data storage means 71; Panoramic image storing means (73); Panoramic image generating means for receiving the image data divided into a plurality of frames from the image data storage means 71 and storing them in the panoramic image storage means 73 in row number order to generate a panoramic image. 72; A three-dimensional graphic processing unit (75, 76) for reading image data of one frame corresponding to the viewpoint of a user from the panoramic image storing means (73), rendering and texture mapping and outputting an RGB image signal; Signal converters (12-1, 12-2) for converting the RGB output of the three-dimensional graphic processing unit into a TV video signal; Image display and motion detection means for receiving and displaying the output of the signal converter and detecting a user's movement; A position and direction tracker (11) which receives a user's motion signal from the image display and motion detection means and calculates the position and direction of the user; And A panoramic view composed of a viewing angle calculator 78 that calculates a view volume of a panoramic image according to a user's movement input from the position and direction tracker 11 and outputs the view volume to the 3D graphic processing units 75 and 76. Virtual reality system for displaying images. The method according to claim 1, wherein said panoramic image generating means (72) The panoramic image storage unit 73 reads sequentially from the first row of the first frame to the nth row of the k-th frame in the order of row number from the image data storage unit 71 in which k frames including n rows are stored. Virtual reality system for displaying a panoramic image, characterized in that the storage. The method of claim 1, wherein the viewing angle calculator 78 An angle change calculator 81 for calculating an angle conversion amount of a user head by analyzing a signal received from the image display motion detection means; and a pixel conversion amount calculation unit 82 calculating a change amount of a pixel on a display according to the angle change amount ), An adder 84 for adding the current viewpoint and the pixel change amount; A view point setting unit 85 for setting a new view point according to the output of the adding unit 84; Virtual reality system for displaying a panoramic image, characterized in that consisting of a view volume calculation unit 86 for calculating the view volume according to the newly set view point. The virtual reality system for displaying a panoramic image according to claim 1, wherein said image display and motion detection means comprise a head mounted display (HMD).