KR20160075315A - Display system for generating virtual reality image - Google Patents
Display system for generating virtual reality image Download PDFInfo
- Publication number
- KR20160075315A KR20160075315A KR1020150162824A KR20150162824A KR20160075315A KR 20160075315 A KR20160075315 A KR 20160075315A KR 1020150162824 A KR1020150162824 A KR 1020150162824A KR 20150162824 A KR20150162824 A KR 20150162824A KR 20160075315 A KR20160075315 A KR 20160075315A
- Authority
- KR
- South Korea
- Prior art keywords
- virtual reality
- virtual
- unit
- image
- monitor
- Prior art date
Links
Images
Classifications
-
- H04N13/04—
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G02B27/22—
Abstract
The present invention relates to a display system for generating a virtual reality image. A display system according to the present invention includes: an HMD device for displaying a virtual reality image; A virtual monitor generating unit for generating a virtual monitor; and a display unit for displaying the virtual monitor generated by the virtual monitor generating unit in the virtual reality space so as to be displayed in the virtual reality space, And a control unit for controlling the virtual monitor generating unit to generate a virtual monitor corresponding to the number of the virtual monitors inputted through the user input unit, so that the generated virtual monitor is displayed in the virtual reality space And a control unit for controlling the virtual reality viewer unit to generate a virtual reality image and controlling the communication unit to transmit the generated virtual reality image to the HMD device.
Description
The present invention relates to a display system for generating a virtual reality image, and more particularly, to a display system for generating a virtual reality image including a plurality of virtual monitors.
A virtual environment is a computer simulated environment, which can provide a virtual reality for a person. In a virtual environment, simulations of the physical presence of a place can exist in the imaginary world as well as the real world. Such a virtual environment can be displayed using a display screen or a head-mounted display (HMD), and in particular, in the case of an HMD, a virtual environment can be displayed in three dimensions. Generally, such a virtual environment is mainly used for displaying contents in an entertainment field such as a game, but such a virtual environment has not been applied to a virtual desktop until now.
On the other hand, there are programs that implement multiple virtual monitors on a single physical computer. This is a program that implements a plurality of virtual monitors on one physical computer so that a plurality of monitors can be virtually implemented without using a physical monitor to be used by the user. However, in the case of the conventional virtual monitor program, the number of virtual monitors that can be implemented is very limited, and the display of the virtual monitor is dependent on the specification of the physical monitor connected to the physical computer. That is, there is a problem that the plurality of virtual monitors can not be displayed on the physical monitor depending on the size, resolution, and the like of the physical monitor.
Accordingly, an object of the present invention is to provide a virtual reality image generating apparatus and a method of generating virtual reality images including a plurality of virtual monitors, and transmitting the generated virtual reality images to a head-mounted display (HMD) And to provide a display system which can be used as many as possible.
The above object is achieved by a display system, comprising: an HMD device for displaying a virtual reality image; A virtual monitor generating unit for generating a virtual monitor; and a display unit for displaying the virtual monitor generated by the virtual monitor generating unit in the virtual reality space so as to be displayed in the virtual reality space, And a control unit for controlling the virtual monitor generating unit to generate a virtual monitor corresponding to the number of the virtual monitors inputted through the user input unit, so that the generated virtual monitor is displayed in the virtual reality space And a control unit for controlling the virtual reality viewer unit to generate a virtual reality image and controlling the communication unit to transmit the generated virtual reality image to the HMD apparatus.
Wherein when the request for changing the number of the virtual monitors is input through the user input unit while the virtual reality image being transmitted is displayed through the HMD device, The monitor generating unit, the virtual reality viewer unit, and the communication unit.
The virtual reality viewer unit may detect a predetermined change in the virtual reality image, and may generate an updated virtual reality image by reflecting the change.
Wherein the virtual reality space includes a virtual reality background, objects included in the virtual reality background, and background audio, and generates a UI capable of receiving the user selection for the virtual reality background, the object, and the background audio And a UI creating unit for creating a UI.
The UI generation unit generates a virtual reality background selection UI, a object selection UI, and a background audio selection UI. The virtual reality viewer unit displays a virtual reality background selected by the UI, a type and position of the object, The virtual reality image may be generated so that the virtual reality image is reflected.
A plurality of virtual reality backgrounds selectable by the user via the virtual reality background selection UI, a plurality of types of objects selectable by the user through the object selection UI, and a plurality of audio items selectable by the user via the background audio selection UI And a storage unit for storing the data.
The HMD device may further include a camera, and the virtual reality viewer unit may generate an image captured by the camera, the second virtual reality image included in the virtual reality image.
The HMD device further includes a gesture recognition sensor, and transmits a gesture signal sensed by the gesture sensor to the communication unit. The control unit recognizes the received gesture signal as a user selection, It is possible to control the virtual reality viewer unit such that a virtual reality image that reflects user selection is generated.
As described above, according to the present invention, a virtual reality image including a plurality of virtual monitors is generated and transmitted to a head-mounted display (HMD) device, There is provided a display system that enables a virtual monitor to be used regardless of the display apparatus. Accordingly, the user can perform various operations using a desired number of virtual monitors without being restricted by the space such as an airplane, a train, a car, and an office.
1 is a schematic control block diagram of a display system of the present invention,
Figure 2 illustrates one embodiment of a virtual reality image that may be implemented by the
3 shows an example of a UI that can be generated by the UI generation unit 150 of the present invention,
4 shows an embodiment of a virtual reality image in which an image captured by a camera of the HMD device 200 is displayed together,
5 illustrates an embodiment of a user selection corresponding to a gesture signal recognized by the HMD device 200. As shown in FIG.
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art to which the present invention pertains. The present invention may be embodied in many different forms and is not limited to the embodiments described herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and the same or similar components are denoted by the same reference numerals throughout the specification.
1 is a schematic control block diagram of a display system of the present invention. 1, a
The computer 100 of the present invention includes any type of devices having functions similar to PCs such as smartbooks, MIDs (Mobile Internet Devices), netbooks, tablet PCs, and the like, as well as general forms of PCs such as laptops, It may also include a mobile display device such as a PDA, a smart phone, etc., which can perform at least some of the PC functions. The computer 100 of the present invention may include a variety of general components that enable an operation according to a user's selection through a predetermined OS such as a window, a Macintosh, and the like.
The HMD device 200 is a head mount display device that is capable of displaying a virtual reality image to a user and includes a head tracking function to display an image corresponding to the motion of the user's gaze in a three- So that the user can enjoy a bodily sensation as if the virtual reality exists in the virtual reality.
The computer 100 includes a communication unit 110, a virtual monitor generation unit 120, a virtual reality viewer unit 130, a user input unit 140, a UI generation unit 150, a storage unit 160, ).
The communication unit 110 is capable of performing communication with the HMD device 200 and may include a wired communication module or a wireless communication module.
The virtual monitor generating unit 120 may generate a desired number of virtual monitors according to a user selection input through the user input unit 140. The virtual monitor generating unit 120 may generate a virtual monitor A virtual display device such as Microsoft, Android and the like.
The virtual reality viewer unit 130 may generate a virtual reality image so that the virtual monitor generated by the virtual monitor generation unit 120 is displayed in the virtual reality space. The virtual reality viewer unit 130 may include a 3D image generator (not shown) capable of generating a 3D image of a virtual reality space. The 3D image generating unit (not shown) may implement functions necessary for 3D image generation such as image creation, depth setting, rendering, and the like in order to display a predetermined virtual reality space. Accordingly, the virtual reality viewer unit 130 displays a virtual reality image in which the virtual monitor generated by the virtual monitor generation unit 120 is displayed in the virtual reality space generated by the 3D image generation unit (not shown) And may transmit it to the HMD device 200 through the communication unit 110. [0033] FIG. The virtual reality viewer 130 may process general image, audio, and data signals in addition to the 3D image generator (not shown), and the processing process required may include all conventional processing processes. Hereinafter, this will be described in more detail with reference to FIG.
The user input unit 140 may transmit various preset control commands or unlimited information to the control unit 170 according to the operation and input of the user. The user input unit 140 may include a menu key and an input panel installed outside the computer 100, or a keyboard, a mouse, and the like, which are connected to the computer through wire / wireless connections. Or a mobile device such as a netbook, a tablet PC, or the like, such as a touch screen or a touch pad.
The UI generating unit 150 may generate a UI for receiving the user selection for the virtual reality background, the objects included in the virtual reality background, and the background audio, and store the created UI in the storage unit 160. [
The UI generating unit 150 may generate and display various interfaces for receiving a user selection, for example, a virtual display screen setting UI, a virtual display screen viewing UI, a setting menu UI, a program introduction And a UI (see Fig. 3 (A)).
The virtual display screen setting UI may include a virtual reality background selection UI, a object selection UI, and a background audio selection UI. When the UI for viewing the virtual display screen is selected, a virtual display screen reflecting the user selection inputted through the UI for setting up the virtual display screen may be displayed to enable user confirmation. The setting menu UI may include, for example, a display screen number setting UI, a camera ON / OFF UI, a volume adjustment UI, a SAFE MODE UI, and the like. Also, the UI for program introduction is a UI for receiving various user's selections for displaying information useful for the user, such as a description related to creation of a plurality of virtual monitors, developer information, usage method, etc. according to the present invention.
The UI generator 150 will be described in more detail with reference to FIGS. 2 and 3. FIG.
The storage unit 160 may store the virtual reality background, objects included in the virtual reality background, and background audio, and may store a UI generated by the UI generation unit 150, A monitor image generated by the monitor 120 may be stored. The storage unit 160 may be implemented as a non-volatile memory such as a flash memory or a hard-disc drive. The storage unit 160 is accessed by the controller 170, and the controller 170 can read / write / modify / delete / update the stored data.
The control unit 170 may control the communication unit 110, the virtual monitor generating unit 120, the virtual reality viewer unit 130, the user input unit 140, the UI generating unit 150, and the storage unit 160 And controls the virtual monitor generating unit 120 to generate a virtual monitor corresponding to the number of the virtual monitors input through the user input unit 140, It is possible to control the virtual reality viewer unit 130 to generate a real image and to control the communication unit to transmit the generated virtual reality image to the HMD apparatus 200. [ Accordingly, the user can enjoy the virtual reality image generated by the virtual reality viewer unit 130 through the HMD device 200. [
Figure 2 illustrates one embodiment of a virtual reality image that may be implemented by the
FIG. 2 includes a left-
In addition, when a request for changing the number of the virtual monitors is input through the user input unit 140 while the transmitted virtual reality image is displayed through the HMD device 200, The virtual reality viewer unit 120, the virtual reality viewer unit 130, and the communication unit 140 so that the number of virtual monitors corresponding to the change is reflected. 3 (A) and 3 (D), the setting menu UI of FIG. 3 (A) is a display screen number setting UI of FIG. 3 (D) A camera ON / OFF UI, a volume control UI, and a SAFE MODE UI, through which a user selection input can be made.
The virtual reality image shown in FIG. 2 is being displayed through the HMD device 200, a request for changing the number of virtual monitors may be input through the user input unit 140. FIG. 2, when the user inputs a request to add two virtual monitors, the control unit 170 generates the virtual monitor generating unit 120 to generate two virtual monitors, And controls the virtual reality viewer unit 130 to generate a virtual reality image in which the two virtual monitors that are generated are located in the virtual reality space, And may control the communication unit 110 to be transmitted to the HMD device 200. [ In the case of a decrease in the number of virtual monitors, the number of virtual monitors located in the virtual reality space under the control of the controller 170 is reduced to a predetermined number of virtual reality And controls the communication unit 110 so that the updated virtual reality image is transmitted to the HMD device 200. The HMD device 200 may be configured to control the communication unit 110 to transmit the updated virtual reality image to the HMD device 200. [
Also, the virtual reality viewer unit 130 may detect a predetermined change in the virtual reality image, and generate the updated virtual reality image by reflecting the change. That is, among the plurality of monitors located within the virtual reality image, a monitor in which a change has occurred is detected through a user input unit such as a mouse, and the updated virtual reality image is generated by reflecting the change, And transmitted to the HMD device 200. When the virtual display screen view UI shown in FIG. 3A is input as a user selection, the virtual reality image updated by the virtual reality viewer unit 130 under the control of the controller 170 is changed to ). ≪ / RTI >
As shown in FIG. 2, the virtual reality space may include a virtual reality background, objects included in the virtual reality background, and background audio. Accordingly, the user can set the virtual reality space, the type and position of objects that can be placed in the virtual reality space, and the background audio. The UI generating unit 150 may generate a virtual reality background selection UI, a object selection UI, and a background audio selection UI (see (B) of FIG. 3) The virtual reality image may be generated so that the virtual reality background, the type and position of the object selected through the background image, and the background audio are reflected.
For example, when the virtual reality background selection UI is selected, the virtual reality background image is displayed as a sub UI or a thumbnail image that can select various virtual reality background images such as an office image, a study image, a beach image, .
For example, when the object selection UI is selected, a sub UI for selecting various types of object images, such as the desk, a chair, a monitor, a frame, and a lighting lamp, or a thumbnail image, have.
For example, when the background audio selection UI is selected, a list of sub-UIs or audio files from which various audio sources can be selected is displayed, and the desired background audio can be selected by the user.
The virtual reality background image, the object image, and audio of various sources are stored in the storage unit 160, and the user-selected data can be read by the control unit 170. [
The HMD device 200 may include a display unit (not shown), an audio output unit (not shown), and a sensor unit (not shown). The HMD device 200 of the present invention can be realized by a conventional HMD device, and motion tracking / head tracking is possible through the sensor unit (not shown). A signal corresponding to the detected motion / head position of the user is received by the communication unit 110, and the controller 170 generates a signal based on the signal corresponding to the received motion / The virtual reality viewer 130 may control the virtual reality viewer 130 to generate an image corresponding to the virtual reality area in which the user's gaze stays and transmit the generated image to the HMD device 200. At this time, the image corresponding to the virtual reality area of the area where the user's gaze does not stay may not be transmitted to the HMD device 200.
On the other hand, when the user wears the HMD device 200, the user can only view the image displayed on the HMD device 200, and can not gaze at the real space outside the HMD device 200. In this case, when the virtual monitor in the virtual reality image displayed on the HMD device 200 is operated, when the user inputs a user selection through the user input unit 140 located in a real space such as a keyboard, So that it is necessary to deal with the keyboard only by the sense of the hand.
In order to solve this problem, the HMD device 200 may further include a camera (not shown), and the HMD device 200 may transmit an image captured by the camera (not shown) And the virtual reality viewer unit 130 generates a second virtual reality image included in the virtual reality image by using the image captured by the camera and transmits the generated second virtual reality image to the HMD device 200 ). ≪ / RTI > This is illustrated in FIG. 4 shows an embodiment of a virtual reality image in which an image captured by a camera of the HMD device 200 is displayed together.
As shown in FIG. 4, virtual monitors 31 ', 33', and 35 'are displayed in the virtual reality image represented by the left eye image 3' and the right eye image 5 ' A second virtual reality image in which
In addition, the sensor unit (not shown) of the HMD device 200 may further include a gesture recognition sensor, and may transmit a gesture signal sensed by the gesture sensor to the communication unit 110, Recognizes the received gesture signal as a user selection, and controls the virtual reality viewer unit 130 to generate a virtual reality image that reflects a user selection corresponding to the gesture signal. See FIG. 5 illustrates an embodiment of a user selection corresponding to a gesture signal recognized by the HMD device 200. As shown in FIG.
5, when the wave of the user's hand is recognized by the gesture recognition sensor of the HMD device 200 in the case of FIG. 5A, when the wave signal is transmitted to the communication unit 110, Recognizes the wave signal as the start of display of the virtual reality image and controls the virtual reality viewer unit 130 to generate a corresponding virtual reality image, and the generated virtual reality image signal is transmitted to the communication unit 110 And then transmitted to the HMD device 200 for display.
5 (B) and 5 (C) are gesture signals corresponding to the movement of the virtual monitor, and Fig. 5 (B) shows that when the gesture signal is moved in the left or right direction with one hand or both hands, Left or right direction. In case (C), when the rotational motion is detected by one hand or both hands, the gesture signal can tilt the direction of the virtual monitor at a predetermined angle.
5D and 5E show a gesture signal corresponding to the disappearance of the virtual monitor. When a specific virtual monitor is selected and the hand is detected in the downward direction (D) or pressed (E) Monitor can be extinguished.
FIG. 5 (F) shows that when a virtual monitor is selected, it can be highlighted in the virtual monitor.
5 (G) and 5 (H) show a case where a virtual monitor is selected as a gesture signal corresponding to a change in the size of a virtual monitor, and a hand movement is not detected for a predetermined time (for example, 2 to 3 seconds ), Or when a motion that pulls both hands diagonally is detected (H), the size of the virtual monitor can be enlarged.
5 (I) shows that all the virtual monitors generated by the user selection are arranged 360 degrees around the user, and when the user detects the movement of the pushing hand, all the virtual monitors rotate in the direction of the movement .
The examples of FIGS. 5B to 5I illustrate a case where a series of gesture signals are sensed by the gesture recognition sensor and when the series of gesture signals are received through the communication unit 110, Command is executed and is reflected in the virtual reality image, and the virtual reality viewer unit 130 may be transmitted to the HMD device 200 through the communication unit 110 and displayed.
Although several embodiments of the present invention have been shown and described, those skilled in the art will readily appreciate that many modifications may be made without departing from the spirit or scope of the invention . The scope of the invention will be determined by the appended claims and their equivalents.
1: Display system
100: Computer
110:
120: Virtual monitor generation unit
130: Virtual Reality Viewer
140: user input section
150: UI generation unit
160:
170:
200: HMD device
Claims (8)
An HMD device for displaying a virtual reality image,
A virtual monitor generating unit for generating a virtual monitor; and a display unit for displaying the virtual monitor generated by the virtual monitor generating unit in the virtual reality space so as to be displayed in the virtual reality space, And a control unit for controlling the virtual monitor generating unit to generate a virtual monitor corresponding to the number of the virtual monitors inputted through the user input unit, so that the generated virtual monitor is displayed in the virtual reality space And a control unit for controlling the virtual reality viewer unit to generate a virtual reality image and controlling the communication unit to transmit the generated virtual reality image to the HMD device.
Wherein,
Wherein the virtual monitor generating unit is configured to, when a request for changing the number of the virtual monitors is inputted through the user input unit while the transmitted virtual reality image is displayed through the HMD device, And controls the virtual reality viewer unit and the communication unit.
Wherein the virtual reality viewer unit detects a predetermined change in the virtual reality image and generates an updated virtual reality image by reflecting the change.
The virtual reality space includes:
A virtual reality background, objects included in the virtual reality background, and background audio,
Further comprising: a UI generation unit for generating a UI capable of receiving the user selection for the virtual reality background, the object, and the background audio.
The UI generation unit may generate,
A virtual reality background selection UI, a object selection UI, and a background audio selection UI,
Wherein the virtual reality viewer unit generates the virtual reality image so that the virtual reality background, the type and position of the object selected through the UI, and the background audio are reflected.
A plurality of virtual reality backgrounds selectable by the user via the virtual reality background selection UI, a plurality of types of objects selectable by the user through the object selection UI, and a plurality of audio items selectable by the user via the background audio selection UI And a storage unit for storing the image data.
Wherein the HMD device further comprises a camera,
Wherein the virtual reality viewer unit generates a second virtual reality image included in the virtual reality image, the image being captured by the camera.
Wherein the HMD device further comprises a gesture recognition sensor, and the gesture signal sensed by the gesture sensor is transmitted to the communication unit,
Wherein the control unit recognizes the received gesture signal as a user selection and controls the virtual reality viewer unit to generate a virtual reality image reflecting a user selection corresponding to the gesture signal.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020140184437 | 2014-12-19 | ||
| KR20140184437 | 2014-12-19 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| KR20160075315A true KR20160075315A (en) | 2016-06-29 |
Family
ID=56365797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020150162824A KR20160075315A (en) | 2014-12-19 | 2015-11-19 | Display system for generating virtual reality image |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR20160075315A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073555A (en) * | 2016-11-17 | 2018-05-25 | 奥多比公司 | For generating the method and system of reality environment from electronic document |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR19990021535A (en) | 1997-08-30 | 1999-03-25 | 윤종용 | Personal Computer with HMD and Eye Angle Recognition |
-
2015
- 2015-11-19 KR KR1020150162824A patent/KR20160075315A/en not_active Application Discontinuation
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR19990021535A (en) | 1997-08-30 | 1999-03-25 | 윤종용 | Personal Computer with HMD and Eye Angle Recognition |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073555A (en) * | 2016-11-17 | 2018-05-25 | 奥多比公司 | For generating the method and system of reality environment from electronic document |
| CN108073555B (en) * | 2016-11-17 | 2023-10-31 | 奥多比公司 | Method and system for generating virtual reality environment from electronic document |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11768579B2 (en) | Devices, methods, and graphical user interfaces for interacting with three-dimensional environments | |
| CN110476142B (en) | Computing device, method and head mounted display device for displaying virtual content | |
| US9704285B2 (en) | Detection of partially obscured objects in three dimensional stereoscopic scenes | |
| US20210097776A1 (en) | Devices, Methods, and Graphical User Interfaces for Interacting with Three-Dimensional Environments | |
| US20220091722A1 (en) | Devices, methods, and graphical user interfaces for interacting with three-dimensional environments | |
| US11557102B2 (en) | Methods for manipulating objects in an environment | |
| US10133345B2 (en) | Virtual-reality navigation | |
| US10854169B2 (en) | Systems and methods for virtual displays in virtual, mixed, and augmented reality | |
| WO2021133572A1 (en) | Devices, methods, and graphical user interfaces for displaying applications in three-dimensional environments | |
| JP5914739B1 (en) | Program to control the head mounted display system | |
| EP3646143A1 (en) | Selection using a multi-device mixed interactivity system | |
| EP3646142A1 (en) | Annotation using a multi-device mixed interactivity system | |
| JP2017187952A (en) | Display control method and program for causing computer to execute the method | |
| US20230333646A1 (en) | Methods for navigating user interfaces | |
| JP6121496B2 (en) | Program to control the head mounted display system | |
| KR102590132B1 (en) | Display device and controlling method thereof | |
| CN114746796A (en) | Dynamic browser stage | |
| KR20160075315A (en) | Display system for generating virtual reality image | |
| TW202328872A (en) | Metaverse content modality mapping | |
| JP2016018363A (en) | Game program for display-controlling object arranged on virtual space plane | |
| WO2020188721A1 (en) | Head-mounted information processing device and head-mounted display system | |
| JP6121495B2 (en) | Program to control the head mounted display system | |
| US20240111390A1 (en) | Translating Interactions on a Two-Dimensional Interface to an Artificial Reality Experience | |
| JP2022025476A (en) | Animation production system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A201 | Request for examination | ||
| E902 | Notification of reason for refusal | ||
| E601 | Decision to refuse application |