CN106990838B - Method and system for locking display content in virtual reality mode - Google Patents
Method and system for locking display content in virtual reality mode Download PDFInfo
- Publication number
- CN106990838B CN106990838B CN201710157596.XA CN201710157596A CN106990838B CN 106990838 B CN106990838 B CN 106990838B CN 201710157596 A CN201710157596 A CN 201710157596A CN 106990838 B CN106990838 B CN 106990838B
- Authority
- CN
- China
- Prior art keywords
- matrix
- display content
- scene
- opengl
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention discloses a method and a system for locking display content in a virtual reality mode, wherein the method comprises the following steps: when the VR equipment worn by a user is detected to rotate, reading gyroscope data arranged in the VR equipment in real time; calculating to obtain a scene rotation matrix and a change matrix of the display content to be locked according to the gyroscope data; and submitting the scene rotation matrix and the matrix of the display content to be locked to OpenGL ES for rendering, and entering the rotated virtual reality scene. The invention realizes the locking of the specific display content, when the VR equipment worn on the head of the user rotates, the virtual reality surrounding scene part still changes along with the head, and the specific display part does not leave the visual field of the user due to the rotation of the head, thereby bringing great convenience for the user to watch the content of the specific display part through the locking mode.
Description
Technical Field
The invention relates to the technical field of virtual reality, in particular to a method and a system for locking display contents in a virtual reality mode.
Background
Virtual Reality (VR) technology is a computer simulation system that can create and experience a Virtual world, and it uses computer simulation to generate a three-dimensional Virtual world, providing simulation of sense organs such as vision, hearing, touch, etc. for a user, and making the user personally on the scene. The implementation of virtual reality usually requires the assistance of a head-mounted display, and of course, the smart phone may also implement an entry-level virtual reality experience through an external device, such as Google Cardboard and samsung Gear VR.
The user can see the virtual scene calculated by the computer in real time through the screen in the virtual reality experience time, and the picture of the virtual scene can change along with the rotation of the head of the user, so that the user can see the scene in each peripheral direction by adjusting the head. This is appropriate for most usage scenarios, however in some specific scenarios the user needs to lock some display content, such as when the user is watching 2D video using a VR device, if the video display part is not locked, the user's head cannot see the complete video once the head is deflected in the direction.
Accordingly, the prior art is yet to be improved and developed.
Disclosure of Invention
In view of the foregoing deficiencies of the prior art, an object of the present invention is to provide a method and a system for locking display contents in a virtual reality mode, which aim to solve the problem that the existing virtual reality mode cannot lock specific display contents and is inconvenient for a user to view the specific display contents.
The technical scheme of the invention is as follows:
a method for locking display content in a virtual display mode comprises the following steps:
A. when the VR equipment worn by a user is detected to rotate, reading gyroscope data arranged in the VR equipment in real time;
B. calculating to obtain a scene rotation matrix and a matrix of the display content to be locked according to the gyroscope data;
C. and submitting the scene rotation matrix and the change matrix of the display content to be locked to OpenGL ES for rendering, and entering the rotated virtual reality scene.
Preferably, the method for locking display content in the virtual display mode, wherein the step a includes:
a0, initializing the OpenGL ES environment and the gyroscope.
Preferably, the method for locking display content in the virtual display mode, wherein the step B specifically includes:
b1, calculating according to the gyroscope data to obtain a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix;
and B2, calculating to obtain a scene rotation matrix according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix.
Preferably, the method for locking display content in the virtual display mode, wherein the step B further includes:
b3, initializing shaders of OpenGL ES, and reading the environment and the texture of the object.
Preferably, the method for locking display content in the virtual display mode, wherein the step C specifically includes:
c1, the environment renderer calls OpenGL ES to render the scene rotation matrix;
and C2, calling OpenGL ES by the object renderer to render the change matrix with the locked display content.
A system for locking display content in a virtual display mode, comprising:
the reading module is used for reading gyroscope data arranged in the VR equipment in real time when the VR equipment worn by a user is detected to rotate;
the calculation module is used for calculating to obtain a scene rotation matrix and a change matrix of the display content to be locked according to the gyroscope data;
and the rendering module is used for submitting the scene rotation matrix and the change matrix of the display content to be locked to OpenGL ES for rendering and entering the rotated virtual reality scene.
Preferably, the system for locking display content in the virtual display mode further comprises:
and the initialization module is used for initializing both the OpenGL ES environment and the gyroscope.
Preferably, the system for locking display content in the virtual display mode, wherein the computing module specifically includes:
the first calculation unit is used for calculating to obtain a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to the gyroscope data;
and the second calculation unit is used for calculating a scene rotation matrix according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix.
Preferably, the system for locking display content in the virtual display mode, wherein the computing module further comprises:
and the reading unit is used for carrying out initialization processing on the shader of the OpenGL ES and reading the environment and the texture of the object.
Preferably, the system for locking display content in the virtual display mode, wherein the rendering module specifically includes:
the environment renderer is used for calling OpenGL ES and rendering the scene rotation matrix;
and the object renderer is used for calling OpenGL ES and rendering the matrix with the locked display content.
Has the advantages that: the method and the system for locking the display content in the virtual reality mode realize the locking of the specific display content, when the VR equipment worn by the head of the user rotates, the scene part around the virtual reality still changes along with the head, and the specific display part does not leave the visual field of the user due to the rotation of the head, so that great convenience is brought to the user for watching the content of the specific display part through the locking mode.
Drawings
FIG. 1 is a flowchart illustrating a method for locking display contents in a virtual reality mode according to a preferred embodiment of the present invention.
FIG. 2 is a block diagram of a system for locking display content in a virtual reality mode according to a preferred embodiment of the present invention.
Detailed Description
The present invention provides a method and system for locking display content in a virtual reality mode, and the following further describes the present invention in detail in order to make the purpose, technical scheme and effect of the present invention clearer and clearer. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for locking display contents in a virtual reality mode according to a preferred embodiment of the present invention, as shown in the figure, the method includes the following steps:
s100, reading gyroscope data arranged in VR equipment in real time when the VR equipment worn by a user is detected to rotate;
s200, calculating according to the gyroscope data to obtain a scene rotation matrix and a matrix of display content to be locked;
and S300, submitting the scene rotation matrix and the change matrix of the display content to be locked to OpenGL ES for rendering, and entering the rotated virtual reality scene.
The invention realizes the locking of the specific display content in the virtual reality mode, namely, when the VR equipment worn by the head of the user rotates, the unlocked part of the virtual reality surrounding scene still changes along with the rotation of the head, but the specific display part does not leave the visual field of the user due to the rotation of the head, thereby bringing great convenience for the user to watch the content of the specific display part through the locking mode.
For example, when a user watches a 2D video by using a VR device, if a video display part is set at a middle fixed position of a virtual reality scene, if the video display part is locked, the video display part is always located at the middle fixed position of the virtual display scene no matter the VR device worn by the user rotates, and no corresponding position change is performed due to the rotation of the VR device worn by the user, so that the user can always see the video at the middle fixed position of the virtual display scene, thereby bringing great convenience to the user.
Further, in the present invention, the step S100 includes, before:
s10, initializing both an OpenGL ES environment and a gyroscope, specifically, initializing shaders of the OpenGL ES environment, and reading textures of the environment and an object; further, a sensor module is arranged in the VR equipment, and the gyroscope is initialized through the sensor module to prepare for reading gyroscope data.
Further, in the present invention, the step S100 specifically includes:
s110, calculating to obtain a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to the gyroscope data;
specifically, in a virtual reality mode, a user can adjust the angle of watching a video by rotating VR equipment at the head, and a sensor module in the VR equipment is responsible for initializing a gyroscope, reading data of the gyroscope in real time and sending the data to a rendering module in the VR equipment;
specifically, the rendering module performs rendering through OpenGL ES, and virtual reality requires a user to see two similar pictures with slight viewing angle difference by the left and right eyes, so that a mobile phone screen needs to be divided into a left part and a right part for rendering respectively; the rendering module comprises two sub-modules which are an environment renderer and an object renderer respectively, and the two sub-modules calculate matrixes respectively according to data provided by the gyroscope; the environment renderer calculates a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to gyroscope data; further, the object renderer calculates a matrix of the display content to be locked according to the fixed matrix.
And S120, calculating to obtain a scene rotation matrix according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix.
Further, the environment renderer calculates a scene rotation matrix according to the VR device rotation matrix, the view transformation matrix and the clipping matrix.
Further, in the present invention, the step S300 specifically includes:
s310, the environment renderer calls OpenGL ES to render the scene rotation matrix;
and S320, the object renderer calls OpenGL ES to render the matrix of the display content to be locked.
Specifically, the environment renderer binds coordinates and textures in advance, and invokes OpenGL ES to render the scene rotation matrix to obtain a rotated virtual reality environment; and simultaneously, the object renderer calls OpenGL ES to render a matrix of the display content to be locked, and a display picture of the display content to be locked in the rotated virtual reality environment is obtained.
Based on the above method, the present invention further provides a system for locking display content in a virtual display mode, as shown in fig. 2, wherein the system comprises:
the reading module 100 is configured to read gyroscope data set in VR equipment in real time when it is detected that the VR equipment worn by a user rotates;
the calculation module 200 is configured to calculate a scene rotation matrix and a change matrix of the to-be-locked display content according to the gyroscope data;
the rendering module 300 is configured to submit the scene rotation matrix and the change matrix of the to-be-locked display content to OpenGL ES for rendering, and enter the rotated virtual reality scene.
Preferably, the system for locking display content in the virtual display mode further comprises:
and the initialization module is used for initializing both the OpenGL ES environment and the gyroscope.
Preferably, the system for locking display content in the virtual display mode, wherein the computing module specifically includes:
the first calculation unit is used for calculating to obtain a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to the gyroscope data;
and the second calculation unit is used for calculating a scene rotation matrix according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix.
Preferably, the system for locking display content in the virtual display mode, wherein the computing module further comprises:
and the reading unit is used for carrying out initialization processing on the shader of the OpenGL ES and reading the environment and the texture of the object.
Preferably, the system for locking display content in the virtual display mode, wherein the rendering module specifically includes:
the environment renderer is used for calling OpenGL ES and rendering the scene rotation matrix;
and the object renderer is used for calling OpenGL ES and rendering the matrix with the locked display content.
In summary, the method and the system for locking the display content in the virtual reality mode provided by the invention realize the locking of the specific display content, when the VR device worn on the head of the user rotates, the scene part around the virtual reality still changes along with the head, and the specific display part does not leave the visual field of the user due to the rotation of the head, so that great convenience is brought to the user for watching the content of the specific display part through the locking mode.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (6)
1. A method for locking display content in a virtual display mode is characterized by comprising the following steps:
a0, initializing shaders of OpenGL ES, reading textures of an environment and an object, and initializing a gyroscope through a sensor module in VR equipment;
A. when the VR equipment worn by a user is detected to rotate, reading gyroscope data arranged in the VR equipment in real time;
B. calculating to obtain a scene rotation matrix and a matrix of the display content to be locked according to the gyroscope data;
C. submitting the scene rotation matrix and the change matrix of the display content to be locked to OpenGL ES for rendering, and entering a rotated virtual reality scene;
the step B specifically comprises the following steps:
b1, calculating a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to gyroscope data;
b2, calculating according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix to obtain a scene rotation matrix;
and calculating a matrix of the display content to be locked according to the fixed matrix.
2. The method for locking display contents in the virtual display mode according to claim 1, wherein the step B further comprises:
b3, initializing shaders of OpenGL ES, and reading the environment and the texture of the object.
3. The method for locking display content in the virtual display mode according to claim 1, wherein the step C specifically includes:
c1, the environment renderer calls OpenGL ES to render the scene rotation matrix;
and C2, calling OpenGL ES by the object renderer to render the matrix of the display content to be locked.
4. A system for locking display content in a virtual display mode, comprising:
the initialization module is used for initializing the shader of the OpenGL ES, reading the environment and the texture of an object, and initializing the gyroscope through a sensor module in VR equipment;
the reading module is used for reading gyroscope data arranged in the VR equipment in real time when the VR equipment worn by a user is detected to rotate;
the calculation module is used for calculating to obtain a scene rotation matrix and a change matrix of the display content to be locked according to the gyroscope data; the matrix is also used for calculating the display content to be locked according to the fixed matrix;
the rendering module is used for submitting the scene rotation matrix and the change matrix of the display content to be locked to OpenGL ES for rendering, and entering a rotated virtual reality scene;
the calculation module specifically includes:
the first calculation unit is used for calculating to obtain a VR equipment rotation matrix, a visual angle transformation matrix and a cutting matrix according to the gyroscope data;
and the second calculation unit is used for calculating a scene rotation matrix according to the VR equipment rotation matrix, the visual angle transformation matrix and the cutting matrix.
5. The system for locking display content in a virtual display mode according to claim 4, wherein the computing module further comprises:
and the reading unit is used for carrying out initialization processing on the shader of the OpenGL ES and reading the environment and the texture of the object.
6. The system for locking display content in a virtual display mode according to claim 4, wherein the rendering module specifically comprises:
the environment renderer is used for calling OpenGL ES and rendering the scene rotation matrix;
and the object renderer is used for calling OpenGL ES and rendering the matrix of the display content to be locked.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710157596.XA CN106990838B (en) | 2017-03-16 | 2017-03-16 | Method and system for locking display content in virtual reality mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710157596.XA CN106990838B (en) | 2017-03-16 | 2017-03-16 | Method and system for locking display content in virtual reality mode |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106990838A CN106990838A (en) | 2017-07-28 |
| CN106990838B true CN106990838B (en) | 2020-11-13 |
Family
ID=59412127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710157596.XA Active CN106990838B (en) | 2017-03-16 | 2017-03-16 | Method and system for locking display content in virtual reality mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106990838B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114265496A (en) * | 2021-11-30 | 2022-04-01 | 歌尔光学科技有限公司 | VR scene switching method and device, VR head-mounted equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183142A (en) * | 2014-06-13 | 2015-12-23 | 中国科学院光电研究院 | Digital information reproduction method by means of space position nailing |
| CN105447898A (en) * | 2015-12-31 | 2016-03-30 | 北京小鸟看看科技有限公司 | Method and device for displaying 2D application interface in virtual real device |
| CN105915877A (en) * | 2015-12-27 | 2016-08-31 | 乐视致新电子科技（天津）有限公司 | Free film watching method and device of three-dimensional video |
| CN105979360A (en) * | 2015-12-04 | 2016-09-28 | 乐视致新电子科技（天津）有限公司 | Rendering image processing method and device |
| CN106249918A (en) * | 2016-08-18 | 2016-12-21 | 南京几墨网络科技有限公司 | Virtual reality image display packing, device and apply its terminal unit |
-
2017
- 2017-03-16 CN CN201710157596.XA patent/CN106990838B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183142A (en) * | 2014-06-13 | 2015-12-23 | 中国科学院光电研究院 | Digital information reproduction method by means of space position nailing |
| CN105979360A (en) * | 2015-12-04 | 2016-09-28 | 乐视致新电子科技（天津）有限公司 | Rendering image processing method and device |
| CN105915877A (en) * | 2015-12-27 | 2016-08-31 | 乐视致新电子科技（天津）有限公司 | Free film watching method and device of three-dimensional video |
| CN105447898A (en) * | 2015-12-31 | 2016-03-30 | 北京小鸟看看科技有限公司 | Method and device for displaying 2D application interface in virtual real device |
| CN106249918A (en) * | 2016-08-18 | 2016-12-21 | 南京几墨网络科技有限公司 | Virtual reality image display packing, device and apply its terminal unit |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106990838A (en) | 2017-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230082705A1 (en) | Reprojecting holographic video to enhance streaming bandwidth/quality | |
| US9582922B2 (en) | System, method, and computer program product to produce images for a near-eye light field display | |
| EP3057066B1 (en) | Generation of three-dimensional imagery from a two-dimensional image using a depth map | |
| US20160267720A1 (en) | Pleasant and Realistic Virtual/Augmented/Mixed Reality Experience | |
| CN112933599B (en) | Three-dimensional model rendering method, device, equipment and storage medium | |
| US20170186219A1 (en) | Method for 360-degree panoramic display, display module and mobile terminal | |
| CN110546951B (en) | Composite stereoscopic image content capture | |
| US10999412B2 (en) | Sharing mediated reality content | |
| US20190259198A1 (en) | Systems and methods for generating visual representations of a virtual object for display by user devices | |
| US11662580B2 (en) | Image display method, apparatus, and system to reduce display latency | |
| WO2023056840A1 (en) | Method and apparatus for displaying three-dimensional object, and device and medium | |
| WO2017113729A1 (en) | 360-degree image loading method and loading module, and mobile terminal | |
| JP2018504009A (en) | Digital video rendering | |
| CN107562185B (en) | Light field display system based on head-mounted VR equipment and implementation method | |
| CN115868158A (en) | Display adjusting method, device, equipment and medium | |
| CN114419226A (en) | Panorama rendering method and device, computer equipment and storage medium | |
| US20190295324A1 (en) | Optimized content sharing interaction using a mixed reality environment | |
| CN106990838B (en) | Method and system for locking display content in virtual reality mode | |
| US20230103814A1 (en) | Image Processing Systems and Methods | |
| Pérez et al. | Geometry-based methods for general non-planar perspective projections on curved displays | |
| US10715783B1 (en) | Stereo-aware panorama conversion for immersive media | |
| CN117591054A (en) | Display method, head-mounted display device, computing device, and storage medium | |
| CN118037766A (en) | Image processing method and device | |
| CN111862338A (en) | Display method and device for simulating glasses wearing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220121 Address after: 518000 a501, building D4, TCL Science Park, No. 1001, Zhongshan Park Road, Shuguang community, Xili street, Nanshan District, Shenzhen, Guangdong Patentee after: Thunderbird innovation technology (Shenzhen) Co.,Ltd. Address before: 516006 Zhongkai hi tech Zone, Huizhou, Guangdong, 86 Chang seven Road West Patentee before: HUIZHOU TCL MOBILE COMMUNICATION Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230921 Address after: No.8-1-2, Lane 999, Yangfan Road, High tech Zone, Ningbo City, Zhejiang Province, 315048 Patentee after: Thunderbird Innovation Technology (Ningbo) Co.,Ltd. Address before: 518000 a501, building D4, TCL Science Park, No. 1001, Zhongshan Park Road, Shuguang community, Xili street, Nanshan District, Shenzhen, Guangdong Patentee before: Thunderbird innovation technology (Shenzhen) Co.,Ltd. |
|
| TR01 | Transfer of patent right |