CN104915979A - System capable of realizing immersive virtual reality across mobile platforms - Google Patents
System capable of realizing immersive virtual reality across mobile platforms Download PDFInfo
- Publication number
- CN104915979A CN104915979A CN201410084307.4A CN201410084307A CN104915979A CN 104915979 A CN104915979 A CN 104915979A CN 201410084307 A CN201410084307 A CN 201410084307A CN 104915979 A CN104915979 A CN 104915979A
- Authority
- CN
- China
- Prior art keywords
- server end
- client
- mobile
- head
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a system capable of realizing immersive virtual reality across mobile platforms. A DirectX3D rendering technology is adopted to perform distortion correction and distortion scale on images rendered by virtual three-dimensional scenes via a Pixel Shader, the images are segmented and drawn on left-eye and right-eye screens of HMD equipment, and a refresh frequency of more than 60 frames per second is kept; position information and facing information of the HMD equipment are acquired from a sensor and used for head tracking, and coordinate and rotating quaternion information are fed back to an application program via a message interface; and interaction between a client end and a server end is realized by adopting a wireless communication technology. The system provided by the invention has the advantages that one product only needs to be developed for once, reality is enhanced, virtual objects not existing in the real environment are generated via a computer graphic technology and a visualization technology, the virtual objects are accurately 'placed' in the real environment via the sensing technology and are integrally mixed with the real environment by the aid of displaying equipment.
Description
Technical field
The present invention relates to virtual reality technology.
Background technology
Virtual reality, be called for short VR technology, be utilize computer simulation to produce a three-dimensional virtual world, provide user about the simulation of the sense organs such as vision, the sense of hearing, sense of touch, allow user as on the spot in person, the things in three-dimensional space can be observed in time, ad lib.
At present, worldwide, mainly contain two to the engine that mobile Internet and virtual reality are supported simultaneously, they are the Unreal3 engine of Epic company of the U.S. and the Unity3D engine of Unity Technology company of Denmark respectively.
Unreal3 engine is the tap of conventional P C and household game machine platform, and thousands of game and virtual reality applications use this engine to develop.When technical development is to the mobile Internet epoch, Unreal3 engine has also issued corresponding engine authorizes for mobile platform.But Unreal3 engine is designed to high-end engine from the beginning, no matter from complexity, the expressive ability of engine or from the price of authorizing, the technology of user itself and hardware configuration requirement are all very harsh.That only has the big companies of minority and listed company to bear plays this engine.He is very slow in mobile platform development always in occupation of the high-end market of exploitation.
Unity3D engine develops the cross-platform engine of 3D rapidly in nearly 2 years, he towards be middle-end developing target market, existing market occupation rate occupies high.
But no matter this two engine is from designing or in service, not customizing for Chinese developer.China developer has inconvenience in use, and for Unity3D, this engine Chinese document is very unsound, and the upper acquiescence of literary composition input is not also supported in the implementation, and developer, when encountering problems, cannot be solved fast and effectively.When running into engine design defect, due to official's not providing source code, developer is by helpless.
At home, 3D virtual reality engine software best is at present the VRP developed by Shenzhen Zhong Shidian digital technology company limited.VRP is a directly towards the Chinese virtual reality software platform of three-dimensional art designing, and its core looks the virtual reality 3D engine of allusion quotation independent research in being.Its all operation is all carry out in the understandable mode of art designing, does not need programmer to participate in.If operator has good 3DMAX modeling and plays up basis, as long as learn a little VRP platform and study the virtual reality scenario just can producing oneself very soon.But the virtual reality 3D engine of VRP and external virtual reality engine gap in technical merit is very large: support point is not painted, and UV layer is only supported two-layer.Do not support UV animation, want that realizing flow effect very wastes pinup picture, pinup picture stacked system is few.And whole platform does not all provide any programmer the interface that can get involved to client, do not support the plug-in unit that any client writes oneself etc. defect also allow VRP with the competition of external engine at a disadvantage.
Summary of the invention
The present invention is the defect overcoming above-mentioned technology, provides a kind of implementation realizing immersive VR across mobile platform.
Realize a system for immersive VR across mobile platform, comprise server end and client, wherein, reality scene is played up by described server end, and the display device transferring to client shows.
Further, wherein client comprises head-mounted display apparatus and mobile client, and server end comprises virtual reality engine; Mobile client is arranged on the hand-held Intelligent mobile equipment of user, described Intelligent mobile equipment is undertaken alternately by radio communication and server end, mobile device is by the input instruction of interactive operation emulating server end, control virtual scene or personage, server end feedback command is to mobile device.
Further, virtual reality engine, the picture adopting DirectX3D Rendering to be rendered by virtual three-dimensional scene by pixel rendering device Pixel Shader carries out distortion correction and distortion convergent-divergent, and segmentation is plotted to the left and right eyes screen of described head-mounted display apparatus, refreshes with frequencies more than 60 frames per second.
Further, described system also comprises one or more sensor, for immediately obtaining positional information and the orientation information of head-mounted display apparatus, carries out the tracking in head movement and eyes direction; Described sensor be accelerometer, gyroscope, compass or direction sensor at least one of them; Described system also comprises message interface, for the position coordinates that obtained by described sensor with rotate hypercomplex number information feed back to the application program operating in server end.
Further, described application program is by the processing module of described virtual reality engine, process is played up, physical impacts, audio effect processing process feedack, coordinates matrix is set, the process of image is carried out according to feedack, the image processed is transferred to client by the hardware abstraction layer HAL of head-mounted display apparatus, the picture that user upgrades after seeing feedback by eyes.
Further, described mobile device is undertaken by Wi-Fi or bluetooth and server end alternately.
Further, the interactive operation that described mobile device performs can be Touch Screen or dummy keyboard operation realizes.
Further, server end feed back to mobile device instruction can by vibrations and/or play sound effect perform.
Further, server end can be PC or Cloud Server.
The present invention also provides a kind of system realizing immersive VR across mobile platform, and this system comprises server end and client, wherein,
Server end comprises:
Message interface, for interactive information between client; Processing module, for the treatment of the information of client feedback, and sends it back client by the result after process; Image rendering module, carries out 3D to image and plays up, and generates 3D scene; Pixel rendering module, carries out distortion correction and distortion convergent-divergent to the 3D scene generated; Segmentation drafting module, draws the Image Segmentation Using played up; Image output module, exports to the image split on the screen of the head-mounted display apparatus of client.
Client comprises head-mounted display apparatus and mobile client.
Wherein head-mounted display apparatus comprises:
Display screen, shows for the image exported by server end, and to make user to watch, it is per second that the refreshing frequency of described display screen is not less than 60 frames; Sensor, for immediately obtaining positional information and the orientation information of head-mounted display apparatus, carries out the tracking in head movement and eyes direction; Message interface, for server end interactive information; Processing module, for the treatment of the information that sensor obtains, is translated into corresponding position coordinates and rotates hypercomplex number information, and the result after transforming is sent to server end.
Mobile client is installed on the mobile apparatus, comprising: control module, by arranging virtual input device on the screen of the mobile device, by the input instruction of interactive operation emulating server end, controls virtual scene or personage; Processing module, for the treatment of the information received by mobile device, and is sent to corresponding server end by the result after process; Message interface, for server end interactive information; Wireless connections module, for setting up wireless connections with server.
The invention has the beneficial effects as follows:
1, a product only need be developed once, and just can be distributed on and comprise PC, iOS, Android, Windows Phone etc. are multi-platform.Developer need not consider the details that each platform interior constructs completely, only need understand the exploitation rule of engine.
2, augmented reality computer graph technology and visualization technique produce non-existent virtual objects in actual environment, and by sensing technology by virtual objects accurately " placement " in true environment, by display device, virtual objects and true environment are combined together.
3, for hall-room class server application, a set of Cloud Server solution is provided.
4, the mutual of Intelligent mobile equipment and computer equipment is realized by Wi-Fi technology.
Accompanying drawing explanation
Fig. 1 is the system construction drawing realizing immersive VR across mobile platform.
Fig. 2 is the scene that the segmentation seen on head-mounted display apparatus HMD is drawn.
Fig. 3 is the exemplary plot being realized Intelligent mobile equipment and computer equipment interactive module by wireless communication technology.
Embodiment
With regard to characteristic and performance, OpenGL and Direct3D two kinds of 3D API can say and respectively have superiority, as OpenGL has good professional platform independence and hardware independence, and the feature development that the Immediate Mode programming that Direct3D provides allows application program to make full use of 3D hardware goes out more high performance 3D application program, this programming mode is adopted by many high performance 3 d application developers.Consider the actual conditions that order first two 3D API uses, develop better performances and support that the application program of two kinds of 3D API is just extremely important simultaneously.
As shown in Figure 1, the invention provides a kind of system realizing immersive VR across mobile platform, it comprises server end and client, and reality scene is played up by described server end, and the display device transferring to client shows.Described server end comprises virtual reality engine.Described client comprises head-mounted display apparatus HMD, is configured to be expanded the realistic space merged by three-dimensional virtual object by 360 degree of full views watchings.
Cross-platform Rendering:
Engine of the present invention encapsulates all details in first floor system storehouse (Direct3D and OpenGL), and be PC, IOS, the systems such as Android and Windows phone provide hardware device and the graphic interface that corresponding interface (hardware abstraction layer HAL) carrys out adaptive different platform, make it cross-platformly easily to realize multiple 3D characteristic.As: Terrain (landform), Water (water surface effect), SkyBox (sky sylphon system) Vegetation (vegetation system), Reflection (reflection), refraction (refraction) and Shader advanced feature.As: Bloom (full frame floodlight), HDR (High-Dynamic Range high dynamic illumination is played up), Depth offield (depth of field), Motion blur (motion blur) etc.
Engine of the present invention mainly comprises following several part: Render System panorama rendering module, Scene Manager scene management, Network System network service, Physical simulation physical simulation, Material-Texture material texture, Shader effect, I/O input and output, GUI user interface, AR (Augmented Reality) augmented reality.
Mobile device physical simulation and operating characteristic:
The present invention meets mobile device to three of physical engine major requirements: engine lightweight, cpu resource consumption less, mobile device functionality.On this basis, except Collision Detection (collision detection) can be realized, Ray Detection (ray detection), Gravity (gravity), outside the basic functions such as Joint (association's simulation) Car (vehicle simulation), also support that the function of mobile device is as Gyroscope (gyrostat compass), gravity sensing sensor.
AR (Augmented Reality) augmented reality:
The present invention uses camera to control, image recognition, signage identification, dummy object editing technique, and by smart mobile phone camera (front camera and rear camera), these images are processed, add 3D dummy object by shooting real-time imaging.And by gesture, signage etc. carry out alternately with it, realize the virtual on the spot in person sense unified with reality.
The technology of immersive VR is realized across mobile platform:
DirectX3D Rendering is adopted to carry out distortion correction (Distortion Correction) and distortion convergent-divergent (Distortion Scale) by the picture that virtual three-dimensional scene renders by Pixel Shader; and segmentation is plotted to the left and right eyes screen of head-mounted display apparatus HMD (Head-mounted display) equipment; and keep the refreshing frequency of more than 60 frames per second, obtain more smooth picture to make user and protect the eyes of user.
See Fig. 2, the image that segmentation is drawn show left and right and seems repetition, edge also with the image of distortion on computer monitor, two width pictures part that right and left eyes should be seen in correspondence HMD equipment respectively in fact.Image, with this angle distortion, HMD equipment adopts convex lens to be amplified by picture, just can realize 110 degree of visual angles.And although left and right two width figure is similar, but the scenery scope that about being in fact corresponding people, two eyes are seen is different and specially adjust, in time seeing for two, just can allow the partly overlapping of two width images so simultaneously on visual field, reach the effect of perfect reduction 360 ° of visual fields.
Described system also comprises one or more sensor, for immediately obtaining positional information and the orientation information of HMD equipment, carries out the tracking in head movement and eyes direction; From sensor, obtain HMD equipment positional information and orientation information carry out head tracking (Head-tracking), and by message interface by coordinate (Position) and rotate hypercomplex number (Quaternion) information feed back to application program.Application program is processed feedack by the rendering module of described virtual reality engine, physical impacts module, audio module, nucleus module, coordinates matrix is set, the image processed is transferred to described equipment by the hardware abstraction layer of head-mounted display apparatus, the picture that user upgrades after seeing feedback by eyes, can also complete some actions controlled simply by nictation by the activity detecting eyes of user.
Sensor be accelerometer, gyroscope, compass and direction sensor at least one of them.
The mutual of Intelligent mobile equipment and computer equipment is realized by wireless communication technology (as Wi-Fi or bluetooth):
Server end is arranged on controlled remote equipment, server end can be PC, Cloud Server or other can the equipment of working procedure.Mobile client is arranged on the hand-held Intelligent mobile equipment of user.User uses mobile device by dummy keyboard at interactive operations such as screen taps, paddling remote analog computer input apparatus (keyboard/mouse), computer equipment feedback command is to mobile device (vibrations/play sound effect), realize the mutual of Intelligent mobile equipment and computer equipment, see Fig. 3.
Before use native system, need present target by build-in services device end in control machine, wait for the access of handheld device.Then user opens handheld device client software, searches LAN (Local Area Network) internal object equipment, after searching target device, clicks connection and can enter control operation.Such as, on cell-phone customer terminal, slip mobile phone screen, moves up and down about can controlling the mouse on remote computer.Click screen, can click by the mouse on analogue computer.Mobile phone screen adds two buttons in left and right, the mouse right and left key button operation on computer can be controlled.Cell-phone customer terminal is opened software dish interface or adopts dummy keyboard, the input through keyboard on computer can be controlled.
Cloud server framework:
For hall-room class server application, as chatroom, athletic game, chess/card game etc., application needs server maintenance room list, generates room information, safeguards player information in room, and instant messaging and message forward.
For the demand, the technology realizing immersive VR across mobile platform provides a set of Cloud Server solution.Client only need in backstage registered application, will obtain a unique authorization code.Beyond the clouds, server is this client distribution hall-room service device space, and is held open state at any time, waits for that user is connected into.Developer uses this authorization code to carry out tag application in engine.Just can connect corresponding server when starting application, use the service that Cloud Server provides.
The following examples are further explained system provided by the present invention.
This system comprises server end and client.Server end can be the server in PC or high in the clouds, and client comprises a head-mounted display apparatus and a mobile client; For the Rift of PC and Oculus company, mobile client is arranged on a mobile device, and for smart mobile phone, this PC and smart mobile phone can wirelessly connect.
Head-mounted display apparatus Oculus Rift is connected with PC by user, and PC is connected by WIFI or bluetooth with the panel computer being provided with mobile client.The Rift of Oculus company can provide horizontal direction 90 degree, the visual field that vertical direction is 110 degree, and vision input delay is lower than 20ms, and sensor delay is lower than 30ms.Add that then built-in gyroscope and accelerometer the head movement of real-time detection wearer can adjust picture displaying contents, user stays that wherein to infiltrate sense stronger.Therefore, although the resolution of Oculus Rift only has 640x800 at present (by 1280x800 split screen, 1080P display screen coupling will be had very soon), but rely on 90 degree of visual fields of its excellence and extremely low delay, in virtual reality, still surpass various commercial model HMD on the market.
User wears and by after Rift device start, data cube computation is set up with the related application run in PC, simultaneously can also select by mobile client, mobile phone to be connected with PC, user can by the action control of Rift equipment or by dummy keyboard that mobile phone shows in PC and Rift equipment to user's eyes provide two screens of picture carry out controls with alternately.
Wherein, the virtual reality engine run in PC, DirectX3D Rendering is adopted to carry out distortion correction and distortion convergent-divergent by the picture that virtual three-dimensional scene renders by pixel rendering device PixelShader, and segmentation is plotted to the left and right eyes screen of described head-mounted display apparatus, refreshes with frequencies more than 60 frames per second.
The control action of user is captured by the sensor be built in Rift equipment, corresponding data message is converted into through process, by being transferred in the handling procedure in PC with the connection of PC, this handling procedure processes obtained data message, then passes the result after process back Rift equipment.According to these results, Rift equipment and/or PC process shown image, and are shown to user.Such as when user carries out rotary head operation, picture changes to corresponding direction along with the rotation of user.
Simultaneously, this system provides the function of augmented reality, by internal or external camera real-time imaging taken and corresponding 3D dummy object projected in the 3D virtual reality image that user sees, this 3D dummy object can carry out with user alternately in the picture, such as can project in screen by a vase, user can carry out some to it and operation such as move.
Wherein, the control action of user can be caught by the sensor that Rift equipment is built-in, when user needs to use mobile device to control, such as user may wear headset equipment for a long time and feel too for a long time, virtual supervisory keyboard can be shown on the mobile apparatus, user controls visual angle by these buttons, action etc.
The connection that user is set up by mobile device and PC the operation of dummy keyboard, be sent to PC to process, result after process is shown to the screen of PC by PC, the screen of Rift equipment, and result is fed back on mobile device and smart mobile phone, with the fluency making user can be experienced operation intuitively by many aspects.
Be cloud server for server, embodiment is with substantially identical above, Rift equipment or mobile device are set up cloud with cloud server by network and are connected, the mutual data of all needs and information are all connected by cloud carries out mutual, and playing up and processing of image beyond the clouds server is carried out.So just make virtual reality technology breach the restriction in space, field widely can be applied to.
The present invention is not limited to above-mentioned preferred implementation, and anyone should learn the structure change made under enlightenment of the present invention, and every have identical or akin technical scheme with the present invention, all belongs to protection scope of the present invention.
Claims (10)
1. realize a system for immersive VR across mobile platform, comprise server end and client, it is characterized in that:
Reality scene is played up by described server end, and the display device transferring to client shows.
2. the system realizing immersive VR across mobile platform according to claim 1, is characterized in that:
Wherein client comprises head-mounted display apparatus and mobile client, and server end comprises virtual reality engine;
Mobile client is arranged on the hand-held Intelligent mobile equipment of user, described Intelligent mobile equipment is undertaken alternately by radio communication and server end, mobile device is by the input instruction of interactive operation emulating server end, control virtual scene or personage, server end feedback command is to mobile device.
3. the system realizing immersive VR across mobile platform according to claim 2, virtual reality engine, the picture adopting DirectX3D Rendering to be rendered by virtual three-dimensional scene by pixel rendering device Pixel Shader carries out distortion correction and distortion convergent-divergent, and segmentation is plotted to the left and right eyes screen of described head-mounted display apparatus, refreshes with frequencies more than 60 frames per second.
4. the system realizing immersive VR across mobile platform according to claim 3, it is characterized in that: described system also comprises one or more sensor, for immediately obtaining positional information and the orientation information of head-mounted display apparatus, carry out the tracking in head movement and eyes direction; Described sensor be accelerometer, gyroscope, compass or direction sensor at least one of them;
Described system also comprises message interface, for the position coordinates that obtained by described sensor with rotate hypercomplex number information feed back to the application program operating in server end.
5. the system realizing immersive VR across mobile platform according to claim 4, is characterized in that:
Described application program is by the processing module of described virtual reality engine, process is played up, physical impacts, audio effect processing process feedack, coordinates matrix is set, the process of image is carried out according to feedack, the image processed is transferred to client by the hardware abstraction layer HAL of head-mounted display apparatus, the picture that user upgrades after seeing feedback by eyes.
6. the system realizing immersive VR across mobile platform according to claim 4, is characterized in that:
Described mobile device is undertaken by Wi-Fi or bluetooth and server end alternately.
7. the system realizing immersive VR across mobile platform according to claim 4, is characterized in that:
The interactive operation that described mobile device performs can be Touch Screen or dummy keyboard operation realizes.
8. the system realizing immersive VR across mobile platform according to claim 4, is characterized in that:
The instruction that server end feeds back to mobile device can be performed by vibrations and/or play sound effect.
9. the system realizing immersive VR across mobile platform according to claim 4, is characterized in that:
Server end can be PC or Cloud Server.
10. realize a system for immersive VR across mobile platform, this system comprises server end and client, and server end comprises:
Message interface, for interactive information between client;
Processing module, for the treatment of the information of client feedback, and sends it back client by the result after process;
Image rendering module, carries out 3D to image and plays up, and generates 3D scene;
Pixel rendering module, carries out distortion correction and distortion convergent-divergent to the 3D scene generated
Segmentation drafting module, draws the Image Segmentation Using played up;
Image output module, exports on the screen of the head-mounted display apparatus of client by the image split;
Client comprises:
Head-mounted display apparatus and mobile client;
Wherein head-mounted display apparatus comprises:
Display screen, shows for the image exported by server end, and to make user to watch, it is per second that the refreshing frequency of described display screen is not less than 60 frames;
Sensor, for immediately obtaining positional information and the orientation information of head-mounted display apparatus, carries out the tracking in head movement and eyes direction;
Message interface, for server end interactive information;
Processing module, for the treatment of the information that sensor obtains, is translated into corresponding position coordinates and rotates hypercomplex number information, and the result after transforming is sent to server end;
Mobile client is installed on the mobile apparatus, comprising:
Control module, by arranging virtual input device on the screen of the mobile device, by the input instruction of interactive operation emulating server end, controls virtual scene or personage;
Processing module, for the treatment of the information received by mobile device, and is sent to corresponding server end by the result after process;
Message interface, for server end interactive information;
Wireless connections module, for setting up wireless connections with server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410084307.4A CN104915979A (en) | 2014-03-10 | 2014-03-10 | System capable of realizing immersive virtual reality across mobile platforms |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410084307.4A CN104915979A (en) | 2014-03-10 | 2014-03-10 | System capable of realizing immersive virtual reality across mobile platforms |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104915979A true CN104915979A (en) | 2015-09-16 |
Family
ID=54085015
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410084307.4A Pending CN104915979A (en) | 2014-03-10 | 2014-03-10 | System capable of realizing immersive virtual reality across mobile platforms |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104915979A (en) |
Cited By (54)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183168A (en) * | 2015-09-17 | 2015-12-23 | 惠州Tcl移动通信有限公司 | Head-wearing type virtual reality device based on mobile terminal and starting method thereof |
| CN105224079A (en) * | 2015-09-17 | 2016-01-06 | 惠州Tcl移动通信有限公司 | Based on wear-type virtual reality device and the control method thereof of mobile terminal |
| CN105260158A (en) * | 2015-09-17 | 2016-01-20 | 深圳市冠旭电子有限公司 | Method and device for audio processing based on augmented reality technology |
| CN105635805A (en) * | 2015-12-18 | 2016-06-01 | 歌尔声学股份有限公司 | Method and apparatus for optimizing motion image in virtual reality scene |
| CN105844688A (en) * | 2016-06-16 | 2016-08-10 | 福建数博讯信息科技有限公司 | Virtual reality rendering method and device |
| CN105869215A (en) * | 2016-03-28 | 2016-08-17 | 上海米影信息科技有限公司 | Virtual reality imaging system |
| CN105892643A (en) * | 2015-12-31 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Multi-interface unified display system and method based on virtual reality |
| CN106020493A (en) * | 2016-03-13 | 2016-10-12 | 成都市微辣科技有限公司 | Product display device and method based on virtual reality |
| CN106095090A (en) * | 2016-06-07 | 2016-11-09 | 北京行云时空科技有限公司 | Control method, device and the system of spatial scene based on intelligence system |
| CN106385348A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Virtual reality entertainment system with step-by-step identification function and method of virtual reality entertainment system |
| CN106385682A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Multi-functional virtual reality entertainment system and method thereof |
| CN106411875A (en) * | 2016-09-21 | 2017-02-15 | 平越 | Virtual reality entertainment system under history record and method thereof |
| CN106445121A (en) * | 2016-09-08 | 2017-02-22 | 三星电子（中国）研发中心 | Virtual reality device and terminal interaction method and apparatus |
| CN106454322A (en) * | 2016-11-07 | 2017-02-22 | 金陵科技学院 | VR Image processing system and method thereof |
| CN106445171A (en) * | 2016-11-11 | 2017-02-22 | 上海远鉴信息科技有限公司 | Method and system for multi-operation synchronous control in virtual reality |
| CN106488205A (en) * | 2016-11-17 | 2017-03-08 | 四川长虹电器股份有限公司 | A kind of projection arrangement for being automatically obtained image mosaic fusion and method |
| CN106527916A (en) * | 2016-09-22 | 2017-03-22 | 乐视控股（北京）有限公司 | Operating method and device based on virtual reality equipment, and operating equipment |
| CN106527713A (en) * | 2016-11-07 | 2017-03-22 | 金陵科技学院 | Three-dimensional data rendering system for VR and method thereof |
| CN106547557A (en) * | 2016-10-10 | 2017-03-29 | 成都斯斐德科技有限公司 | A kind of multi-screen interactive exchange method based on virtual reality and bore hole 3D |
| CN106792241A (en) * | 2016-12-26 | 2017-05-31 | 上海今尚数码科技有限公司 | A kind of WIFI controls many Samsung VR glasses to play panorama system and method |
| WO2017088481A1 (en) * | 2015-11-26 | 2017-06-01 | 乐视控股（北京）有限公司 | Method and apparatus for starting application |
| WO2017113733A1 (en) * | 2015-12-27 | 2017-07-06 | 乐视控股（北京）有限公司 | Method and device for viewing three-dimensional video freely |
| CN106997234A (en) * | 2016-01-22 | 2017-08-01 | 阿里巴巴集团控股有限公司 | Virtual real mode processing method, equipment and system |
| CN107277736A (en) * | 2016-03-31 | 2017-10-20 | 株式会社万代南梦宫娱乐 | Simulation System, Sound Processing Method And Information Storage Medium |
| CN107566224A (en) * | 2016-07-01 | 2018-01-09 | 桂林电子科技大学 | A kind of wear-type virtual reality intelligent domestic system |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN108021229A (en) * | 2016-10-31 | 2018-05-11 | 迪斯尼企业公司 | High fidelity numeral immersion is recorded by computed offline to experience |
| CN108139815A (en) * | 2016-02-22 | 2018-06-08 | 谷歌有限责任公司 | For the scene of the display of virtual reality content and the discrete time warp of object |
| CN108415654A (en) * | 2017-02-10 | 2018-08-17 | 上海真曦通信技术有限公司 | Virtual input system and correlation technique |
| CN108427195A (en) * | 2017-02-14 | 2018-08-21 | 深圳梦境视觉智能科技有限公司 | A kind of information processing method and equipment based on augmented reality |
| WO2018149267A1 (en) * | 2017-02-14 | 2018-08-23 | 深圳梦境视觉智能科技有限公司 | Display method and device based on augmented reality |
| CN108701371A (en) * | 2015-10-09 | 2018-10-23 | 华纳兄弟娱乐公司 | It is controlled for the film of virtual reality and augmented reality |
| CN108986232A (en) * | 2018-07-27 | 2018-12-11 | 广州汉智网络科技有限公司 | A method of it is shown in VR and AR environment picture is presented in equipment |
| CN109035133A (en) * | 2018-08-29 | 2018-12-18 | 长春博立电子科技有限公司 | The method for visualizing and system of manufacturing process based on virtual simulation environment |
| CN109074149A (en) * | 2016-05-17 | 2018-12-21 | 谷歌有限责任公司 | For enhance or the wear-type referential of reality environment in image tracing |
| US20190088177A1 (en) * | 2017-09-20 | 2019-03-21 | Acer Incorporated | Image processing system and method |
| US20190088000A1 (en) * | 2017-09-20 | 2019-03-21 | Acer Incorporated | Image processing system and method |
| CN109558001A (en) * | 2017-09-27 | 2019-04-02 | 宏碁股份有限公司 | Image processing system and method |
| CN109561298A (en) * | 2017-09-27 | 2019-04-02 | 宏碁股份有限公司 | Image processing system and method |
| CN109856993A (en) * | 2019-01-29 | 2019-06-07 | 北京奥特贝睿科技有限公司 | A kind of autonomous driving emulation platform |
| CN109937394A (en) * | 2016-10-04 | 2019-06-25 | 脸谱公司 | Control and interface for user's interaction in Virtual Space |
| CN110270088A (en) * | 2018-03-16 | 2019-09-24 | 索尼互动娱乐美国有限责任公司 | Asynchronous virtual reality interaction |
| CN110362209A (en) * | 2019-07-23 | 2019-10-22 | 辽宁向日葵教育科技有限公司 | A kind of MR mixed reality intelligent perception interactive system |
| CN110457227A (en) * | 2019-08-20 | 2019-11-15 | 冯世如 | Auxiliary development personnel or student carry out the system and development approach of VR application and development |
| CN110574375A (en) * | 2017-04-28 | 2019-12-13 | 苹果公司 | Video pipeline |
| CN110800033A (en) * | 2017-06-29 | 2020-02-14 | 威博外科公司 | Virtual reality peritoneoscope formula instrument |
| CN111275612A (en) * | 2020-01-17 | 2020-06-12 | 成都库珀区块链科技有限公司 | VR (virtual reality) technology-based K-line display and interaction method and device |
| CN112002017A (en) * | 2020-06-30 | 2020-11-27 | 国网河北省电力有限公司培训中心 | Virtual scene generation method, device, terminal, medium and virtual training system |
| CN113542849A (en) * | 2021-07-06 | 2021-10-22 | 腾讯科技（深圳）有限公司 | Video data processing method and device, electronic equipment and storage medium |
| CN114115518A (en) * | 2020-08-28 | 2022-03-01 | 明日基金知识产权有限公司 | System and method for enabling interaction in a virtual environment |
| US11284955B2 (en) | 2017-06-29 | 2022-03-29 | Verb Surgical Inc. | Emulation of robotic arms and control thereof in a virtual reality environment |
| US11296398B2 (en) | 2016-10-06 | 2022-04-05 | Avery Dennison Retail Information Services Llc | Methods for creating RFID tags based on flexible antenna materials |
| US11580882B2 (en) | 2017-06-29 | 2023-02-14 | Verb Surgical Inc. | Virtual reality training, simulation, and collaboration in a robotic surgical system |
| US11816820B2 (en) | 2017-07-21 | 2023-11-14 | Apple Inc. | Gaze direction-based adaptive pre-filtering of video data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060284792A1 (en) * | 2000-01-28 | 2006-12-21 | Intersense, Inc., A Delaware Corporation | Self-referenced tracking |
| CN202041945U (en) * | 2010-10-26 | 2011-11-16 | 广州市艺捷数码科技有限公司 | 3D live-action interactive apparatus |
| CN102253713A (en) * | 2011-06-23 | 2011-11-23 | 康佳集团股份有限公司 | Display system orienting to three-dimensional images |
| CN103442244A (en) * | 2013-08-30 | 2013-12-11 | 北京京东方光电科技有限公司 | 3D glasses, 3D display system and 3D display method |
| CN104035760A (en) * | 2014-03-04 | 2014-09-10 | 苏州天魂网络科技有限公司 | System capable of realizing immersive virtual reality over mobile platforms |
-
2014
- 2014-03-10 CN CN201410084307.4A patent/CN104915979A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060284792A1 (en) * | 2000-01-28 | 2006-12-21 | Intersense, Inc., A Delaware Corporation | Self-referenced tracking |
| CN202041945U (en) * | 2010-10-26 | 2011-11-16 | 广州市艺捷数码科技有限公司 | 3D live-action interactive apparatus |
| CN102253713A (en) * | 2011-06-23 | 2011-11-23 | 康佳集团股份有限公司 | Display system orienting to three-dimensional images |
| CN103442244A (en) * | 2013-08-30 | 2013-12-11 | 北京京东方光电科技有限公司 | 3D glasses, 3D display system and 3D display method |
| CN104035760A (en) * | 2014-03-04 | 2014-09-10 | 苏州天魂网络科技有限公司 | System capable of realizing immersive virtual reality over mobile platforms |
Cited By (72)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183168A (en) * | 2015-09-17 | 2015-12-23 | 惠州Tcl移动通信有限公司 | Head-wearing type virtual reality device based on mobile terminal and starting method thereof |
| CN105224079A (en) * | 2015-09-17 | 2016-01-06 | 惠州Tcl移动通信有限公司 | Based on wear-type virtual reality device and the control method thereof of mobile terminal |
| CN105260158A (en) * | 2015-09-17 | 2016-01-20 | 深圳市冠旭电子有限公司 | Method and device for audio processing based on augmented reality technology |
| CN105224079B (en) * | 2015-09-17 | 2018-07-10 | 惠州Tcl移动通信有限公司 | Wear-type virtual reality device and its control method based on mobile terminal |
| CN105183168B (en) * | 2015-09-17 | 2019-02-26 | 惠州Tcl移动通信有限公司 | Wear-type virtual reality device and its starting method based on mobile terminal |
| CN105260158B (en) * | 2015-09-17 | 2018-09-25 | 深圳市冠旭电子股份有限公司 | A kind of audio-frequency processing method and device based on augmented reality |
| CN108701371A (en) * | 2015-10-09 | 2018-10-23 | 华纳兄弟娱乐公司 | It is controlled for the film of virtual reality and augmented reality |
| WO2017088481A1 (en) * | 2015-11-26 | 2017-06-01 | 乐视控股（北京）有限公司 | Method and apparatus for starting application |
| CN105635805A (en) * | 2015-12-18 | 2016-06-01 | 歌尔声学股份有限公司 | Method and apparatus for optimizing motion image in virtual reality scene |
| CN105635805B (en) * | 2015-12-18 | 2018-12-21 | 潍坊歌尔电子有限公司 | Optimize the method and apparatus of moving image in a kind of virtual reality scenario |
| WO2017113733A1 (en) * | 2015-12-27 | 2017-07-06 | 乐视控股（北京）有限公司 | Method and device for viewing three-dimensional video freely |
| CN105892643A (en) * | 2015-12-31 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Multi-interface unified display system and method based on virtual reality |
| WO2017113718A1 (en) * | 2015-12-31 | 2017-07-06 | 乐视控股（北京）有限公司 | Virtual reality-based method and system for unified display of multiple interfaces |
| CN106997234B (en) * | 2016-01-22 | 2020-08-11 | 阿里巴巴集团控股有限公司 | Virtual reality mode processing method, equipment and system |
| CN106997234A (en) * | 2016-01-22 | 2017-08-01 | 阿里巴巴集团控股有限公司 | Virtual real mode processing method, equipment and system |
| CN108139815B (en) * | 2016-02-22 | 2020-11-10 | 谷歌有限责任公司 | Discrete time warping of scenes and objects for display of virtual reality content |
| CN108139815A (en) * | 2016-02-22 | 2018-06-08 | 谷歌有限责任公司 | For the scene of the display of virtual reality content and the discrete time warp of object |
| CN106020493A (en) * | 2016-03-13 | 2016-10-12 | 成都市微辣科技有限公司 | Product display device and method based on virtual reality |
| CN105869215A (en) * | 2016-03-28 | 2016-08-17 | 上海米影信息科技有限公司 | Virtual reality imaging system |
| CN107277736A (en) * | 2016-03-31 | 2017-10-20 | 株式会社万代南梦宫娱乐 | Simulation System, Sound Processing Method And Information Storage Medium |
| CN109074149A (en) * | 2016-05-17 | 2018-12-21 | 谷歌有限责任公司 | For enhance or the wear-type referential of reality environment in image tracing |
| CN106095090A (en) * | 2016-06-07 | 2016-11-09 | 北京行云时空科技有限公司 | Control method, device and the system of spatial scene based on intelligence system |
| CN105844688A (en) * | 2016-06-16 | 2016-08-10 | 福建数博讯信息科技有限公司 | Virtual reality rendering method and device |
| CN105844688B (en) * | 2016-06-16 | 2019-06-28 | 福建数博讯信息科技有限公司 | Virtual reality rendering method and device |
| CN107566224A (en) * | 2016-07-01 | 2018-01-09 | 桂林电子科技大学 | A kind of wear-type virtual reality intelligent domestic system |
| CN106445121A (en) * | 2016-09-08 | 2017-02-22 | 三星电子（中国）研发中心 | Virtual reality device and terminal interaction method and apparatus |
| CN106411875A (en) * | 2016-09-21 | 2017-02-15 | 平越 | Virtual reality entertainment system under history record and method thereof |
| CN106385682A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Multi-functional virtual reality entertainment system and method thereof |
| CN106385348A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Virtual reality entertainment system with step-by-step identification function and method of virtual reality entertainment system |
| CN106527916A (en) * | 2016-09-22 | 2017-03-22 | 乐视控股（北京）有限公司 | Operating method and device based on virtual reality equipment, and operating equipment |
| CN109937394A (en) * | 2016-10-04 | 2019-06-25 | 脸谱公司 | Control and interface for user's interaction in Virtual Space |
| US11296398B2 (en) | 2016-10-06 | 2022-04-05 | Avery Dennison Retail Information Services Llc | Methods for creating RFID tags based on flexible antenna materials |
| CN106547557A (en) * | 2016-10-10 | 2017-03-29 | 成都斯斐德科技有限公司 | A kind of multi-screen interactive exchange method based on virtual reality and bore hole 3D |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN108021229A (en) * | 2016-10-31 | 2018-05-11 | 迪斯尼企业公司 | High fidelity numeral immersion is recorded by computed offline to experience |
| CN106527713A (en) * | 2016-11-07 | 2017-03-22 | 金陵科技学院 | Three-dimensional data rendering system for VR and method thereof |
| CN106454322A (en) * | 2016-11-07 | 2017-02-22 | 金陵科技学院 | VR Image processing system and method thereof |
| CN106527713B (en) * | 2016-11-07 | 2019-11-08 | 金陵科技学院 | The three-dimensional data rendering system and its method of VR |
| CN106445171A (en) * | 2016-11-11 | 2017-02-22 | 上海远鉴信息科技有限公司 | Method and system for multi-operation synchronous control in virtual reality |
| CN106488205A (en) * | 2016-11-17 | 2017-03-08 | 四川长虹电器股份有限公司 | A kind of projection arrangement for being automatically obtained image mosaic fusion and method |
| CN106792241A (en) * | 2016-12-26 | 2017-05-31 | 上海今尚数码科技有限公司 | A kind of WIFI controls many Samsung VR glasses to play panorama system and method |
| CN108415654A (en) * | 2017-02-10 | 2018-08-17 | 上海真曦通信技术有限公司 | Virtual input system and correlation technique |
| WO2018149267A1 (en) * | 2017-02-14 | 2018-08-23 | 深圳梦境视觉智能科技有限公司 | Display method and device based on augmented reality |
| CN108427195A (en) * | 2017-02-14 | 2018-08-21 | 深圳梦境视觉智能科技有限公司 | A kind of information processing method and equipment based on augmented reality |
| US11727619B2 (en) | 2017-04-28 | 2023-08-15 | Apple Inc. | Video pipeline |
| CN110574375A (en) * | 2017-04-28 | 2019-12-13 | 苹果公司 | Video pipeline |
| CN110800033A (en) * | 2017-06-29 | 2020-02-14 | 威博外科公司 | Virtual reality peritoneoscope formula instrument |
| US11944401B2 (en) | 2017-06-29 | 2024-04-02 | Verb Surgical Inc. | Emulation of robotic arms and control thereof in a virtual reality environment |
| US11580882B2 (en) | 2017-06-29 | 2023-02-14 | Verb Surgical Inc. | Virtual reality training, simulation, and collaboration in a robotic surgical system |
| US11284955B2 (en) | 2017-06-29 | 2022-03-29 | Verb Surgical Inc. | Emulation of robotic arms and control thereof in a virtual reality environment |
| US11900578B2 (en) | 2017-07-21 | 2024-02-13 | Apple Inc. | Gaze direction-based adaptive pre-filtering of video data |
| US11816820B2 (en) | 2017-07-21 | 2023-11-14 | Apple Inc. | Gaze direction-based adaptive pre-filtering of video data |
| US20190088177A1 (en) * | 2017-09-20 | 2019-03-21 | Acer Incorporated | Image processing system and method |
| US20190088000A1 (en) * | 2017-09-20 | 2019-03-21 | Acer Incorporated | Image processing system and method |
| CN109561298A (en) * | 2017-09-27 | 2019-04-02 | 宏碁股份有限公司 | Image processing system and method |
| CN109558001A (en) * | 2017-09-27 | 2019-04-02 | 宏碁股份有限公司 | Image processing system and method |
| CN110270088B (en) * | 2018-03-16 | 2023-05-02 | 索尼互动娱乐美国有限责任公司 | Asynchronous virtual reality interactions |
| CN110270088A (en) * | 2018-03-16 | 2019-09-24 | 索尼互动娱乐美国有限责任公司 | Asynchronous virtual reality interaction |
| CN108986232B (en) * | 2018-07-27 | 2023-11-10 | 江苏洪旭德生科技发展集团有限公司 | Method for presenting AR environment picture in VR display device |
| CN108986232A (en) * | 2018-07-27 | 2018-12-11 | 广州汉智网络科技有限公司 | A method of it is shown in VR and AR environment picture is presented in equipment |
| CN109035133B (en) * | 2018-08-29 | 2022-11-22 | 长春博立电子科技有限公司 | Visualization method and system of manufacturing process based on virtual simulation environment |
| CN109035133A (en) * | 2018-08-29 | 2018-12-18 | 长春博立电子科技有限公司 | The method for visualizing and system of manufacturing process based on virtual simulation environment |
| CN109856993B (en) * | 2019-01-29 | 2022-05-31 | 北京奥特贝睿科技有限公司 | Autonomous driving simulation platform |
| CN109856993A (en) * | 2019-01-29 | 2019-06-07 | 北京奥特贝睿科技有限公司 | A kind of autonomous driving emulation platform |
| CN110362209A (en) * | 2019-07-23 | 2019-10-22 | 辽宁向日葵教育科技有限公司 | A kind of MR mixed reality intelligent perception interactive system |
| CN110362209B (en) * | 2019-07-23 | 2023-08-11 | 辽宁向日葵教育科技有限公司 | MR mixed reality intelligent perception interactive system |
| CN110457227B (en) * | 2019-08-20 | 2023-04-07 | 深圳天富创科技有限公司 | System and method for assisting developers or students in VR application development |
| CN110457227A (en) * | 2019-08-20 | 2019-11-15 | 冯世如 | Auxiliary development personnel or student carry out the system and development approach of VR application and development |
| CN111275612A (en) * | 2020-01-17 | 2020-06-12 | 成都库珀区块链科技有限公司 | VR (virtual reality) technology-based K-line display and interaction method and device |
| CN112002017A (en) * | 2020-06-30 | 2020-11-27 | 国网河北省电力有限公司培训中心 | Virtual scene generation method, device, terminal, medium and virtual training system |
| CN114115518A (en) * | 2020-08-28 | 2022-03-01 | 明日基金知识产权有限公司 | System and method for enabling interaction in a virtual environment |
| CN113542849A (en) * | 2021-07-06 | 2021-10-22 | 腾讯科技（深圳）有限公司 | Video data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104915979A (en) | System capable of realizing immersive virtual reality across mobile platforms | |
| CN104035760A (en) | System capable of realizing immersive virtual reality over mobile platforms | |
| US11494995B2 (en) | Systems and methods for virtual and augmented reality | |
| CN105981076B (en) | Synthesize the construction of augmented reality environment | |
| CN107852573B (en) | Mixed reality social interactions | |
| CN110147231B (en) | Combined special effect generation method and device and storage medium | |
| KR102595150B1 (en) | Method for controlling multiple virtual characters, device, apparatus, and storage medium | |
| WO2018188499A1 (en) | Image processing method and device, video processing method and device, virtual reality device and storage medium | |
| JP7008730B2 (en) | Shadow generation for image content inserted into an image | |
| US10192363B2 (en) | Math operations in mixed or virtual reality | |
| US9717988B2 (en) | Rendering system, rendering server, control method thereof, program, and recording medium | |
| US20150091891A1 (en) | System and method for non-holographic teleportation | |
| EP3814876B1 (en) | Placement and manipulation of objects in augmented reality environment | |
| CN110832442A (en) | Optimized shading and adaptive mesh skin in point-of-gaze rendering systems | |
| KR20160103897A (en) | System for augmented reality image display and method for augmented reality image display | |
| CN107274491A (en) | A kind of spatial manipulation Virtual Realization method of three-dimensional scenic | |
| CN112581571B (en) | Control method and device for virtual image model, electronic equipment and storage medium | |
| US20190259198A1 (en) | Systems and methods for generating visual representations of a virtual object for display by user devices | |
| CN111467803B (en) | Display control method and device in game, storage medium and electronic equipment | |
| GB2590871A (en) | System and method for providing a computer-generated environment | |
| CN109725956A (en) | A kind of method and relevant apparatus of scene rendering | |
| JP2017033294A (en) | Three-dimensional drawing system and three-dimensional drawing program | |
| CN114942737A (en) | Display method, display device, head-mounted device and storage medium | |
| Nivedha et al. | Enhancing user experience through physical interaction in handheld augmented reality | |
| Zhu et al. | Integrated Co-Designing Using Building Information Modeling and Mixed Reality with Erased Backgrounds for Stock Renovation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: Suzhou Tian Hun network technology company limited Document name: Notification of Publication of the Application for Invention |
|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice |
Addressee: Zhang Yu Document name: Notification of Passing Examination on Formalities |
|
| CB02 | Change of applicant information |
Address after: High tech Zone Suzhou city Jiangsu province 215011 Chuk Yuen Road No. 209 Applicant after: Suzhou Soul Network Polytron Technologies Inc Address before: High tech Zone Suzhou city Jiangsu province 215011 Chuk Yuen Road No. 209 Applicant before: Suzhou Tian Hun network technology company limited |
|
| COR | Change of bibliographic data | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150916 |
|
| WD01 | Invention patent application deemed withdrawn after publication |