CN111179437A - Cloud VR connectionless streaming system and connection method - Google Patents
Cloud VR connectionless streaming system and connection method Download PDFInfo
- Publication number
- CN111179437A CN111179437A CN201911394536.5A CN201911394536A CN111179437A CN 111179437 A CN111179437 A CN 111179437A CN 201911394536 A CN201911394536 A CN 201911394536A CN 111179437 A CN111179437 A CN 111179437A
- Authority
- CN
- China
- Prior art keywords
- cloud
- client
- driver
- steam
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Computing Systems (AREA)
- Human Computer Interaction (AREA)
- Information Transfer Between Computers (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses a Cloud VR connectionless streaming system and a connection method, and relates to the technical field of virtual reality. The system comprises: at least one head display device; a 3D application system; a Steam VR system; a Cloud VR system including a Cloud VR server, a Cloud VR driver and a Cloud VR client; under the same local area network, the Cloud VR system is integrated with the Steam VR system; the Cloud VR server side obtains rendering image data of the 3D application system through the Steam VR system and encodes the rendering image data into video stream data with a set format; the Cloud VR driver transmits the video to a Cloud VR client of the head display device through a network; and the Cloud VR client decodes and renders the video. According to the Cloud VR connectionless streaming system and the connection method, the traditional head display is replaced by the VR all-in-one machine, data are transmitted through WIFI, redundant cable connection is not needed, the Cloud VR connectionless streaming system is achieved, and VR experience of a user is comprehensively improved.
Description
Technical Field
The application relates to the technical field of virtual reality, in particular to a Cloud VR connectionless streaming system and a connection method.
Background
Virtual Reality (VR) technology integrates technologies such as computer three-dimensional model processing, stereoscopic display, natural human-computer interaction, electronic information, simulation, and the like, and a Virtual simulation environment with high Reality sense is simulated by a computer, thereby giving people an environmental immersion sense. Immersive virtual reality (Immersive VR) provides a fully Immersive experience for the participants, giving the user a visual experience that is self-contained in the virtual world. In recent years, the VR industry has developed rapidly, and cloudvrs supporting cloud rendering have become an inevitable trend in VR development driven by four major requirements of reducing user VR consumption cost, improving user experience, popularizing VR business scenes, protecting VR content copyright, and the like. The Cloud VR solution comprises a content platform, a service platform, a network scheme, a terminal scheme and the like, and supports deployment of five scenes including a megascreen cinema, live broadcast, 360-degree video, games and education. However, the conventional PC connected to the helmet requires the use of a data cable to transmit data, which greatly affects the VR experience, so that the user can only move near the PC, and there is a risk of being entangled by the cable and even stumbled and injured.
Therefore, it is desirable to provide a Cloud VR connectionless streaming system and a connection method, in which a conventional head display is replaced by a VR all-in-one machine, data is transmitted by using WIFI, unnecessary cable connection is not required, and the Cloud VR connectionless streaming system is implemented to comprehensively improve VR experience of a user.
Disclosure of Invention
According to a first aspect of some embodiments of the present application, there is provided a Cloud VR connectionless streaming system for use in a terminal (e.g., an electronic device, etc.), the system may include: at least one head display device; a 3D application system; a Steam VR system; a Cloud VR system including a Cloud VR server, a Cloud VR driver and a Cloud VR client; under the same local area network, the Cloud VR system is integrated with the Steam VR system, wherein the Cloud VR drive is integrated inside the Steam VR system; the Cloud VR server side obtains rendering image data of the 3D application system through the Steam VR system and encodes the rendering image data into video stream data with a set format; the Cloud VR driver transmits the video to a Cloud VR client of the head display device through a network, and the Cloud VR client is installed on the head display device; and the Cloud VR client decodes and renders the video.
In some embodiments, the Cloud VR server is for connection management, further comprising: starting a Steam VR system, and acquiring the state of a Cloud VR client through a Cloud VR drive integrated in the Steam VR system; controlling the Cloud VR driver to connect to the Cloud VR client by sending instructions to the Cloud VR driver.
In some embodiments, the Cloud VR server may control the resolution of the rendering, the encoding format of the video stream data, the output location of the log.
In some embodiments, the Cloud VR client is installed at the head-up device, further comprising: receiving video stream data sent by the Cloud VR driver through a network; decoding the video stream data coded into the set format by using a decoding tool; and drawing the video stream data to a screen by utilizing OpenGL.
In some embodiments, the Cloud VR client is an android native application and the decoding tool comprises MediaCodec.
In some embodiments, the Cloud VR driver is integrated inside the Steam VR system, the Cloud VR driver is a hub of the Cloud VR server and the Cloud VR client, further comprising: the Steam VR system receives image data sent by the 3D application; the Cloud VR drive encodes the image data; and the coded video stream data is sent to the Cloud VR client through the network.
In some embodiments, the encoding comprises H264 or H265.
In some embodiments, the 3D application system includes Idea VR, Unity, and urea 4.
According to a second aspect of some embodiments of the present application, there is provided a method of concatenating a Cloud VR connectionless streaming system, the method comprising: writing the configuration file into a public memory of the system; starting a Cloud VR drive through a Steam VR, and reading a configuration file and initializing the configuration file by the Cloud VR drive; starting a Cloud VR client through a head display device, and sending a broadcast of head display information to a local area network; receiving the broadcast through the Cloud VR drive and storing the head display information; inquiring and displaying head display information through the Cloud VR server, and sending a connection instruction to the Cloud VR driver; connect to the Cloud VR client through the Cloud VR driver and return the results to the Cloud VR server.
In some embodiments, the overhead information comprises: head ID, resolution, left and right eye FOV, device type (OcuusQuest, OcuusGo, XiaomiVR); the resolution can be sent to a CloudVR driver for application to an encoding setting; the connection instruction comprises address information of the head display information.
Therefore, according to the Cloud VR connectionless streaming system and the connection method of some embodiments of the application, the traditional head display is replaced by the VR all-in-one machine, WIFI is used for transmitting data, redundant cable connection is not needed, the Cloud VR connectionless streaming system is realized, and VR experience of users is comprehensively improved.
Drawings
For a better understanding and appreciation of some embodiments of the present application, reference will now be made to the description of embodiments taken in conjunction with the accompanying drawings, in which like reference numerals designate corresponding parts in the figures.
Fig. 1 is an exemplary schematic diagram of a head-up device connectionless streaming system over the same lan according to some embodiments of the present application.
Fig. 2 is an exemplary schematic diagram of a Cloud VR system architecture provided in accordance with some embodiments of the present application.
Fig. 3 is an exemplary schematic diagram of a Cloud VR connectionless streaming system provided in accordance with some embodiments of the present application.
Fig. 4 is an exemplary flow diagram of a connection method of a Cloud VR connectionless streaming system provided in accordance with some embodiments of the present application.
Detailed Description
The following description, with reference to the accompanying drawings, is provided to facilitate a comprehensive understanding of various embodiments of the application as defined by the claims and their equivalents. These embodiments include various specific details for ease of understanding, but these are to be considered exemplary only. Accordingly, those skilled in the art will appreciate that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present application. In addition, descriptions of well-known functions and constructions will be omitted herein for brevity and clarity.
The terms and phrases used in the following specification and claims are not to be limited to the literal meaning, but are merely for the clear and consistent understanding of the application. Accordingly, it will be appreciated by those skilled in the art that the description of the various embodiments of the present application is provided for illustration only and not for the purpose of limiting the application as defined by the appended claims and their equivalents.
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in some embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is to be understood that the terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only, and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The expressions "first", "second", "the first" and "the second" are used for modifying the corresponding elements without regard to order or importance, and are used only for distinguishing one element from another element without limiting the corresponding elements.
A terminal according to some embodiments of the present application may be an electronic device, which may include one or a combination of several of a virtual reality device (VR), a renderer, a personal computer (PC, e.g., tablet, desktop, notebook, netbook, PDA), a smart phone, a mobile phone, an e-book reader, a Portable Multimedia Player (PMP), an audio/video player (MP 3/MP 4), a camera, and a wearable device, etc. According to some embodiments of the present application, the wearable device may include an accessory type (e.g., watch, ring, bracelet, glasses, or Head Mounted Device (HMD)), an integrated type (e.g., electronic garment), a decorative type (e.g., skin pad, tattoo, or built-in electronic device), and the like, or a combination of several. In some embodiments of the present application, the electronic device may be flexible, not limited to the above devices, or may be a combination of one or more of the above devices. In this application, the term "user" may indicate a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).
The embodiment of the application provides a Cloud VR connectionless streaming system. In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Fig. 1 is an exemplary schematic diagram of a head-up device connectionless streaming system over the same lan according to some embodiments of the present application. As shown in fig. 1, the head-up device connectionless streaming system 100 under the same lan may include a 5G router 110, a PC 120, a head-up device 130, and the like. Specifically, the PC 120 and the head display device 130 are in the same router, and the network environment of the same router may be any network environment of the same lan. Further, the PC 120 is connected to the router 110 via a network cable, and the head display device 130 establishes a communication connection with the router 110 via WIFI. The head display device 130 may include a VR all-in-one machine or the like.
In some embodiments of the present application, the head-up device connectionless streaming system 100 under the same lan may omit one or more elements, or may further include one or more other elements. As an example, the head-up device connectionless streaming system 100 may include a plurality of head-up devices 130, such as a plurality of VR integrators, and the like. The Local Area Network may be another type of communication Network, which may include a computer Network (e.g., a Local Area Network (LAN) or a Wide Area Network (WAN)), the internet, and/or a telephone Network, etc., or a combination of several. In some embodiments, 5G router 110 may be other types of wireless communication networks. The wireless communication may include microwave communication and/or satellite communication, among others. The Wireless communication may include cellular communication (e.g., global System for Mobile Communications), Code Division Multiple Access (CDMA), third Generation Mobile communication (3G, The 3rd Generation Telecommunication), fourth Generation Mobile communication (4G), fifth Generation Mobile communication (5G), Long Term Evolution (LTE-a, LTE-Advanced), Wideband Code Division multiple access (WCDMA, Wideband Code Division multiple access), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro, Wireless Broadband), etc., or a combination thereof, in some embodiments, PC 120 may be other electronic device with equivalent functionality, which may include a virtual reality rendering (VR), a personal computer rendering (PC), or a combination thereof, Desktop computers, notebooks, netbooks, PDA's (personal digital assistants), smart phones, mobile phones, e-book readers, Portable Multimedia Players (PMP), audio/video players (MP 3/MP 4), cameras, wearable devices, and the like. In some embodiments, the head-up device 130 may be other wearable devices with equivalent functional modules. The wearable device may include an accessory type (e.g., watch, ring, bracelet, glasses, or Head Mounted Device (HMD)), an integrated type (e.g., electronic garment), an ornamental type (e.g., skin pad, tattoo, or built-in electronic device), etc., or a combination of several.
In some embodiments, the WIFI may be other types of wireless communication technologies. According to some embodiments of the present application, the Wireless communication may include Wireless local Area Network (WiFi), Bluetooth Low Energy (BLE), ZigBee (ZigBee), Near Field Communication (NFC), magnetic security transmission, radio frequency and Body Area Network (BAN), or the like, or a combination of several. According to some embodiments of the present application, the wired communication may include a Global Navigation Satellite System (Global Navigation Satellite System), a Global Positioning System (GPS), a beidou Navigation Satellite System, galileo (european Global Satellite Navigation System), or the like. The wired communication may include a Universal Serial Bus (USB), a High-Definition multimedia interface (HDMI), a recommended Standard 232 (RS-232, recommended Standard 232), and/or Plain Old Telephone Service (POTS), or the like, or a combination of several.
It should be noted that the above description of the connection-less streaming system 100 for the head display devices in the same lan is for convenience of description only, and the present application is not limited to the scope of the embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the principles of the system, which may be combined in any manner or combined with other elements to form a subsystem for use in a field of application in which the method and system described above is practiced. Such variations are within the scope of the present application.
Fig. 2 is an exemplary schematic diagram of a Cloud VR system architecture provided in accordance with some embodiments of the present application. As shown in fig. 2, the Cloud VR system architecture 200 includes a Cloud VR server 210, a Cloud VR driver 220, and a Cloud VR client 230.
According to some embodiments of the present application, the Cloud VR server 210 is primarily responsible for connection management for starting the Steam VR system (fig. 3) through the Cloud VR driver 220 to obtain the status of the Cloud VR client 230. In some embodiments, the Cloud VR server 210 may control the driver to connect to the Cloud VR client 230 by sending instructions to the Cloud VR driver 220. In some embodiments, the Cloud VR server 210 may control the resolution of the rendering, the encoding format, the log output location, and the like.
According to some embodiments of the present application, the Cloud VR driver 220 may be integrated inside the Steam VR, being the hub of the Cloud VR server 210 and the Cloud VR client 230. Wherein the Steam VR can receive image data sent by the 3D application and then transmit to the Cloud VR driver 220 for encoding. The code may include H264 or H265, etc. Among them, the bandwidth utilization rate of the H265 coding is higher, and there is a higher requirement for performance during coding. In some embodiments, display cards in the great series GeForce GTX 950 and above may support H265 encoding, and when the display cards are below this configuration, the system may use H264 encoding. The encoded video stream data may be sent over a network to the Cloud VR client 230. In some embodiments, the Cloud VR driver 220 may be used to perform video encoding, network transmission adaptive code rate, command control, and the like. The video coding may include AVC and/or HEVC.
According to some embodiments of the present Application, the Cloud VR client 230 may be an android native Application (APP). The Cloud VR client 230 can be installed in the head display device 130, receive the video stream data sent by the Cloud VR driver 220 through the network, then decode by the decoding tool, and finally draw on the screen by using OpenGL. The decoding tool may include MediaCodec, a decoding tool provided by the android official, and the like.
It should be noted that the above description of the Cloud VR system architecture 200 is merely for convenience of description and is not intended to limit the scope of the present application. It will be understood by those skilled in the art that, based on the principle of the present system, various modifications and changes in form and detail may be made in the implementation of the functions of the above-described modules and units without departing from the principle, by arbitrarily combining the respective units or constituting sub-modules to be connected with other units. Such variations are within the scope of the present application.
Fig. 3 is an exemplary schematic diagram of a Cloud VR connectionless streaming system provided in accordance with some embodiments of the present application. As shown in fig. 3, the Cloud VR connectionless streaming system 300 may include at least one head-up device 330; a 3D application system 310; a Steam VR system 320; a Cloud VR system 200. The 3D application system may include IdeaVR, Unity, and urea 4, etc. In some embodiments according to the present application, the Cloud VR system 200 can include a Cloud VR server 210, a Cloud VR driver 220, and a Cloud VR client 230.
According to some embodiments of the present application, the Cloud VR system 200 can be integrated with the Steam VR system 320 under the same local area network, wherein the Cloud VR driver 220 can be integrated inside the Steam VR system 320. The CloudVR server 210 can obtain the rendering image data of the 3D application system 310 through the Steam VR system 320 and encode the rendering image data into video stream data with a set format. The formatted video stream data may include h.264 and/or h.265 formatted video, etc. The Cloud VR driver 220 can transmit the video stream data over a network to the Cloud VR client 230 of the head up device 330. The Cloud VR client 230 can be installed in the head-up device 330. The Cloud VR client 230 may decode and render the video stream data.
According to some embodiments of the present application, the Cloud VR server 210 is used for connection management. The Cloud VR server 210 can start the Steam VR system 320 and obtain the state of the Cloud VR client 230 through the Cloud VR driver 220 integrated within the Steam VR system 320. The Cloud VR server 210 may control the Cloud VR driver 220 to connect to the Cloud VR client 230 by sending instructions to the Cloud VR driver 220. As an example, the Cloud VR server 210 may control the resolution of rendering, the encoding format of the video stream data, the output location of the log. The code may include H264 or H265, etc.
According to some embodiments of the present application, the Cloud VR driver 220 may be integrated inside the Steam VR system 320, and the Cloud VR driver 220 may be a hub for the Cloud VR server 210 and the Cloud VR client 230. When the SteamVR system 320 receives image data sent by the 3D application system 310, the Cloud VR driver 220 can encode the image data. The encoded video stream data may be sent over a network to the Cloud VR client 230. In some embodiments, the Cloud VR driver 220 may be used to perform video encoding, network transmission adaptive code rate, command control, and the like. The video coding may include AVC and/or HEVC, among others.
According to some embodiments of the present application, the Cloud VR client 230 may be installed in the head-up device 330. The Cloud VR client 230 may receive the video stream data sent by the Cloud VR driver 220 over the network. Further, the Cloud VR client 230 may decode the video stream data encoded into the set format using a decoding tool. In some embodiments, the Cloud VR client 230 may draw video stream data to the screen using OpenGL. The Cloud VR client 230 may be an android native application or the like. The decoding tool may comprise MediaCodec, etc.
It should be noted that the above description of the Cloud VR connectionless streaming system 300 is for convenience only and should not limit the scope of the present application. It will be understood by those skilled in the art that, based on the principle of the present system, various modifications and changes in form and detail may be made in the implementation of the functions of the above-described modules and units without departing from the principle, by arbitrarily combining the respective units or constituting sub-modules to be connected with other units. Such variations are within the scope of the present application.
Fig. 4 is an exemplary flow diagram of a connection method of a Cloud VR connectionless streaming system provided in accordance with some embodiments of the present application. As shown in fig. 4, the process 400 may be implemented by the Cloud VR connectionless streaming system 300. In some embodiments, the Cloud VR connectionless streaming system 300 may be initiated automatically or by command. The instructions may include user instructions, system instructions, action instructions, and the like, or a combination of the several. As an example, the system instructions may be generated from information obtained by a sensor. The user instructions may include voice, gestures, actions, client 117, and/or virtual keys, etc., or a combination of the several.
At 401, a configuration file is written to a common memory of a system. The operations 401 may be implemented by the CloudVR server 210 of the Cloud VR system 200. In some embodiments, when the Cloud VR connectionless streaming system 300 is started, the Cloud VR server 210 is started, and the Cloud VR server 210 may write the configuration file into the public memory of the Cloud VR connectionless streaming system 300.
At 402, the Cloud VR drive is started by the Steam VR, the configuration file is read and initialized. Operation 402 may be implemented by the Cloud VR driver 220 of the Cloud VR system 200. In some embodiments, in the Cloud VR connectionless streaming system 300, the Steam VR system may be activated with one key, and the Steam VR system may activate the Cloud VR driver 220. The Cloud VR driver 220 may read the configuration file of operation 401 to obtain the configuration information. Further, the Cloud VR driver 220 may be initialized.
At 403, the Cloud VR client is started by the head-up device, sending a broadcast of the head-up information to the local area network. Operation 403 may be implemented by the Cloud VR client 230 of the Cloud VR system 200. In some embodiments, the Cloud VR client 230 starts up through the head-up device 130, at which time the Cloud VR client 230 can send a broadcast of the head-up information to the local area network. The head display information may include, but is not limited to, head display ID, resolution, left and right eye FOVs, device type, etc. The resolution may be the best resolution for the device, which may be sent directly to the Cloud VR driver 220 and applied to the encoding setup, as an example. The device type may include, but is not limited to, one or a combination of several of OculusQuest, OculusGo, XiaomiVR, and the like.
At 404, the broadcast is received and the head up information is saved by the Cloud VR driver. Operation 404 may be implemented by the Cloud VR driver 220 of the Cloud VR system 200. In some embodiments, the Cloud VR driver 220 may receive the broadcast. The broadcast contains overhead information that the Cloud VR driver 220 can store for query by the Cloud VR server 210.
At 405, the head display information is queried and displayed by the Cloud VR server and a connect command is sent to the Cloud VR driver. Operation 405 may be implemented by the Cloud VR server 210 of the Cloud VR system 200. In some embodiments, the Cloud VR server 210 can query the overhead information, and further, the Cloud VR server 210 can display the queried overhead information in the interface. In some embodiments, a user may select an overhead message to connect to via the interface. At this time, the Cloud VR server 210 can send a connection command to the Cloud VR driver 220 and specify the head display information of the connection. The overhead information may include address information.
At 406, connect to the Cloud VR client through the Cloud VR driver and return the result to the Cloud VR server. Operation 406 may be implemented by the Cloud VR driver 220 of the Cloud VR system 200. In some embodiments, the CloudVR driver 220 may connect to the Cloud VR client 230 and return the results to the Cloud VR server 210. To this end, the CloudVR connectionless streaming system 300 has completed supporting the connection.
It should be noted that the above description of the process 400 is merely for convenience of description and is not intended to limit the present application to the scope of the illustrated embodiments. It will be understood by those skilled in the art that various modifications and changes in form and detail may be made in the functions implementing the above-described processes and operations based on the principles of the present system, in any combination of operations or in combination with other operations constituting sub-processes without departing from the principles. For example, the flow 400 may further include completing operations to support the connection, and the like. Such variations are within the scope of the present application.
In summary, according to the Cloud VR connectionless streaming system and the connection method of the embodiment of the application, the VR all-in-one machine replaces the traditional head display, and data is transmitted through WIFI without redundant cable connection, so that the Cloud VR connectionless streaming system is realized, and VR experience of a user is comprehensively improved.
It is to be noted that the above-described embodiments are merely examples, and the present application is not limited to such examples, but various changes may be made.
It should be noted that, in the present specification, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that the series of processes described above includes not only processes performed in time series in the order described herein, but also processes performed in parallel or individually, rather than in time series.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer program instructions, and the program can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the invention has been described with reference to a number of illustrative embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.
Claims (10)
1. A Cloud VR connectionless streaming system, comprising:
at least one head display device;
a 3D application system;
a Steam VR system;
a Cloud VR system including a Cloud VR server, a Cloud VR driver and a Cloud VR client;
under the same local area network, the Cloud VR system is integrated with the Steam VR system, wherein the Cloud VR drive is integrated inside the Steam VR system;
the Cloud VR server side obtains rendering image data of the 3D application system through the Steam VR system and encodes the rendering image data into video stream data with a set format;
the Cloud VR driver transmits the video to a Cloud VR client of the head display device through a network, and the Cloud VR client is installed on the head display device;
and the Cloud VR client decodes and renders the video.
2. The system of claim 1, wherein the Cloud VR server is configured for connection management, further comprising:
starting a Steam VR system, and acquiring the state of a Cloud VR client through a Cloud VR drive integrated in the Steam VR system;
controlling the Cloud VR driver to connect to the Cloud VR client by sending instructions to the Cloud VR driver.
3. The system of claim 2, wherein the Cloud VR server can control the resolution of the rendering, the encoding format of the video stream data, and the output location of the log.
4. The system of claim 1, wherein the Cloud VR client is installed at the head-up device, further comprising:
receiving video stream data sent by the Cloud VR driver through a network;
decoding the video stream data coded into the set format by using a decoding tool;
and drawing the video stream data to a screen by utilizing OpenGL.
5. The system of claim 4, wherein the Cloud VR client is an android native application and the decoding tool comprises MediaCodec.
6. The system of claim 1, wherein the Cloud VR driver is integrated within the Steam VR system, the Cloud VR driver being a hub of a Cloud VR server and a Cloud VR client, further comprising:
the Steam VR system receives image data sent by the 3D application;
the Cloud VR drive encodes the image data;
and the coded video stream data is sent to the Cloud VR client through the network.
7. The system of claim 6, wherein the code comprises H264 or H265.
8. The system of claim 1, wherein the 3D application system comprises Idea VR, Unity, and urea 4.
9. A method for connecting a Cloud VR connectionless streaming system, comprising:
writing the configuration file into a public memory of the system;
starting a Cloud VR drive through a Steam VR, and reading a configuration file and initializing the configuration file by the Cloud VR drive;
starting a Cloud VR client through a head display device, and sending a broadcast of head display information to a local area network;
receiving the broadcast through the Cloud VR drive and storing the head display information;
inquiring and displaying head display information through the Cloud VR server, and sending a connection instruction to the Cloud VR driver;
connect to the Cloud VR client through the Cloud VR driver and return the results to the Cloud VR server.
10. The method of claim 9, wherein the overhead information comprises: head display ID, resolution, FOV of left and right eyes, and equipment type; the resolution can be sent to a CloudVR driver for application to an encoding setting; the connection instruction comprises address information of the head display information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394536.5A CN111179437B (en) | 2019-12-30 | 2019-12-30 | Cloud VR connectionless streaming system and connection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394536.5A CN111179437B (en) | 2019-12-30 | 2019-12-30 | Cloud VR connectionless streaming system and connection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111179437A true CN111179437A (en) | 2020-05-19 |
| CN111179437B CN111179437B (en) | 2023-08-08 |
Family
ID=70654238
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911394536.5A Active CN111179437B (en) | 2019-12-30 | 2019-12-30 | Cloud VR connectionless streaming system and connection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111179437B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111901628A (en) * | 2020-08-03 | 2020-11-06 | 江西科骏实业有限公司 | Cloud rendering method based on zSpace desktop VR all-in-one machine |
| CN112256317A (en) * | 2020-10-21 | 2021-01-22 | 上海曼恒数字技术股份有限公司 | Rapid construction method, medium and equipment of virtual reality immersion type large-screen tracking system |
| CN112363478A (en) * | 2020-11-20 | 2021-02-12 | 上海电气集团股份有限公司 | Cloud VR digital twin production line architecture based on industrial internet platform and construction method thereof |
| CN112947751A (en) * | 2021-02-07 | 2021-06-11 | 杭州小派智能科技有限公司 | Method and system for adjusting head display and picture display effect in virtual reality picture |
| CN113141346A (en) * | 2021-03-16 | 2021-07-20 | 青岛小鸟看看科技有限公司 | Streaming-based VR-multiservice system and method |
| CN113473180A (en) * | 2021-08-06 | 2021-10-01 | 北京博良胜合科技有限公司 | 5G or wireless-based Cloud XR data transmission method and device, storage medium and electronic device |
| CN113784105A (en) * | 2021-09-10 | 2021-12-10 | 上海曼恒数字技术股份有限公司 | Information processing method and system for immersive VR terminal |
| WO2021249562A1 (en) * | 2020-06-12 | 2021-12-16 | 华为技术有限公司 | Information transmission method, related device, and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120224694A1 (en) * | 2012-05-14 | 2012-09-06 | Wei Lu | Classified relation networking optimization platform in open wireless architecture (owa) mobile cloud terminal device |
| CN102930705A (en) * | 2012-09-26 | 2013-02-13 | 腾讯科技（深圳）有限公司 | Device control method and device |
| CN105592103A (en) * | 2016-02-02 | 2016-05-18 | 郭小虎 | Synchronous display method of virtual reality equipment and mobile equipment based on Unity3D |
| US20170180800A1 (en) * | 2015-09-09 | 2017-06-22 | Vantrix Corporation | Method and System for Selective Content Processing Based on a Panoramic Camera and a Virtual-Reality Headset |
| CN110266664A (en) * | 2019-06-05 | 2019-09-20 | 中国联合网络通信有限公司广州市分公司 | A kind of Cloud VR video living transmission system based on 5G and MEC |
| US20190370926A1 (en) * | 2018-05-30 | 2019-12-05 | Sony Interactive Entertainment LLC | Multi-server cloud virtual reality (vr) streaming |
-
2019
- 2019-12-30 CN CN201911394536.5A patent/CN111179437B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120224694A1 (en) * | 2012-05-14 | 2012-09-06 | Wei Lu | Classified relation networking optimization platform in open wireless architecture (owa) mobile cloud terminal device |
| CN102930705A (en) * | 2012-09-26 | 2013-02-13 | 腾讯科技（深圳）有限公司 | Device control method and device |
| US20170180800A1 (en) * | 2015-09-09 | 2017-06-22 | Vantrix Corporation | Method and System for Selective Content Processing Based on a Panoramic Camera and a Virtual-Reality Headset |
| CN105592103A (en) * | 2016-02-02 | 2016-05-18 | 郭小虎 | Synchronous display method of virtual reality equipment and mobile equipment based on Unity3D |
| US20190370926A1 (en) * | 2018-05-30 | 2019-12-05 | Sony Interactive Entertainment LLC | Multi-server cloud virtual reality (vr) streaming |
| CN110266664A (en) * | 2019-06-05 | 2019-09-20 | 中国联合网络通信有限公司广州市分公司 | A kind of Cloud VR video living transmission system based on 5G and MEC |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021249562A1 (en) * | 2020-06-12 | 2021-12-16 | 华为技术有限公司 | Information transmission method, related device, and system |
| CN111901628A (en) * | 2020-08-03 | 2020-11-06 | 江西科骏实业有限公司 | Cloud rendering method based on zSpace desktop VR all-in-one machine |
| CN112256317A (en) * | 2020-10-21 | 2021-01-22 | 上海曼恒数字技术股份有限公司 | Rapid construction method, medium and equipment of virtual reality immersion type large-screen tracking system |
| CN112256317B (en) * | 2020-10-21 | 2022-07-29 | 上海曼恒数字技术股份有限公司 | Rapid construction method, medium and equipment of virtual reality immersion type large-screen tracking system |
| CN112363478A (en) * | 2020-11-20 | 2021-02-12 | 上海电气集团股份有限公司 | Cloud VR digital twin production line architecture based on industrial internet platform and construction method thereof |
| CN112947751A (en) * | 2021-02-07 | 2021-06-11 | 杭州小派智能科技有限公司 | Method and system for adjusting head display and picture display effect in virtual reality picture |
| CN113141346A (en) * | 2021-03-16 | 2021-07-20 | 青岛小鸟看看科技有限公司 | Streaming-based VR-multiservice system and method |
| WO2022193582A1 (en) * | 2021-03-16 | 2022-09-22 | 青岛小鸟看看科技有限公司 | Streaming-based vr multi-split system and method |
| US11663795B2 (en) | 2021-03-16 | 2023-05-30 | Qingdao Pico Technology Co., Ltd. | Streaming-based VR multi-split system and method |
| CN113473180A (en) * | 2021-08-06 | 2021-10-01 | 北京博良胜合科技有限公司 | 5G or wireless-based Cloud XR data transmission method and device, storage medium and electronic device |
| CN113473180B (en) * | 2021-08-06 | 2024-01-30 | 北京博良胜合科技有限公司 | Wireless-based Cloud XR data transmission method and device, storage medium and electronic device |
| CN113784105A (en) * | 2021-09-10 | 2021-12-10 | 上海曼恒数字技术股份有限公司 | Information processing method and system for immersive VR terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111179437B (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111179437B (en) | Cloud VR connectionless streaming system and connection method | |
| CN113038287B (en) | Method and device for realizing multi-user video live broadcast service and computer equipment | |
| CN103024063A (en) | Data sharing method, clients and data sharing system | |
| CN106576158A (en) | Immersive video | |
| CN104573540A (en) | Mobile terminal user privacy protection method and system | |
| US20170195617A1 (en) | Image processing method and electronic device | |
| CN103780949B (en) | A kind of multi-medium data method for recording | |
| CN107995482B (en) | Video file processing method and device | |
| CN110969572B (en) | Face changing model training method, face exchange device and electronic equipment | |
| US20180373736A1 (en) | Method and apparatus for storing resource and electronic device | |
| CN104050040A (en) | Media playback workload scheduler | |
| CN105550934A (en) | System and method for pushing WeChat soft advertisement in virtual reality | |
| CN106572353A (en) | Wireless transmission method and wireless transmission device for virtual reality, terminal, and head-mounted display equipment | |
| CN106850577B (en) | Data interaction method and device, first virtual reality terminal and conference server | |
| US10043302B2 (en) | Method and apparatus for realizing boot animation of virtual reality system | |
| CN115103175A (en) | Image transmission method, device, equipment and medium | |
| CN104837046A (en) | Multi-media file processing method and device | |
| CN111208962A (en) | Display equipment connection control method and system | |
| CN101489190A (en) | Mobile terminal, multimedia information sharing method and system thereof | |
| CN112363791A (en) | Screen recording method and device, storage medium and terminal equipment | |
| CN203241620U (en) | Head-mounted display | |
| CN112604279A (en) | Special effect display method and device | |
| CN113784105A (en) | Information processing method and system for immersive VR terminal | |
| US20140330957A1 (en) | Widi cloud mode | |
| CN109300177B (en) | Picture processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |