CN114998494A - Digital twin stream pushing method based on cloud rendering service - Google Patents
Digital twin stream pushing method based on cloud rendering service Download PDFInfo
- Publication number
- CN114998494A CN114998494A CN202210515908.0A CN202210515908A CN114998494A CN 114998494 A CN114998494 A CN 114998494A CN 202210515908 A CN202210515908 A CN 202210515908A CN 114998494 A CN114998494 A CN 114998494A
- Authority
- CN
- China
- Prior art keywords
- streaming
- browser
- application program
- signaling
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000011664 signaling Effects 0.000 claims abstract description 20
- 230000006835 compression Effects 0.000 claims abstract description 4
- 238000007906 compression Methods 0.000 claims abstract description 4
- 238000004891 communication Methods 0.000 claims description 5
- 238000012800 visualization Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/31—Programming languages or programming paradigms
- G06F8/315—Object-oriented languages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Computer Graphics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a digital twin stream pushing method based on cloud rendering service, which is mainly completed by a three-dimensional application program, a signaling, a Web server and a client browser which run on a rendering server; integrating a pixel streaming plug-in the three-dimensional application program, wherein the pixel streaming plug-in encodes a final result of each rendering frame by using H.264 video compression, packages the video frames into a media streaming together with audio, and sends the streaming to one or more connected browsers; the signaling and Web server is used to negotiate the connection between the browser and the pixel streaming plug-in and provide the browser with HTML and JavaScript environments to play the media stream. The cloud-side rendering server has a powerful GPU, does not need to download large executable files or content files in advance, does not need to install any content, and can support multiple platforms without creating and issuing multiple independent packages.
Description
Technical Field
The invention relates to the technical field of three-dimensional visualization, in particular to a digital twin stream pushing method based on cloud rendering service.
Background
Three-dimensional visualization may be understood as the process of creating graphics and rendering designs and presentations using three-dimensional software. In short, the data of various industries are displayed in a three-dimensional manner through the three-dimensional visual effect. The application of the three-dimensional visualization technology is to perform efficient and high-precision rendering modeling on one hand, so that the whole elements of a physical world are restored in a digital scene; on the other hand, the communication between the digital world and the physical world is realized by combining the processing and calculation of real-time data, and a digital twin of the physical world is constructed. The three-dimensional visualization technology is one of key foundations for realizing city twinning and industry twinning.
Conventional three-dimensional visualization applications are largely classified into the following two modes.
Firstly, the method is developed based on html5, a browser is used for running, a WebGL interface of the browser is called through a JavaScript script, and loading of a three-dimensional model and display of a user interface are achieved. And receiving input of a mouse and a keyboard through a JavaScript script to realize interaction between a three-dimensional interface and a user. When the three-dimensional model is large and the scene is complex, the time consumption for downloading the model from the server is long, the loading is slow, and the rendering effect is general.
And secondly, a three-dimensional engine based on c/c + + is used for development and is packaged into a client program to run, a three-dimensional model is integrated in the client, only data information to be displayed needs to be acquired from a server, the three-dimensional scene access is smooth in the running process, and the rendering effect is good. The mode needs to be accessed by installing a client, has higher requirements on the performance of a machine running a client program, cannot be accessed across platforms, and needs to be developed independently for each platform.
Disclosure of Invention
Aiming at the defects of the existing three-dimensional visualization technology, the invention provides a digital twin stream pushing method based on cloud rendering service, and the loading of a three-dimensional model and the effect realization of a scene adopt a pixel streaming mode.
A digital twin stream pushing method based on cloud rendering services is mainly completed by a three-dimensional application program running on a rendering server, a signaling, a Web server and a client browser.
A pixel streaming plug-in is integrated in the three-dimensional application program, encodes the final result of each rendered frame by using H.264 video compression, packages the video frames into a media streaming along with audio, and sends the streaming to one or more connected browsers.
The signaling and Web server is used to negotiate the connection between the browser and the pixel streaming plug-in and provide the browser with HTML and JavaScript environments for playing the media streaming.
The specific process comprises the following steps:
step 1, when all pixel streaming components are started, a pixel streaming plug-in running in a three-dimensional application program firstly establishes a link to a signaling and a Web server;
step 2, the client is connected to the signaling and Web server, and the server provides an HTML page to the client, wherein the HTML page comprises a player control and a control code written in JavaScript;
step 3, when the user starts streaming, the signaling and the Web server are involved, and direct connection is established between the browser and the three-dimensional application program;
and 4, after the browser is connected with the three-dimensional application program, the pixel streaming plug-in unit directly streams the media to the browser, the input from the client is directly sent back to the three-dimensional application program by the JavaScript code of the player page, and the three-dimensional application program executes corresponding user operation and updates the media streaming.
Preferably, during the media streaming playing process, the signaling and Web server maintains its connection with the browser and the three-dimensional application.
Preferably, a WebRTC player is integrated in the page, and pixel streaming is performed between the pixel streaming plug-in and the browser by using a WebRTC point-to-point communication frame.
The invention has the beneficial effects that: 1. the rendering server at the cloud end has a powerful GPU, can easily run the rendering function of a three-dimensional application program, displays a complex scene at high resolution, does not need to download large executable files or content files in advance, does not need to install any content, can support multiple platforms without creating and issuing a plurality of independent packages, and can experience projects by using browsers of any platforms by users after Windows packaging application programs; 2. a WebRTC player is integrated in a page, and pixel streaming is carried out between a pixel streaming plug-in and a browser by using a WebRTC point-to-point communication frame, so that the interaction delay between a user and a three-dimensional application program is reduced to be low, and the experience of the user is greatly improved.
Drawings
FIG. 1 is a block diagram of a pixel streaming system architecture;
fig. 2 is a schematic diagram of a pixel streaming process.
Detailed Description
The invention is described in further detail below with reference to the drawings and the detailed description. The embodiments of the present invention have been presented for purposes of illustration and description, and are not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Example 1
A digital twin stream pushing method based on cloud rendering services is mainly completed by a three-dimensional application program running on a rendering server, a signaling, a Web server and a client browser together, as shown in figure 1.
A pixel streaming plug-in is integrated in the three-dimensional application program, encodes the final result of each rendered frame by using H.264 video compression, packages the video frames into a media streaming along with audio, and sends the streaming to one or more connected browsers.
The signaling and Web server is used to negotiate the connection between the browser and the pixel streaming plug-in and provide the browser with HTML and JavaScript environments to play the media stream.
The cloud rendering server has a powerful GPU, can easily run the rendering function of the three-dimensional application program, displays complex scenes at high resolution, does not need to download large executable files or content files in advance, does not need to install any content, and can support multiple platforms without creating and releasing multiple independent packages. After the Windows packaging application program is targeted, a user can experience the project by using a browser of any platform.
The specific process is shown in fig. 2, and comprises the following steps:
step 1, when all pixel streaming components are started, a pixel streaming plug-in running in a three-dimensional application program firstly establishes a link to a signaling and a Web server;
step 2, the client is connected to the signaling and Web server, and the server provides an HTML page to the client, wherein the HTML page comprises a player control and a control code written in JavaScript;
step 3, when the user starts streaming, the signaling and the Web server are involved, and direct connection is established between the browser and the three-dimensional application program;
and 4, after the browser is connected with the three-dimensional application program, the pixel streaming plug-in unit directly streams the media to the browser, the input from the client is directly sent back to the three-dimensional application program by the JavaScript code of the player page, and the three-dimensional application program executes corresponding user operation and updates the media streaming.
To reduce the requirements on machine performance and enable cross-platform access, pages were developed using HTML5, the loading of three-dimensional models and the effects of scenes enable the use of WebRTC-based pixel streaming. The specific implementation mode is as follows: a WebRTC player is integrated in a page, and pixel streaming is carried out between a pixel streaming plug-in and a browser by using a WebRTC point-to-point communication frame, so that the interaction delay between a user and a three-dimensional application program is reduced to be low, and the experience of the user is greatly improved.
During the media streaming playing process, the signaling and Web server maintains its connection with the browser and the three-dimensional application program, so as to remove the user from the streaming when necessary and handle the connection disconnection caused by the browser.
It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by one of ordinary skill in the art and related arts based on the embodiments of the present invention without any creative effort, shall fall within the protection scope of the present invention.
Claims (4)
1. A digital twin stream pushing method based on cloud rendering service is characterized by being mainly completed by a three-dimensional application program, a signaling, a Web server and a client browser which run on a rendering server;
integrating a pixel streaming plug-in the three-dimensional application program, wherein the pixel streaming plug-in encodes a final result of each rendering frame by using H.264 video compression, packages the video frames into a media streaming together with audio, and sends the streaming to one or more connected browsers;
the signaling and Web server is used to negotiate the connection between the browser and the pixel streaming plug-in and provide the browser with HTML and JavaScript environments for playing the media streaming.
2. The cloud rendering service-based digital twin stream pushing method according to claim 1, wherein the specific process comprises the following steps:
step 1, when all pixel streaming components are started, a pixel streaming plug-in running in a three-dimensional application program firstly establishes a link to a signaling and a Web server;
step 2, the client is connected to the signaling and the Web server, and the server provides an HTML page to the client, wherein the HTML page comprises a player control and a control code written in JavaScript;
step 3, when the user starts streaming, the signaling and the Web server are involved, and direct connection is established between the browser and the three-dimensional application program;
and 4, after the browser is connected with the three-dimensional application program, the pixel streaming plug-in unit directly streams the media to the browser, the input from the client is directly sent back to the three-dimensional application program by the JavaScript code of the player page, and the three-dimensional application program executes corresponding user operation and updates the media streaming.
3. The cloud rendering service based digital twin stream pushing method according to claim 2, wherein during the media streaming playing process, the signaling and Web server maintains its connection with the browser and the three-dimensional application program.
4. The cloud rendering service based digital twin stream pushing method according to claim 2, wherein a WebRTC player is integrated in a page, and a WebRTC peer-to-peer communication frame is used between the pixel streaming plug-in and the browser for pixel streaming.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210515908.0A CN114998494A (en) | 2022-05-12 | 2022-05-12 | Digital twin stream pushing method based on cloud rendering service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210515908.0A CN114998494A (en) | 2022-05-12 | 2022-05-12 | Digital twin stream pushing method based on cloud rendering service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114998494A true CN114998494A (en) | 2022-09-02 |
Family
ID=83027150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210515908.0A Pending CN114998494A (en) | 2022-05-12 | 2022-05-12 | Digital twin stream pushing method based on cloud rendering service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114998494A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514747A (en) * | 2022-09-23 | 2022-12-23 | 深圳市瑞云科技有限公司 | Distributed preview method of real-scene three-dimensional model based on webrtc |
| CN115904514A (en) * | 2022-11-17 | 2023-04-04 | 广州极数科技有限公司 | Cloud rendering pixel stream implementation method based on three-dimensional scene and terminal equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100045662A1 (en) * | 2006-10-02 | 2010-02-25 | Aftercad Software Inc. | Method and system for delivering and interactively displaying three-dimensional graphics |
| US20180143756A1 (en) * | 2012-06-22 | 2018-05-24 | Matterport, Inc. | Defining, displaying and interacting with tags in a three-dimensional model |
| CN114020474A (en) * | 2021-11-25 | 2022-02-08 | 元能星泰(广州)数字科技有限公司 | Dynamic distribution method and system for energy industry metacosmic digital twin cloud rendering |
-
2022
- 2022-05-12 CN CN202210515908.0A patent/CN114998494A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100045662A1 (en) * | 2006-10-02 | 2010-02-25 | Aftercad Software Inc. | Method and system for delivering and interactively displaying three-dimensional graphics |
| US20180143756A1 (en) * | 2012-06-22 | 2018-05-24 | Matterport, Inc. | Defining, displaying and interacting with tags in a three-dimensional model |
| CN114020474A (en) * | 2021-11-25 | 2022-02-08 | 元能星泰(广州)数字科技有限公司 | Dynamic distribution method and system for energy industry metacosmic digital twin cloud rendering |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514747A (en) * | 2022-09-23 | 2022-12-23 | 深圳市瑞云科技有限公司 | Distributed preview method of real-scene three-dimensional model based on webrtc |
| CN115514747B (en) * | 2022-09-23 | 2024-07-09 | 深圳市瑞云科技有限公司 | Distributed preview method of live-action three-dimensional model based on webrtc |
| CN115904514A (en) * | 2022-11-17 | 2023-04-04 | 广州极数科技有限公司 | Cloud rendering pixel stream implementation method based on three-dimensional scene and terminal equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114998494A (en) | Digital twin stream pushing method based on cloud rendering service | |
| CN110415325B (en) | Cloud rendering three-dimensional visualization realization method and system | |
| JP7446468B2 (en) | Video special effects processing methods, devices, electronic equipment and computer programs | |
| CN102323882B (en) | Data processing device and method applied to Web3D | |
| CN108337545A (en) | Media playback and media serving device for reproduced in synchronization video and audio | |
| CN102254292A (en) | Remote 3D instruction rendering system and method | |
| CN105262825A (en) | SPICE cloud desktop transporting and displaying method and system on the basis of H.265 algorithm | |
| Mouton et al. | Collaborative visualization: current systems and future trends | |
| CN108337560A (en) | Media playback and media serving device for playing media in web browser | |
| CN112669428B (en) | BIM model rendering method based on cooperation of server and client | |
| US20140344469A1 (en) | Method of in-application encoding for decreased latency application streaming | |
| CN113946402A (en) | Cloud mobile phone acceleration method, system, equipment and storage medium based on rendering separation | |
| KR20120119504A (en) | System for servicing game streaming according to game client device and method | |
| WO2020220773A1 (en) | Method and apparatus for displaying picture preview information, electronic device and computer-readable storage medium | |
| EP4018673A1 (en) | Data model for representation and streaming of heterogeneous immersive media | |
| CN111225232A (en) | Video-based sticker animation engine, realization method, server and medium | |
| CN113181646A (en) | Game data method and device, electronic equipment and storage medium | |
| WO2024037110A1 (en) | Data processing method and apparatus, device, and medium | |
| CN112316433A (en) | Game picture rendering method, device, server and storage medium | |
| WO2024082878A1 (en) | Rendering processing method and apparatus, electronic device, computer-readable storage medium, and computer program product | |
| CN113111277A (en) | Method for controlling large-screen of virtual engine PC (personal computer) by mobile terminal | |
| CN112565869A (en) | Window fusion method, device and equipment for video redirection | |
| KR20140133096A (en) | Virtual web iptv and streaming method using thereof | |
| CN115225615B (en) | Illusion engine pixel streaming method and device | |
| CN114222185B (en) | Video playing method, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |