CN114513520A - Web three-dimensional visualization technology based on synchronous rendering of client and server - Google Patents
Web three-dimensional visualization technology based on synchronous rendering of client and server Download PDFInfo
- Publication number
- CN114513520A CN114513520A CN202111616474.5A CN202111616474A CN114513520A CN 114513520 A CN114513520 A CN 114513520A CN 202111616474 A CN202111616474 A CN 202111616474A CN 114513520 A CN114513520 A CN 114513520A
- Authority
- CN
- China
- Prior art keywords
- data
- rendering
- loading
- screen
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 84
- 238000005516 engineering process Methods 0.000 title claims abstract description 24
- 238000012800 visualization Methods 0.000 title claims abstract description 22
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 15
- 230000005540 biological transmission Effects 0.000 claims abstract description 27
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 230000003068 static effect Effects 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 36
- 230000008569 process Effects 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 16
- 230000000750 progressive effect Effects 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 11
- 230000011664 signaling Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 230000006978 adaptation Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 claims description 2
- 238000012805 post-processing Methods 0.000 claims description 2
- 210000003128 head Anatomy 0.000 description 9
- 230000006870 function Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 244000035744 Hura crepitans Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 239000013585 weight reducing agent Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9577—Optimising the visualization of content, e.g. distillation of HTML documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
- H04L9/0618—Block ciphers, i.e. encrypting groups of characters of a plain text message using fixed encryption transformation
- H04L9/0631—Substitution permutation network [SPN], i.e. cipher composed of a number of stages or rounds each involving linear and nonlinear transformations, e.g. AES algorithms
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a Web three-dimensional visualization technology based on synchronous rendering of a client and a server; rendering of the rear-end server and rendering of the browser end are carried out synchronously, rendering and loading time is reduced by preprocessing three-dimensional geographic data, the front-end browser improves the first screen performance of the system in modes of rendering of first screen skeleton information, static file loading, lazy loading of second screen information and the like, the current three-dimensional system loading white screen problem is avoided, better use experience is brought to a user, meanwhile, the complexity of the front end is not too large, and meanwhile, system data safety can be guaranteed by using WebRTC peer-to-peer transmission.
Description
Technical Field
The invention relates to the technical field of three-dimensional visualization, in particular to a Web three-dimensional visualization technology based on synchronous rendering of a client and a server.
Background
The early website development whole system is rendered by a server, and the server directly produces a rendered corresponding HTML page and returns the HTML page to a client for display, usually PHP or JSP. Here JSP is taken as an example. The server finishes loading by analyzing the template file stored at the server side, in this case, the browser only analyzes HTML, and the java code is used for reading data from the database and dynamically placing the data in the page. With the occurrence of the front end Ajax, a development mode of separating the front end from the back end is provided, the back end only provides an API (application programming interface) to return data, the front end acquires the data through the Ajax and can render the data into a page through js (java script).
In recent years, a three-dimensional visualization technology is rapidly developed, so that applications need to achieve a better rendering effect, a development mode of isomorphic rendering appears, a server side performs rendering, and a client side performs interaction. The method comprises the steps that a server renders to generate HTML and initialization data, a client side receives codes and the initialization data, then patch and event binding are carried out on the DOM of the HTML to activate the client side (client-side rendering), and the whole process is isomorphic rendering. WebGL is a 3D drawing standard, JavaScript and OpenGL ES 2.0 are combined together in the drawing technical standard, a JavaScript binding of OpenGL ES 2.0 is added, WebGL can provide a hardware three-dimensional rendering function for HTML Canvas, application can load a three-dimensional scene and a model in a browser by means of a system display card, and complex navigation and data visualization can be created. However, the modules that the server renders the entire page for one instance are written and maintained by back-end personnel; another situation is that if a front-end person wants to develop a page, the front-end person needs to write page code in languages such as PHP and Java, and usually, HTML code and data and corresponding logic are mixed together, and writing and maintenance are very troublesome. Moreover, the traditional Web three-dimensional geographic information system uses a WebGL technology as a rendering technology, and is mainly limited in that the display effect completely depends on the functions of a client browser and the hardware performance of a terminal; the image quality depends on the display capability of the browser; the data must be downloaded to the client for experience and the raw material data is exposed. Meanwhile, the SPA frequently used in browser-side rendering integrally packages all JS, and the problem that the file is too large and long time is required before rendering is solved. Especially, when the network speed is poor, it is not a good experience to let the user wait for the white screen to end.
For example, a "point cloud data three-dimensional visualization rendering method and calculation method" disclosed in chinese patent literature, publication No. CN111127610A, specifically includes the following steps: s1, firstly, without excluding other data, generating point cloud data with a standardized definition format by a conversion means, and the invention relates to the technical field of point cloud data three-dimensional visualization. The point cloud data three-dimensional visualization rendering method and the point cloud data three-dimensional visualization rendering calculation method realize multiple rendering algorithms of point cloud data by performing point cloud rendering after data normalization and classification processing, and by performing vertex rendering, mapping rendering based on a three-dimensional grid, grid reconstruction rendering based on a triangulation algorithm, point cloud motion animation and point cloud particle animation. However, the operation of the algorithm needs the support of the background, and the full rendering aiming at the three-dimensional visualization under the condition of poor network can lead to complex data loading, thereby affecting the speed of the picture rendering and still failing to solve the limitation existing in the traditional rendering.
Disclosure of Invention
The method mainly aims at the problems of page jitter, memory overflow and long rendering first screen time caused by the difference between server rendering and browser rendering in the prior art; the Web three-dimensional visualization technology based on synchronous rendering of the client and the server is provided; the rendering of the rear-end server and the rendering of the browser end are carried out synchronously, the rendering and loading time is reduced by preprocessing three-dimensional geographic data, the front-end browser improves the first screen performance of the system in modes of rendering of first screen skeleton information, loading of static files, lazy loading of second screen information and the like, the problem of loading of a white screen of the current three-dimensional system is avoided, better use experience is provided for a user, and meanwhile the complexity of the front end is not too large.
The technical problem of the invention is mainly solved by the following technical scheme:
a Web three-dimensional visualization technology based on synchronous rendering of a client and a server comprises the following steps:
step S1, the back-end server carries out model lightweight, scene fine-grained and progressive loading pretreatment on the three-dimensional geographic data; the model is split into model units by the model lightweight, so that redundancy is reduced; the method comprises the steps that a scene fine-grained analysis is carried out on floor distribution conditions of buildings and communication relations among floors, and asymptotic transmission and loading of scene data in a roaming process are controlled; the basic data is subjected to progressive loading and priority processing, and the model precision is increased according to the viewpoint position; after preprocessing, the three-dimensional data is transmitted to the client side in a peer-to-peer mode based on WebRTC;
step S2, rendering the user data and the first screen data through an EJS browser by the client, namely writing the related parameters into an EJS template, and returning the rendered HTML document to the browser through node.js analysis; the client rendering comprises first screen framework information rendering, static file loading and picture loading;
step S3, layering according to DIV in the secondary screen, wherein each layer is independently used as a data interface for asynchronous loading, and each DIV layer only reserves the DIV label of the outermost layer when the page is initialized;
and step S4, after the user interacts, the browser makes independent judgment and data is re-rendered.
The loading and rendering of three-dimensional data are optimized, the loading amount is reduced for multiple times, the precision of the model is adjusted in real time by adding corresponding incremental networks according to the viewpoint positions, the first screen performance of the system is improved, the problem of loading a white screen of the current three-dimensional system is avoided, and better use experience is provided for users; the transmission speed of the three-dimensional data can be remarkably improved through peer-to-peer transmission based on the WebRTC, and a user can smoothly load the system under the conditions of lower hardware configuration and bandwidth without installing a special rendering plug-in.
Preferably, the model weight reduction preprocessing process in step S1 is as follows:
step A1, the three-dimensional model often has reusable units, which are different only in direction and spatial position, so that some model units can be rotated, scaled and translated to obtain other model units. Processing the repeated and symmetrical model, performing semantic analysis on the component level of the original data, removing geometric repeatability detection and repeatability redundancy, and only transmitting a minimum model unit;
and step A2, rotating, scaling and translating the minimum model unit to obtain a complex model unit, and reducing redundant data of the scene.
The model is split and the smallest unit element is found, only the smallest model unit is transmitted, the transmission complexity can be reduced, and the process of rendering and transmitting is accelerated. Reducing redundancy and increasing transfer speed.
Preferably, the scene refinement preprocessing procedure in step S1 is as follows:
b1, analyzing floors and space objects containing semantic information in BIM data of the building information model to form a primary structure space structure;
step B2, acquiring special members, such as floors of a spatial structure of a building, and analyzing by combining the spatial structure of the preliminary building to form floor distribution conditions of the whole building and communication relations among floors; in each floor, further in-floor communication relation analysis is carried out by combining a preliminary structure space structure, and an indoor subspace division and communication relation structure chart of the whole building is formed;
step B3, when the scene is roaming, the user's view is limited, and only interested in the scene around the user, so as to transmit and load the visible objects around the user's avatar, the current roaming requirement can be satisfied. Transmitting and loading visible objects around the user avatar; and judging the subspace of the user virtual avatar according to the current position of the user virtual avatar, calculating the object in the current view cone, reducing the loading data volume, effectively controlling the data volume needing to be loaded in the current view point range in the real-time roaming process, and realizing the asymptotic transmission and loading of the scene data.
A cone is a range of cones in a scene that are visible to a camera, the scene within the cone being visible and not visible otherwise. Therefore, the overall loading capacity can be reduced by directly calculating and displaying the objects in the view cones, the loading capacity in the roaming process can be further reduced by the mode, the transmission speed is improved as much as possible by the mode of reducing the loading capacity, and the situations of white screen blockage and the like can be avoided in the loading process.
Preferably, the progressive mesh preprocessing in step S1 is as follows:
step C1, the page loading emphasizes the look and feel of the user, so in a complex scene, the look and feel of the user are mainly considered during loading. When progressive grid preprocessing is carried out on a complex model which cannot be transmitted in real time in a low bandwidth in a scene, an original model is processed into a base network part and an incremental grid part which contain less data volume;
and step C2, when scene roaming is carried out, the base network data is processed in one step, and in the subsequent roaming process, the precision of the model is adjusted in real time by adding corresponding incremental networks according to the viewpoint positions.
The objective fact that the human eyes observe objects with a degree of clarity inversely proportional to distance is that objects are rough when they are farther away from the user's avatar and fine when they are closer. The page loading focuses on the look and feel of the user, and in order to accelerate the transmission speed without influencing the look and feel of the user, the method can be used when a complex scene is transmitted, so that the transmission quantity is reduced, and the user experience is guaranteed.
Preferably, the WebRTC-based peer-to-peer transmission in step S1 is peer-to-peer transmission; the method comprises the steps of establishing connection between two nodes by using RTCPeerConnection, creating RTCDataChannel and establishing a bidirectional data channel in peer-to-peer connection. The method is characterized in that a browser is used as a center, a protocol is established at an application level, and the purpose is to provide a function that a web developer can realize real-time communication based on simple JavaScript api, so that rich browser-side multimedia application can be rapidly developed. The application using the WebRTC does not need to install any browser plug-in and supports cross-platform. WebRTC provides a core technology required for video conferencing, and is currently widely applied to browser-based video conferencing. The basic data transmission by using the WebRTC only needs to use two APIs, and the RTCPeerConnection is a component used by the WebRTC for constructing stable and efficient stream transmission between point and point. RTCDataChannel enables a high throughput, low latency channel to be established between browsers (point-to-point) for the transmission of arbitrary data.
Preferably, the two nodes perform signaling transfer via the server, and the exchanged signaling includes: information of session; network configuration; and (5) media adaptation. Firstly, a connection is established between nodes by using an RTCPeerConnection, which requires two nodes to transmit signaling through a server, any mode can be selected and any protocol is adopted to transmit the signaling, the signaling to be exchanged has three types, namely, session information: to initialize communication and also to report errors; network configuration: such as IP addresses and ports; media adaptation: what encoders and resolutions the browsers of the sender and receiver are capable of accepting.
Preferably, in the step S2, the static file loading includes the following steps:
step S211, changing codes of the CSS and the JS related to the first screen rendering from a quoted form to an inline form, and reducing HTTP requests;
step S212, putting the JS code of the data request section to the head; placing the CSS style and JS data request rendered by the first screen at the head;
and step S213, after the style of the first screen and the data rendering are ensured to be successful, the logic delay post-processing execution of the second screen is carried out, and the related second screen data is triggered by the front end to be transmitted and reported in a delay manner, so that the network congestion is reduced.
The JS code of the data request part is put at the head part for ensuring that the data can be returned at the first time, compared with the data request put outside, the CSS style of the first screen rendering and the JS data request of the time of the loading round trip of the JS file are reduced to be put at the head part, and the purpose is to ensure that the CSS first screen style rendering and the request data can be immediately executed and returned when the HTML document is returned by the server.
Preferably, in step S2, the loading of the picture includes the following steps:
step S221, reducing HTTP requests of picture resources, merging the low-color-Position small pictures of the same module, merging the low-color-Position small pictures into one picture by using CSS-SPIRIT technology, and controlling display by using background Image and background Position attributes;
step S222, directly loading the picture displayed on the first screen to request the corresponding path resource; the secondary screen picture is lazily loaded, and an HTTP request is sent to the background only when a user requests secondary screen data; when a page is loaded, a resource path of an IMG label is stored in a custom attribute, and an SRC attribute points to a small picture as a placeholder to prevent an empty HTTP request;
step S223, after the rendering of the first screen is completed, traversing the DOM tree to obtain a real path of the lazy loading IMG and the offset distance of the current picture relative to the top of the document, and storing the attributes in a Hash array; after judging that the current page position appears in the visible range, assigning a real path address to the SRC, and deleting the loaded picture from the array;
and step S224, setting a feedback loading pre-known prompt in the process of requesting to wait.
The picture data preprocessed by the back-end server is processed, the request times are reduced, the HTTP request is sent to the background only when the user requests the secondary screen data, and the loaded data is deleted, so that loading errors are avoided. When the network environment cannot be guaranteed to cause the slow picture loading speed, the loading of the pre-known prompt symbols is given to the user, the user is guaranteed to know, and therefore the user experience is guaranteed.
Preferably, in step S4, the user interacts with the system to determine whether a mouse sliding event or a scrolling event occurs, calculates user operation data such as an offset of the browser, a height of the current viewing angle, and an angle of the current viewing angle, determines data required for the current screen display, and re-renders the screen display data. The rendering process can ensure real-time rendering of the front-end interface and meet the user experience.
The invention has the beneficial effects that:
1. the Web three-dimensional visualization technology based on the synchronous rendering of the client and the server is provided, the loading and rendering of three-dimensional data can be optimized, the first screen performance of the system is improved, the problem of loading a white screen of the current three-dimensional system can be avoided, and better use experience is provided for users.
WebRTC can guarantee system data security: WebRTC forced encryption, wherein all transmitted data of media and data are encrypted by using standard AES; WebRTC can only be used if the user explicitly needs to use it; sanboxed, No Plugins, WebRTC runs inside the sandbox of the Chrome browser. Therefore, the browser and the user can be completely protected even if someone tries to attack the webRTC inside the Chrome; and the WebRTC Security Architecture protocol is adopted to ensure the Security of data.
3. The transmission speed of the three-dimensional data can be remarkably improved through peer-to-peer transmission based on the WebRTC, and a user can smoothly load the system under the conditions of lower hardware configuration and bandwidth without installing a special rendering plug-in.
Drawings
FIG. 1 is a rendering flow chart of the method;
FIG. 2 is a back-end server pre-rendering flow diagram;
FIG. 3 is a flow chart of browser-side rendering;
FIG. 4 is a schematic diagram of peer-to-peer transmission based on WebRTC;
FIG. 5 is a static file load.
Detailed Description
It should be understood that the examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
The technical scheme of the invention is further specifically described by the following embodiments.
The scheme is based on the principle that the first screen loading time is shortest, and adopts a mixed mode of client rendering and server rendering to shunt Web data and realize asynchronous loading and local caching of different modules.
The method comprises the steps that a first client performs model lightweight, scene fine-grained and progressive loading on three-dimensional data with huge data volume, and the three-dimensional data is pre-rendered through a high-performance rear-end server and is transmitted to a client on the basis of WebRTC;
the client renders the user data and the first screen data through an EJS (JavaScript template library) browser, namely writes the related parameters into an EJS template, and returns a rendered HTML document to the browser through node.js analysis. And layering the secondary screen according to DIV, wherein each layer is independently made into a data interface for asynchronous loading, and each DIV layer only reserves the DIV label at the outermost layer when the page is initialized.
And after the user interacts, the browser judges and renders the data again. The method has the advantages that the first screen performance of the system is improved, the problem of loading a white screen of the current three-dimensional system can be avoided ideally, better use experience is provided for a user, and meanwhile, the complexity of the front end is not too high. The flow chart of the scheme is shown in figure 1; the back-end rendering and browser rendering flowcharts are respectively shown in fig. 2 and fig. 3.
The three-dimensional geographic data preprocessing comprises the following steps:
firstly, model lightweight, processing a plurality of repeated symmetrical models, such as axisymmetric models, and taking one half of the axisymmetric model as a minimum model unit. The semantic analysis of a component level, the geometric repeatability detection and the repeatability redundancy removal are carried out on the original data, only the minimum model unit is transmitted, the complex model unit is obtained by rotating, zooming and translating the minimum model unit, and the redundant data of the whole scene is reduced.
Secondly, scene fine-grained, namely, a preliminary structure Space structure is formed by analyzing a floor (storage) and a Space (Space) object which contain semantic Information in Building Information Model (BIM) (building Information modeling) data; secondly, obtaining special components such as the floor of the whole building, carrying out further floor analysis by combining with a primary building space structure to form the floor distribution condition of the whole building and the communication relation among floors, and finally carrying out further intra-floor communication relation analysis by combining with the primary space structure inside each floor to form the indoor subspace division and communication relation structure chart of the whole building. When the scene roams, the visual field of the user is limited, and only the surrounding scene is interested, so that the current roaming requirement can be met only by transmitting and loading visible objects around the virtual avatar of the user. A viewing cone refers to a cone range of a scene in which a camera is visible, and within which a scene is visible, and not visible otherwise. The subspace of the user virtual avatar is judged according to the current position of the user virtual avatar, the object in the current view cone is calculated, the loaded data volume can be further reduced, the data volume needing to be loaded in the current view point range can be effectively controlled all the time in the real-time roaming process, and the progressive transmission and loading of scene data are realized.
Thirdly, Progressive Mesh (Progressive Mesh, PM) is also a common means for current complex model transmission. The objective fact that the degree of clearness of the human eyes for observing objects is inversely proportional to the distance is utilized, and the objects are rough when the objects are far away from the virtual avatar of the user and fine when the objects are close to the virtual avatar of the user. In the PM preprocessing of some complex models which cannot be transmitted in real time at low bandwidth in a scene, the original models are processed into a base network and an incremental grid which contain a small amount of data. When scene roaming is carried out, the base network data is processed firstly, and in the subsequent roaming process, the accuracy of the model is adjusted in real time by adding corresponding incremental networks according to the viewpoint positions. Therefore, in the whole roaming process, the user experience quality is not influenced, and the incremental data can be processed as required.
While a server rendering process based on peer-to-peer transmission of WebRTC is shown in fig. 4. The protocol target established at an application level by taking a browser as a center is to provide a function that a web developer can realize real-time communication based on simple JavaScript, so that rich browser-side multimedia application is developed quickly. The application using the WebRTC does not need to install any browser plug-in and supports cross-platform. WebRTC provides a core technology required for video conferencing, and is currently widely applied to browser-based video conferencing. The basic data transmission by using the WebRTC only needs to use two APIs, and the RTCPeerConnection is a component used by the WebRTC for constructing stable and efficient stream transmission between point and point. RTCDataChannel enables a high throughput, low latency channel to be established between browsers (point-to-point) for the transmission of arbitrary data. Firstly, a connection is established between nodes by using an RTCPeerConnection, which requires two nodes to transmit signaling through a server, any mode can be selected and any protocol is adopted to transmit the signaling, the signaling to be exchanged has three types, namely, session information: to initialize communication and also to report errors; network configuration: such as IP addresses and ports; media adaptation: what encoders and resolutions the browsers of the sender and receiver are capable of accepting. After the connection is established, an RTCDataChannel is created to establish a bi-directional data channel in the peer-to-peer connection based on which three-dimensional geographic data can be passed between the browsers.
When the client renders user data and first screen data through an EJS browser, the first screen framework information rendering, static file loading and picture loading are mainly performed through three steps; wherein the picture loading is picture lazy loading.
The first screen skeleton information is rendered by rendering the core part such as a menu preferentially in an isomorphic mode. The menu and page Skeleton (skeletton Screen) can be rendered first using isomorphism. And information is prompted to the user, and the endless waiting time is reduced.
The static file loading process is shown in FIG. 5; first, the code for CSS and JS related to the head screen rendering changes the referenced form to an inline form, thereby reducing HTTP requests. Second, the JS code of the data request portion is put at the head in order to ensure that data can be returned at the first time, reducing the time to and from the loading of the JS file compared to putting the data request externally. Third, the CSS style and JS data request for the head screen rendering is placed at the head to ensure that CSS head screen style rendering and request data can be executed and returned immediately when the server returns the HTML document. Fourthly, after the style of the first screen and the data rendering are ensured to be successful, the logic delay back processing of the second screen is executed, and related second screen data are triggered by the front end to be sent and reported in a delay mode, so that network congestion is reduced.
The picture loading mainly adopts a picture lazy loading mode, and a picture lazy loading strategy is realized from three aspects: the number of picture requests is reduced, the picture is asynchronously requested in the secondary screen, and the downward compatibility is improved.
The HTTP requests of picture resources are reduced, merging processing is carried out on the small pictures with low color positions of the same module, the small pictures are merged into one picture by using a CSS-SPIRIT technology, and display is controlled by using background Image and background Position attributes.
Directly loading a picture displayed on a first screen to request a corresponding path resource; the secondary screen picture is lazily loaded and only when the user requests the secondary screen data, an HTTP request is sent to the background. When a page is loaded, a resource path of the img tag is stored in a custom attribute, and the src attribute points to a small picture as a placeholder to prevent an empty HTTP request. And after the rendering of the first screen is finished, traversing the DOM tree to obtain the true path of the lazy loading img and the offset distance of the current picture relative to the top of the document, and storing the attributes in the hash array. And after judging that the current page position appears in the visual field range, assigning the real path address to src, and deleting the loaded picture from the array.
Finally, the downward compatibility means that the user experience is still guaranteed due to slow picture speed or failed request under the condition that the network environment cannot be guaranteed. Factors influencing the picture loading time are many, such as server response time, network bandwidth, picture size and the like; in the process of requesting for waiting, a feedback loading pre-known prompt is set, and good user experience during waiting is guaranteed.
In the final user interaction stage, the data after the map operation is re-rendered; the method comprises the steps of interacting a user with a system, particularly operating a three-dimensional scene, judging whether a mouse sliding event or a rolling event occurs or not, calculating user operation data such as the offset of a browser, the height of a current visual angle, the angle of the current visual angle and the like, judging data required by current screen display, and re-rendering screen display data.
Claims (9)
1. A Web three-dimensional visualization technology based on synchronous rendering of a client and a server is characterized by comprising the following steps:
step S1, the back-end server carries out model lightweight, scene fine-grained and progressive loading pretreatment on the three-dimensional geographic data; the model is divided into model units by model lightweight, so that redundancy is reduced; analyzing the floor distribution condition of the building and the communication relation among floors in a scene fine-grained manner, and controlling the asymptotic transmission and loading of scene data in the roaming process; the basic data is subjected to progressive loading and priority processing, and the model precision is increased according to the viewpoint position; after preprocessing, the three-dimensional data is transmitted to the client side in a peer-to-peer mode based on the WebRTC;
step S2, rendering the user data and the first screen data through an EJS browser by the client, namely writing the related parameters into an EJS template, and returning the rendered HTML document to the browser through node.js analysis; the client rendering comprises first screen framework information rendering, static file loading and picture loading;
step S3, layering according to DIV in the secondary screen, wherein each layer is independently used as a data interface for asynchronous loading, and each DIV layer only reserves the DIV label of the outermost layer when the page is initialized;
and step S4, after the user interacts, the browser makes independent judgment and data is re-rendered.
2. The technology for Web three-dimensional visualization based on client-server synchronous rendering of claim 1, wherein the model lightweight preprocessing procedure in step S1 is as follows:
step A1, processing the repeated and symmetrical model, performing semantic analysis of component level on the original data, removing geometric repeatability detection and repeatability redundancy, and only transmitting the minimum model unit;
and step A2, rotating, scaling and translating the minimum model unit to obtain a complex model unit, and reducing redundant data of the scene.
3. The technology for Web three-dimensional visualization based on synchronous rendering by client and server in claim 1, wherein the scene refinement preprocessing in step S1 is as follows:
b1, analyzing floors and space objects containing semantic information in BIM data of the building information model to form a primary structure space structure;
step B2, acquiring special components, and analyzing by combining with the space structure of the primary structure to form the floor distribution condition of the whole building and the communication relation among floors; in each floor, further in-floor communication relation analysis is carried out by combining a preliminary structure space structure, and an indoor subspace division and communication relation structure chart of the whole building is formed;
step B3, when the scene is roaming, transmitting and loading visible objects around the user virtual avatar; and judging the subspace of the user virtual avatar according to the current position of the user virtual avatar, calculating the object in the current view cone, reducing the loading data volume, effectively controlling the data volume needing to be loaded in the current view point range in the real-time roaming process, and realizing the asymptotic transmission and loading of the scene data.
4. The Web three-dimensional visualization technology based on synchronous rendering by client and server in claim 1, wherein the process of the progressive mesh preprocessing in step S1 is as follows:
step C1, when carrying out progressive grid preprocessing on a complex model which cannot be transmitted in real time at low bandwidth in a scene, processing the original model into a base network and an incremental grid which contain less data;
and step C2, when scene roaming is carried out, the base network data is processed firstly, and in the subsequent roaming process, the accuracy of the model is adjusted in real time by adding a corresponding incremental network according to the viewpoint position.
5. The Web three-dimensional visualization technology based on synchronous rendering of the client and the server according to claim 1, wherein the WebRTC-based peer-to-peer transmission in step S1 is peer-to-peer transmission; the method comprises the steps of establishing connection between two nodes by using RTCPeerConnection, creating RTCDataChannel and establishing a bidirectional data channel in peer-to-peer connection.
6. The Web three-dimensional visualization technology based on client-side and server synchronous rendering of claim 5, wherein two nodes are in signaling communication via a server, and the exchanged signaling comprises: information of session; network configuration; and (5) media adaptation.
7. The technology for Web three-dimensional visualization based on synchronized rendering of client and server in claim 1, wherein in the step S2, the static file loading comprises the following steps:
step S211, changing codes of the CSS and the JS related to the first screen rendering from a quoted form to an inline form, and reducing HTTP requests;
step S212, putting the JS code of the data request section to the head; placing the CSS style and JS data request for the first screen rendering at the head;
and step S213, after the style of the first screen and the data rendering are ensured to be successful, the logic delay post-processing execution of the second screen is carried out, and the related second screen data is triggered by the front end to be transmitted and reported in a delay manner, so that the network congestion is reduced.
8. The technology for Web three-dimensional visualization based on synchronous rendering by client and server in claim 1, wherein in the step S2, the loading of the picture includes the following steps:
step S221, reducing HTTP requests of picture resources, merging the low-color-Position small pictures of the same module, merging the low-color-Position small pictures into one picture by using CSS-SPIRIT technology, and controlling display by using background Image and background Position attributes;
step S222, directly loading the picture displayed on the first screen to request the corresponding path resource; the secondary screen picture is lazily loaded, and an HTTP request is sent to the background only when a user requests secondary screen data; when a page is loaded, a resource path of an IMG label is stored in a custom attribute, and an SRC attribute points to a small picture as a placeholder to prevent an empty HTTP request;
step S223, after the rendering of the first screen is completed, traversing the DOM tree to obtain a real path of the lazy loading IMG and the offset distance of the current picture relative to the top of the document, and storing the attributes in a Hash array; after judging that the current page position appears in the visible range, assigning a real path address to the SRC, and deleting the loaded picture from the array;
step S224, in the process of requesting for waiting, setting a feedback loading prediction prompt.
9. The technology of claim 1, wherein in step S4, the user interacts with the system to determine whether a mouse sliding event or a scrolling event occurs, calculates user operation data such as a browser offset, a current viewing angle height, and a current viewing angle, determines data required for current screen display, and re-renders screen display data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111616474.5A CN114513520B (en) | 2021-12-27 | 2021-12-27 | Web three-dimensional visualization method based on synchronous rendering of client and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111616474.5A CN114513520B (en) | 2021-12-27 | 2021-12-27 | Web three-dimensional visualization method based on synchronous rendering of client and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114513520A true CN114513520A (en) | 2022-05-17 |
| CN114513520B CN114513520B (en) | 2023-06-30 |
Family
ID=81548465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111616474.5A Active CN114513520B (en) | 2021-12-27 | 2021-12-27 | Web three-dimensional visualization method based on synchronous rendering of client and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114513520B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114756798A (en) * | 2022-06-13 | 2022-07-15 | 中汽创智科技有限公司 | Point cloud rendering method and system based on Web end and storage medium |
| CN115514747A (en) * | 2022-09-23 | 2022-12-23 | 深圳市瑞云科技有限公司 | Distributed preview method of real-scene three-dimensional model based on webrtc |
| CN115828007A (en) * | 2022-11-21 | 2023-03-21 | 上海秉匠信息科技有限公司 | Page display method, device, system, electronic equipment and medium |
| CN115934179A (en) * | 2022-09-26 | 2023-04-07 | 贝壳找房(北京)科技有限公司 | Service function control method and equipment |
| CN116977523A (en) * | 2023-07-25 | 2023-10-31 | 深圳市快速直接工业科技有限公司 | STEP format rendering method at WEB terminal |
| CN117095105A (en) * | 2023-08-29 | 2023-11-21 | 广州立心科技有限责任公司 | Light client browsing system and method for oversized three-dimensional model |
| CN117437342A (en) * | 2023-12-20 | 2024-01-23 | 深圳图为技术有限公司 | Three-dimensional scene rendering method, device and equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100045662A1 (en) * | 2006-10-02 | 2010-02-25 | Aftercad Software Inc. | Method and system for delivering and interactively displaying three-dimensional graphics |
| CN110008421A (en) * | 2018-11-09 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Page processing method, device, equipment and computer readable storage medium |
| CN111080782A (en) * | 2019-10-17 | 2020-04-28 | 湖北泰跃卫星技术发展股份有限公司 | Visual system based on live-action three-dimensional geographic information system |
| CN111369656A (en) * | 2020-03-04 | 2020-07-03 | 杭州群核信息技术有限公司 | WebGL-based editable large-scene progressive real-time rendering method |
| CN111639289A (en) * | 2020-05-13 | 2020-09-08 | 北京三快在线科技有限公司 | Webpage loading method and device |
| CN112163175A (en) * | 2020-10-14 | 2021-01-01 | 南京焦点领动云计算技术有限公司 | Speed optimization method for website mobile terminal |
| CN112765513A (en) * | 2020-06-16 | 2021-05-07 | 吉林吉动盘古网络科技股份有限公司 | Fine-grained Web3D online visualization method for large-scale building scene |
| CN112948043A (en) * | 2021-03-05 | 2021-06-11 | 吉林吉动盘古网络科技股份有限公司 | Fine-grained Web3D online visualization method for large-scale building scene |
-
2021
- 2021-12-27 CN CN202111616474.5A patent/CN114513520B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100045662A1 (en) * | 2006-10-02 | 2010-02-25 | Aftercad Software Inc. | Method and system for delivering and interactively displaying three-dimensional graphics |
| CN110008421A (en) * | 2018-11-09 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Page processing method, device, equipment and computer readable storage medium |
| CN111080782A (en) * | 2019-10-17 | 2020-04-28 | 湖北泰跃卫星技术发展股份有限公司 | Visual system based on live-action three-dimensional geographic information system |
| CN111369656A (en) * | 2020-03-04 | 2020-07-03 | 杭州群核信息技术有限公司 | WebGL-based editable large-scene progressive real-time rendering method |
| CN111639289A (en) * | 2020-05-13 | 2020-09-08 | 北京三快在线科技有限公司 | Webpage loading method and device |
| CN112765513A (en) * | 2020-06-16 | 2021-05-07 | 吉林吉动盘古网络科技股份有限公司 | Fine-grained Web3D online visualization method for large-scale building scene |
| CN112163175A (en) * | 2020-10-14 | 2021-01-01 | 南京焦点领动云计算技术有限公司 | Speed optimization method for website mobile terminal |
| CN112948043A (en) * | 2021-03-05 | 2021-06-11 | 吉林吉动盘古网络科技股份有限公司 | Fine-grained Web3D online visualization method for large-scale building scene |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114756798A (en) * | 2022-06-13 | 2022-07-15 | 中汽创智科技有限公司 | Point cloud rendering method and system based on Web end and storage medium |
| CN115514747A (en) * | 2022-09-23 | 2022-12-23 | 深圳市瑞云科技有限公司 | Distributed preview method of real-scene three-dimensional model based on webrtc |
| CN115934179A (en) * | 2022-09-26 | 2023-04-07 | 贝壳找房(北京)科技有限公司 | Service function control method and equipment |
| CN115934179B (en) * | 2022-09-26 | 2023-11-24 | 贝壳找房(北京)科技有限公司 | Service function control method and equipment |
| CN115828007A (en) * | 2022-11-21 | 2023-03-21 | 上海秉匠信息科技有限公司 | Page display method, device, system, electronic equipment and medium |
| CN115828007B (en) * | 2022-11-21 | 2023-08-04 | 上海秉匠信息科技有限公司 | Page display method, device and system, electronic equipment and medium |
| CN116977523A (en) * | 2023-07-25 | 2023-10-31 | 深圳市快速直接工业科技有限公司 | STEP format rendering method at WEB terminal |
| CN116977523B (en) * | 2023-07-25 | 2024-04-26 | 快速直接(深圳)精密制造有限公司 | STEP format rendering method at WEB terminal |
| CN117095105A (en) * | 2023-08-29 | 2023-11-21 | 广州立心科技有限责任公司 | Light client browsing system and method for oversized three-dimensional model |
| CN117437342A (en) * | 2023-12-20 | 2024-01-23 | 深圳图为技术有限公司 | Three-dimensional scene rendering method, device and equipment |
| CN117437342B (en) * | 2023-12-20 | 2024-04-30 | 深圳图为技术有限公司 | Three-dimensional scene rendering method and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114513520B (en) | 2023-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114513520B (en) | Web three-dimensional visualization method based on synchronous rendering of client and server | |
| US11087553B2 (en) | Interactive mixed reality platform utilizing geotagged social media | |
| CN110070613B (en) | Large three-dimensional scene webpage display method based on model compression and asynchronous loading | |
| US9240070B2 (en) | Methods and systems for viewing dynamic high-resolution 3D imagery over a network | |
| CA2848215C (en) | Geospatial multiviewer | |
| US20100045662A1 (en) | Method and system for delivering and interactively displaying three-dimensional graphics | |
| CN111770325A (en) | Three-dimensional GIS real-time cloud rendering display method, terminal, cloud server and storage medium | |
| CN114531553B (en) | Method, device, electronic equipment and storage medium for generating special effect video | |
| EP4262217A1 (en) | System and method enabling private to public media experiences | |
| EP4254943A1 (en) | Head-tracking based media selection for video communications in virtual environments | |
| Zhu et al. | Towards peer-assisted rendering in networked virtual environments | |
| Noguera et al. | A scalable architecture for 3D map navigation on mobile devices | |
| CN115802076A (en) | Three-dimensional model distributed cloud rendering method and system and electronic equipment | |
| Maamar et al. | Streaming 3D meshes over thin mobile devices | |
| JP2023518676A (en) | Placement of immersive media and delivery from immersive media to heterogeneous client endpoints | |
| Park et al. | InstantXR: Instant XR environment on the web using hybrid rendering of cloud-based NeRF with 3d assets | |
| CN116467318A (en) | Media content processing method, device and storage medium based on cloud rendering | |
| CN116366827B (en) | High-precision large-scene image processing and transmitting method and device facing web end | |
| Liu et al. | Web‐Cloud Collaborative Mobile Online 3D Rendering System | |
| Stirbu et al. | Experiences building a multi-display mobile application for exploring mirror worlds | |
| Seligmann | SmmmmS lient for Remote Rendered irtual Heality | |
| Junghare et al. | Technical analysis of remote 3D visualization on mobile devices | |
| Zhang et al. | Transmission Method of the Rendered Image based on Cube Environment Mapping | |
| WO2023129162A1 (en) | Real-time lightweight video tracking, processing, and rendering | |
| CN114417204A (en) | Information generation method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 313200 building 6, No. 11, Keyuan Road, Wuyang street, Deqing County, Huzhou City, Zhejiang Province Patentee after: Zhejiang Zhongce Spacetime Technology Co.,Ltd. Address before: 313200 building 6, No. 11, Keyuan Road, Wuyang street, Deqing County, Huzhou City, Zhejiang Province Patentee before: ZHEJIANG TOPRS GEOGRAPHIC INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Web 3D Visualization Method Based on Client Server Synchronous Rendering Granted publication date: 20230630 Pledgee: Zhejiang Deqing rural commercial bank Limited by Share Ltd. Pledgor: Zhejiang Zhongce Spacetime Technology Co.,Ltd. Registration number: Y2024980016347 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |