CN113888704A - Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method - Google Patents
Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method Download PDFInfo
- Publication number
- CN113888704A CN113888704A CN202111450580.0A CN202111450580A CN113888704A CN 113888704 A CN113888704 A CN 113888704A CN 202111450580 A CN202111450580 A CN 202111450580A CN 113888704 A CN113888704 A CN 113888704A
- Authority
- CN
- China
- Prior art keywords
- rendering
- space
- training
- urban
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
Abstract
The invention discloses a low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method, which comprises the following steps: 1. dividing a multi-dimensional space of the urban training activities into a physical space, an entity space and an application space; 2. designing a rendering framework; 3. establishing a relation dependency between a channel node and a resource node, and rendering a channel to read the resource as input and write data into the resource; 4. and defining the execution sequence of the rendering channels, and performing resource scheduling and rendering optimization on the global angle through a rendering frame. The invention realizes the refined display of micro-scenes in urban training activities, provides various visual effects based on microscopic target characteristics for training commanders, can provide various training visual effects based on the microscopic target characteristics for the training commanders in the small-scale training action process of the urban training activities, quickly focuses on the battlefield environment in a specific area, accurately grasps the training scene conditions, quickly makes a decision and implements action control.
Description
Technical Field
The invention belongs to the field of urban training information systems, and particularly relates to a low-delay interaction-oriented micro scene grading time-sharing drawing optimization method.
Background
In urban training activities, the real-time sensing abilities of training commanders and training members on water pipes, power grids and the like in urban ground buildings, especially underground facilities are poor, the sensing advantages on urban environments are difficult to quickly form, and the survival abilities of the training members are seriously threatened. The urban training environment twin reconstruction is realized by breaking through the urban training activity multi-dimensional spatial information structure model construction and display technology, the urban labyrinth fog can be set, various training visual effects based on microscopic target characteristics are provided for training commanders in the urban training activity complex environment, and particularly, the urban training visual effects have the fine display capability on urban detail elements such as ground buildings, underground tunnels, hydrological weather, road networks, power networks, air networks and pipelines, the real-time interaction requirements of the training commanders and urban scenes are met, and the sense of reality and the fine degree of the virtual battlefield environment are further improved.
The real-time display of the multi-dimensional spatial information of the urban training activities needs to be based on the efficient drawing of complex and fine urban micro-scenes, the optimization of scene drawing efficiency mainly takes LOD preprocessing and efficient shielding and removing as main ideas at present, but both the technologies cannot solve the problem of low similarity among massive scene objects, and further material combination processing is difficult to perform, so that the performance bottleneck generated by the GPU due to frequent change of drawing states is caused, and the performance bottleneck cannot be eliminated through a local optimization method. The concept connotation of the multi-dimensional spatial information structure of the urban training activities needs to be determined from the application angle of the urban training activities, the hierarchical time division of the structure connotation, the element composition and the field division is carried out from the perspective of unified concept aiming at the multi-dimensional spatial information of the urban training activities with various elements, and a new engine frame is designed from the global aspect, so that the resource scheduling and rendering optimization from the training application angle are realized in rendering.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide a micro scene grading time-sharing drawing optimization method for low-delay interaction, which realizes the refined display of micro scenes in urban training activities and provides various visual effects based on microscopic target characteristics for training commanders.
The technical scheme is as follows: the invention relates to a low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method, which comprises the following steps of:
step 1: dividing a multi-dimensional space structure of the urban training activities, constructing a real-time graph drawing engine frame, and dividing the multi-dimensional space structure of the urban training activities into a physical space, an entity space and an application space, wherein a Z-dimensional (vertical), XY-dimensional (plane) and T-dimensional (time-dimensional) resource library is established on the physical space and a resource view is formed; designing an analysis expression frame of the urban space target to be displayed on the physical space; designing a thematic situation map layer from the view of a commander on an application space;
step 2: designing a rendering framework: adding a middle layer between a basic rendering module at the bottom layer of the frame and a special effect and terrain module at the upper layer, abstracting a channel concept in a graphic API, combining pipelines with the same function and resource input and output into an abstract channel node, and abstracting content pipeline resources requested by a rendering engine to the graphic environment and resources allocated for realizing a rendering algorithm into resource nodes;
and step 3: establishing a relation dependency between a channel node and a resource node, and rendering a channel to read the resource as input and write data into the resource;
and 4, step 4: defining an execution sequence of a rendering channel, performing resource scheduling and rendering optimization on a global angle through a rendering frame, and rendering and displaying the urban training activity micro-scene on a plane dimension, a vertical dimension and a time dimension according to a physical space; rendering and displaying the urban space target according to the entity space; and displaying and training the application thematic situation according to the rendering of the application space.
The physical space, i.e., the "3 +1 dimensional space-time" space (three spatial dimensions plus one time dimension), can be defined as: TS = (X, Y, Z, T). The physical space describes the position distribution and time attributes of a space entity and can be divided into a vertical dimension (Z dimension), a planar dimension (X, Y dimension), and a time dimension (T dimension).
The entity space refers to objective entities, namely empty entities, existing in the physical space of the city training activities. The space-time entity helps the training members to know the state and the relationship of each entity in the current stage of the urban environment where the training members are located and predict the later development of the urban environment.
The application space refers to a space which runs through application dimensions such as a physical domain, a social domain and a cognitive domain and is constructed by flexibly analyzing and establishing the association of the entity information of the multidimensional city training activity space in the dimensions such as space, time, attributes, cause and effect, cognition and the like aiming at a specific training task while a training director grasps the entity information of the universe in real time in the practice of the actual city training activity.
Further, in step 1 and step 4, the plane dimension includes layout forms of hubs, meshes, linearity, segments and the like of the city; the vertical dimension comprises the overground, the earth surface and the underground space of a city; the time dimension includes history, current and future.
Further, in step 1 and step 4, the urban space target includes urban climate, electromagnetic environment, urban traffic, urban power grid, urban water grid, urban gas grid, training facilities, underground tunnel.
Further, in step 1 and step 4, the thematic situation comprises a single building, a life net, an electromagnetic, a maneuvering corridor, an approach road, a traffic line, a communication area and a communication area.
Further, in step 2, the abstract channel nodes include a channel labeled as Opaque geomtypass, which is a rendering channel for all Opaque mesh writing to the geometric cache.
Further, in step 2, the abstract channel node includes a channel marked as Lighting Pass, which is a rendering channel for performing Lighting calculation on each segment of the geometric data in the geometric cache.
Further, in step 4, the execution sequence of the rendering channels is:
1) and (3) a geometric processing stage: rendering the geometric information of the target object to be rendered, and storing the rendered geometric information in a geometric cache, wherein the mainly stored geometric information comprises a position vector, a normal vector, a color vector and the like.
2) And (3) an illumination treatment stage: and carrying out illumination calculation on each segment of the geometric data in the geometric cache in an iterative mode on each pixel.
Has the advantages that: compared with the prior art, the invention has the following remarkable advantages: the invention realizes the refined display of micro-scenes in urban training activities, provides various visual effects based on micro-target characteristics for training commanders, in particular to the fine display of city detail elements such as ground buildings, facilities, underground tunnels, hydrological weather, road networks, power grids, air networks, pipelines and the like, meets the real-time interaction requirements of training commanders and training scenes, further improves the sense of realism and the fine degree of a virtual battlefield environment, carries out hierarchical time division on a multi-dimensional space structure of city training activities from the perspective of training application, and provides a new rendering architecture design, a new middle layer is added between the basic rendering module at the bottom layer and the modules of special effect, terrain and the like at the upper layer, when a scene is rendered, the rendering engine can acquire an abstract rendering process of the whole scene by using the rendering frame diagram, so that resource scheduling and rendering optimization are performed in a global angle. In the small-scale training action process of urban training activities, various training visual effects based on microscopic target characteristics can be provided for training commanders, the training visual effects can be quickly focused in a battlefield environment in a specific area, training scene conditions can be accurately mastered, and quick decision-making and action control can be implemented.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a physical space diagram of a multi-dimensional space structure of the city training activities of the present invention;
FIG. 3 is a physical space diagram of a multi-dimensional space structure of a city training activity of the present invention;
FIG. 4 is a diagram of the application space of the multi-dimensional space structure of the city training activities of the present invention;
FIG. 5 a single building envelope renders a display view;
FIG. 6 rendering a display view of a subsurface space;
FIG. 7 is a city local waterpipe rendering display diagram.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
As shown in fig. 1, a flow chart of the method of the present invention is constructed to construct a real-time graph drawing engine framework, the framework divides a city training multidimensional space structure into three levels, namely a physical space, an entity space and an application space, the three types of spaces are not stacked layer by layer, but explain the connotation of the city training multidimensional space structure from different perspectives, wherein the physical space provides physical space-time constraint for the application space, and the entity space constitutes an entity composition of the application space. The novel real-time graph drawing engine framework is used for drawing the training physical space, the training entity elements and the training application elements in a layered and graded mode, so that the rendering and drawing efficiency of the urban training micro scene is greatly improved, and the drawing time delay is reduced.
As shown in fig. 2, the physical space diagram of the multi-dimensional space structure of the city training activities is shown, and the city training multi-dimensional space can be divided into layout forms such as pivot, mesh, linear, segment, etc. in the plane dimension; dividing the space into overground space, surface space and underground space of a city in the vertical dimension; the time dimension includes history, current and future.
FIG. 3 is a diagram of an entity space of a multidimensional space structure of urban training activities, the entity training space is divided into normal entity spaces, and training forces are superimposed to form a training entity space when training activities are performed; the training entity space mainly comprises urban climate, electromagnetic environment, urban traffic, urban power grid, urban water network, urban gas network, training facilities, underground tunnels and the like.
Fig. 4 shows an application space diagram of the multidimensional space structure of the urban training activities according to the present invention, which is implemented by building different thematic map layers and performing application space rendering according to training applications such as a single building, a living network, an electromagnetic, a mobile corridor, an approach road, a traffic line, a communication area, and a communication area.
As shown in fig. 5, a display diagram is rendered for a single building shell, all the passages and entrances of a single building can be displayed by rotating and roaming the single building, and a topological map is provided for path planning in the building.
As shown in fig. 6, the underground space is a rendering display diagram, and in the urban training activities, the underground space can be an important channel for penetration assault and an important support for strong defense. Through roaming, rotating to underground space, watch underground pipeline, underground parking garage, underground engineering facility etc..
If figure 7 is a local water pipe in city and renders up the show picture, the city water pipe belongs to city life net part, city life net includes power grid, gas net, supplies/drainage network, heating power net, wired communication network etc. supports city wisdom body existence and operation, through the rendering up and show to city life net, can fix a position important position such as power station, sluice, data server fast, masters UNICOM's relation and relevant control hub, realizes that key node is transparent, logic relation is transparent and control core is transparent.
Claims (7)
1. A low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method is characterized by comprising the following steps:
step 1: dividing a multi-dimensional space structure of the urban training activities, constructing a real-time graph drawing engine frame, and dividing the multi-dimensional space structure of the urban training activities into a physical space, an entity space and an application space, wherein a Z-dimensional, XY-dimensional and T-dimensional resource library is established and a resource view is formed on the physical space, wherein the Z-dimensional is a vertical dimension, the XY-dimensional is a plane dimension, and the T-dimensional is a time dimension; designing an analysis expression frame of the urban space target to be displayed on the physical space; designing a thematic situation map layer from the view of a commander on an application space;
step 2: designing a rendering framework: adding a middle layer between a basic rendering module at the bottom layer of the frame and a special effect and terrain module at the upper layer, abstracting a channel concept in a graphic API, combining pipelines with the same function and resource input and output into an abstract channel node, and abstracting content pipeline resources requested by a rendering engine to the graphic environment and resources allocated for realizing a rendering algorithm into resource nodes;
and step 3: establishing a relation dependency between a channel node and a resource node, and rendering a channel to read the resource as input and write data into the resource;
and 4, step 4: defining an execution sequence of a rendering channel, performing resource scheduling and rendering optimization on a global angle through a rendering frame, and rendering and displaying the urban training activity micro-scene on a plane dimension, a vertical dimension and a time dimension according to a physical space; rendering and displaying the urban space target according to the entity space; and displaying and training the application thematic situation according to the rendering of the application space.
2. The low-latency interaction-oriented micro-scene hierarchical time-sharing rendering optimization method of claim 1, wherein in the step 1 and the step 4, the plane dimension includes a hub, mesh, linear and segment layout form of a city; the vertical dimension comprises the overground, the earth surface and the underground space of a city; the time dimension includes history, current and future.
3. The low-latency interaction-oriented micro-scene hierarchical time-sharing drawing optimization method according to claim 1, wherein in the step 1 and the step 4, the urban space target comprises urban climate, electromagnetic environment, urban traffic, urban power grid, urban water grid, urban gas grid, training facilities and underground tunnels.
4. The low-latency interaction-oriented micro-scene hierarchical time-sharing rendering optimization method according to claim 1, wherein in the step 1 and the step 4, the thematic situation comprises a single building, a life net, electromagnetism, a maneuvering corridor, an approach road, a traffic line, a communication area and a communication area.
5. The low-latency interaction-oriented micro-scene hierarchical time-sharing rendering optimization method according to claim 1, wherein in the step 2, the abstract channel nodes include a channel marked as Opaque geoentrypass, which is a rendering channel for all Opaque grids written into a geometric cache.
6. The low-latency interaction-oriented micro-scene hierarchical time-sharing rendering optimization method according to claim 1, wherein in the step 2, the abstract channel node includes a channel marked as Lighting Pass, which is a rendering channel for performing illumination calculation on each segment of the geometric data in the geometric cache.
7. The low-latency interaction-oriented micro-scene hierarchical time-sharing rendering optimization method according to claim 1, wherein in the step 4, the execution sequence of the rendering channels is as follows:
1) and (3) a geometric processing stage: rendering geometric information of a target object to be rendered, and storing the rendered geometric information in a geometric cache, wherein the mainly stored geometric information comprises a position vector, a normal vector and a color vector;
2) and (3) an illumination treatment stage: and carrying out illumination calculation on each segment of the geometric data in the geometric cache in an iterative mode on each pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111450580.0A CN113888704B (en) | 2021-12-01 | 2021-12-01 | Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111450580.0A CN113888704B (en) | 2021-12-01 | 2021-12-01 | Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113888704A true CN113888704A (en) | 2022-01-04 |
| CN113888704B CN113888704B (en) | 2022-03-04 |
Family
ID=79016184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111450580.0A Active CN113888704B (en) | 2021-12-01 | 2021-12-01 | Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113888704B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463160A (en) * | 2022-01-30 | 2022-05-10 | 摩尔线程智能科技(北京)有限责任公司 | Parallel processing method and device for graphics pipeline and readable storage medium |
| CN115017138A (en) * | 2022-06-30 | 2022-09-06 | 北京亚控科技发展有限公司 | Personnel plan state digital twin method, device and equipment |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090153556A1 (en) * | 2007-12-17 | 2009-06-18 | Electronics And Telecommunications Research Institute | Ray tracing device based on a pixel processing element and method thereof |
| CN101702244A (en) * | 2009-11-10 | 2010-05-05 | 南京大学 | Parallel drawing load balancing method for free division of drawing area |
| US20130016109A1 (en) * | 2011-07-15 | 2013-01-17 | Kirill Garanzha | Out-of-core ray tracing with memory-efficient page generation |
| US20130328857A1 (en) * | 2012-06-11 | 2013-12-12 | Disney Enterprises, Inc. (Burbank, Ca) | Streaming Hierarchy Traversal Renderer |
| CN106952330A (en) * | 2017-03-20 | 2017-07-14 | 南京师范大学 | A kind of mobile terminal three-dimensional city dynamic modelling method based on geographical semantics |
| CN110163964A (en) * | 2019-04-23 | 2019-08-23 | 中国电子科技集团公司第二十八研究所 | Physical simulation system and its application method based on three-dimensional digital earth |
| CN110532666A (en) * | 2019-08-26 | 2019-12-03 | 中国石油大学(华东) | Urban micro-climate evolution analysis method based on wind environment variation |
| CN111815787A (en) * | 2020-07-13 | 2020-10-23 | 北京优锘科技有限公司 | Three-dimensional digital plan making system and method for petrochemical enterprises |
| CN112001993A (en) * | 2020-07-14 | 2020-11-27 | 深圳市规划国土房产信息中心(深圳市空间地理信息中心) | Multi-GPU (graphics processing Unit) city simulation system for large scene |
| CN112184864A (en) * | 2020-09-08 | 2021-01-05 | 中国电子科技集团公司第二十八研究所 | Real-time drawing method of million-magnitude three-dimensional situation target |
| CN112270756A (en) * | 2020-11-24 | 2021-01-26 | 山东汇颐信息技术有限公司 | Data rendering method applied to BIM model file |
-
2021
- 2021-12-01 CN CN202111450580.0A patent/CN113888704B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090153556A1 (en) * | 2007-12-17 | 2009-06-18 | Electronics And Telecommunications Research Institute | Ray tracing device based on a pixel processing element and method thereof |
| CN101702244A (en) * | 2009-11-10 | 2010-05-05 | 南京大学 | Parallel drawing load balancing method for free division of drawing area |
| US20130016109A1 (en) * | 2011-07-15 | 2013-01-17 | Kirill Garanzha | Out-of-core ray tracing with memory-efficient page generation |
| US20130328857A1 (en) * | 2012-06-11 | 2013-12-12 | Disney Enterprises, Inc. (Burbank, Ca) | Streaming Hierarchy Traversal Renderer |
| CN106952330A (en) * | 2017-03-20 | 2017-07-14 | 南京师范大学 | A kind of mobile terminal three-dimensional city dynamic modelling method based on geographical semantics |
| CN110163964A (en) * | 2019-04-23 | 2019-08-23 | 中国电子科技集团公司第二十八研究所 | Physical simulation system and its application method based on three-dimensional digital earth |
| CN110532666A (en) * | 2019-08-26 | 2019-12-03 | 中国石油大学(华东) | Urban micro-climate evolution analysis method based on wind environment variation |
| CN111815787A (en) * | 2020-07-13 | 2020-10-23 | 北京优锘科技有限公司 | Three-dimensional digital plan making system and method for petrochemical enterprises |
| CN112001993A (en) * | 2020-07-14 | 2020-11-27 | 深圳市规划国土房产信息中心(深圳市空间地理信息中心) | Multi-GPU (graphics processing Unit) city simulation system for large scene |
| CN112184864A (en) * | 2020-09-08 | 2021-01-05 | 中国电子科技集团公司第二十八研究所 | Real-time drawing method of million-magnitude three-dimensional situation target |
| CN112270756A (en) * | 2020-11-24 | 2021-01-26 | 山东汇颐信息技术有限公司 | Data rendering method applied to BIM model file |
Non-Patent Citations (4)
| Title |
|---|
| T. TSUJI 等: "Interactive rendering with LOD control and occlusion culling based on polygon hierarchies", 《PROCEEDINGS COMPUTER GRAPHICS INTERNATIONAL, 2004.》 * |
| 冯磊 等: "大规模3DGIS场景的多通道渲染算法研究", 《高技术通讯》 * |
| 智明 等: "HLODs模型在场景快速绘制中的应用", 《计算机应用》 * |
| 陈刘忠 等: "云平台容灾技术研究与实现", 《网络安全技术与应用》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463160A (en) * | 2022-01-30 | 2022-05-10 | 摩尔线程智能科技(北京)有限责任公司 | Parallel processing method and device for graphics pipeline and readable storage medium |
| CN114463160B (en) * | 2022-01-30 | 2022-10-28 | 摩尔线程智能科技(北京)有限责任公司 | Parallel processing method and device for graphics pipeline and readable storage medium |
| CN115017138A (en) * | 2022-06-30 | 2022-09-06 | 北京亚控科技发展有限公司 | Personnel plan state digital twin method, device and equipment |
| CN115017138B (en) * | 2022-06-30 | 2024-04-02 | 北京亚控科技发展有限公司 | Personnel planning state digital twin method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113888704B (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113888704B (en) | Low-delay interaction-oriented micro scene hierarchical time-sharing drawing optimization method | |
| Wu et al. | Generative Adversarial Networks in the built environment: A comprehensive review of the application of GANs across data types and scales | |
| Brown et al. | Spatial process and data models: Toward integration of agent-based models and GIS | |
| CN106599332A (en) | Three-dimensional digital scheme aided design and display method | |
| Chen | The development of 3D city model and its applications in urban planning | |
| Luo et al. | Analysis of urban ventilation potential using rule-based modeling | |
| CN115272637B (en) | Large-area-oriented three-dimensional virtual ecological environment visual integration and optimization system | |
| CN112528508A (en) | Electromagnetic visualization method and device | |
| CN105335478A (en) | Method and device constructing city land space stereo investigation data semantic association | |
| Guo et al. | Large-scale and refined green space identification-based sustainable urban renewal mode assessment | |
| CN111125291A (en) | WebGIS engine design method and system for watershed water resource management decision support system | |
| Zhang et al. | Implementation methods and applications of flow visualization in a watershed simulation platform | |
| Hua et al. | Geographic information systems | |
| Jiang et al. | Urban Development Boundary Simulation Based on" Double Evaluation" and FLUS Model. | |
| CN110457410A (en) | A kind of construction method and system of three-dimensional GIS platform | |
| Leng et al. | A data integration and simplification framework for improving site planning and building design | |
| Wan et al. | Virtual geographical environment | |
| Yang | Energy resilient urban planning | |
| Chen et al. | Data-driven Reconstruction for Massive Buildings within Urban Scenarios: A Case Study | |
| Jackson et al. | Digital City: an urban perspective on digital earth | |
| Zhang et al. | Comprehensive framework for the integration and analysis of geo-environmental data for urban geohazards | |
| CN117523063B (en) | Rendering method of high-performance ray tracing simulation based on three-dimensional space | |
| Zhang et al. | Analysis of human geography space based on geographic information system in the context of rural revitalization | |
| Jirwankar et al. | Sustainable Smart City Planning By Means Of Land Use Models For Indian Cities | |
| CN117422839A (en) | GIS system based on multidimensional space geographic information big data and GIS system service method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |