CN109508090B - Augmented reality panel system with interchangeability - Google Patents
Augmented reality panel system with interchangeability Download PDFInfo
- Publication number
- CN109508090B CN109508090B CN201811311765.1A CN201811311765A CN109508090B CN 109508090 B CN109508090 B CN 109508090B CN 201811311765 A CN201811311765 A CN 201811311765A CN 109508090 B CN109508090 B CN 109508090B
- Authority
- CN
- China
- Prior art keywords
- subsystem
- augmented reality
- model
- virtual scene
- display board
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses an interactive augmented reality display board system, which comprises mobile intelligent equipment, an augmented reality subsystem and a display board device, wherein the mobile intelligent equipment is connected with the augmented reality subsystem through a network; the mobile intelligent device comprises an intelligent mobile phone or an intelligent tablet serving as an input end and a processor of the system, and is used for scanning an actual image and carrying out image recognition; the augmented reality subsystem is used for importing a model of an object or a scene to be displayed into virtual simulation software; the augmented reality subsystem comprises a virtual scene subsystem and a man-machine interaction subsystem; the exhibition board device comprises an identification picture for triggering the virtual scene subsystem, theme content to be displayed and a download special link of the augmented reality subsystem in an embedded manner; the download special link is positioned at the lower left corner of the display board; the identification picture is positioned at the lower right corner of the display board. The system can complete the omnibearing display of the displayed object without a mobile network or WiFi, can enhance the interactivity and artistry of the display, has low price and is convenient to popularize.
Description
Technical Field
The invention belongs to the field of exhibition display and augmented reality, and particularly relates to an augmented reality display board system with interchangeability.
Background
Augmented Reality (AR) technology is a technology that can integrate a real scene and a virtual scene "seamlessly". In recent years, the government and the scientific community in China pay high attention, and the augmented reality technology is widely applied to the fields of real estate, education, media publishing and the like. However, the number of augmented reality products facing the exhibition field in the market is not large. Most of the display boards used in the existing conferences, exhibitions, expositions, trade fairs, exhibition and exhibition are spliced or welded by using portable equipment. Such as the existing folding exhibition board, the portable exhibition board, the expandable exhibition board with a turnover device or a slide rail, etc. Chinese patent publication No. CN201410672551.2 discloses a magnetic adhesive tape display board, which is made of small magnetic steel plates, and is rolled in a rotating box, and the length of the display board can be freely adjusted, so that the display board is convenient to install. The disadvantages of the method are too monotonous and lack of interest and interactivity. In order to increase the interest of the display board, a display board with an interactive function is also invented, for example, chinese patent CN201310039639.6 discloses a fingerprint display board, which includes a touch display board, a fingerprint scanner and a microprocessor, and can use fingerprints to create a personal account for storing information required by viewers. Chinese patent 201510081904.6 discloses an interactive propaganda show board based on multiple choice answer mode, carries out the interactive "developments" of many sense organs through the multiple choice answer mode based on many knowledge points to show the design for artificial center carries out, can arouse during audience's initiative participated in the interaction of show content, arouses audience to the interest of show content, promotes propaganda show effect. The invention has the disadvantages that the fingerprint touch plate is not suitable for being popularized in public places and needs to be maintained in time; the answer type display board cannot be applied to non-answer type application occasions.
In order to improve the propaganda and display effect of the display board, the interactive experience is increased, and the interest of audiences in the displayed content is stimulated. Today, smart devices such as smart phones and tablet computers are widely used, a new interactive display board needs to be developed urgently.
The exhibition system based on the augmented reality is established by the invention, a vivid virtual environment is constructed by establishing a 3D scene for an object to be exhibited in reality, the virtual environment is superposed on a real object, and the object can be visited and browsed by scanning the real object. The AR display board increases interaction functions through C # language programming, can play a role in expanding and supplementing real objects, and enhances interactivity and artistry. The research of the technology is also gradually becoming a trend of the current society.
Disclosure of Invention
The invention overcomes the defects in the prior art and provides an augmented reality display board system with interactivity.
In order to solve the technical problems, the invention is realized by the following technical scheme:
an interactive augmented reality display board system comprises a mobile intelligent device, an augmented reality subsystem and a display board device;
the mobile intelligent device comprises an intelligent mobile phone or an intelligent tablet serving as an input end and a processor of the system, and is used for scanning an actual image and carrying out image recognition;
the augmented reality subsystem is used for importing a model of an object or scene to be displayed into virtual simulation software, associating video, audio and text information related to the model, setting gesture interaction and voice interaction modes of a three-dimensional model to trigger information interesting for a user, compiling the information into App to be installed on a smart phone or a smart tablet, and completing recognition and enhancement operations of a recognition graph without a cellular mobile network or WiFi;
the augmented reality subsystem comprises a virtual scene subsystem and a man-machine interaction subsystem;
the virtual scene subsystem comprises a virtual model established by taking an actual displayed object as a reference, a database corresponding to the displayed object and data attribute information;
the human-computer interaction subsystem is used for identifying the problems input by actions and voices of exhibitors through touching a screen or sliding the screen or clicking the screen, converting the identification result into an execution command and completing multi-angle display and problem solution of the virtual scene subsystem; as a spectator of the mobile terminal, the gesture change and the voice question can also be converted into an execution command, so that the change of the virtual scene and the information display are caused, and the interaction with the scene is realized; the virtual scene can be a virtual model or a series of virtual models;
the exhibition board device comprises an identification picture for triggering the virtual scene subsystem, theme content to be displayed and a download special link of the augmented reality subsystem in an embedded manner; the download special link is positioned at the lower left corner of the display board; the identification picture is positioned at the lower right corner of the display board;
the system can realize the combination of the traditional propaganda mode and the augmented reality technology through a method of installing an App and scanning an identification picture by a smart phone or a smart tablet.
Due to the adoption of the technical scheme, compared with the prior art, the augmented reality display board system provided by the invention has the following beneficial effects:
(1) the on-site experience is improved. The invention can not only complete the introduction of the displayed object, enhance the interactivity and artistry of the exhibition, but also enhance the display effect, so that the user can know the displayed content in a multi-angle and all-round way;
(2) low cost, and wide application. The display board device of the invention is not much different from the common display board, and only embeds the recognition graph of the link augmented reality subsystem, thereby having low cost. The intelligent mobile equipment is popularized at present, and the user cost is not increased, so that the intelligent mobile equipment is suitable for large-scale popularization;
(3) improving the reality of the displayed content. The invention provides the display for the user by adopting the three-dimensional virtual scene, the gesture interaction and the character and sound interaction, and has strong sense of reality.
The system can complete the omnibearing display of the displayed object without a mobile network or WiFi, can enhance the interactivity and artistry of the display, has low price and is convenient to popularize.
Drawings
FIG. 1 is an overall block diagram of the system of the present invention;
FIG. 2 is a schematic view of the display board assembly;
FIG. 3 is an augmented reality subsystem interface schematic;
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples.
An interactive augmented reality exhibition board system of the present invention, as shown in fig. 1, comprises a mobile smart device, an exhibition board device and an augmented reality subsystem;
the mobile intelligent device adopts a smart phone with a camera sold on the market or an intelligent tablet with a camera, and the embodiment adopts an android smart phone for scanning the actual content of the display board and identifying the image.
The exhibition board device comprises an identification picture for triggering the virtual scene subsystem, theme content to be displayed and a download special link of the augmented reality subsystem in an embedded manner; the download special link is positioned at the lower left corner of the display board; the identification picture is positioned at the lower right corner of the display board. The composition of the display board device in this embodiment is shown in fig. 2, and the brief description of the displayed object is an introduction to the displayed object in a conventional display device, including an introduction of pictures and characters; the "target recognition graph" is shown as a two-dimensional code symbol identifying the object at the present exhibition. The link of the augmented reality subsystem is placed at the lower left corner of the display board, and the target recognition graph placement position is the lower right corner.
The augmented reality subsystem is used for leading a model of an object or a scene to be displayed into virtual simulation software, associating video, audio and text information related to the model, setting gesture interaction and voice interaction modes of a three-dimensional model to trigger information interesting for a user, compiling the information into App to be installed in an android smart phone, and completing recognition and enhancement operation of a recognition graph under the condition of no need of a cellular mobile network or WiFi.
The augmented reality subsystem comprises a virtual scene subsystem and a man-machine interaction subsystem.
The virtual scene subsystem comprises various 3D models established by taking an actual exhibited object as reference and processing of three-dimensional virtual scene data corresponding to the exhibited object; the three-dimensional virtual scene is modeled through 3D MAX software or a three-dimensional laser scanner, and various types of data such as introductory videos, characters, pictures, audios and the like related to the model are automatically linked in the 3D model so as to enrich the content of the 3D model. The three-dimensional virtual scene is mainly three-dimensional enhanced display of a real scene, model creation needs to be completed in a display board manufacturing process, and data association and action setting of the model are completed through 3D simulation software (Unity 3D is adopted in the embodiment). The three-dimensional virtual scene is not single, and can be composed of a plurality of virtual scenes or three-dimensional objects, and the scenes or the three-dimensional objects have content consistency with the text and picture introduction in the board viewing device. The man-machine interaction subsystem is used for analyzing and identifying actions of a touch screen, a sliding screen, a clicking screen and the like of an exhibitor and related voice problems related to displayed contents; and the recognition result is converted into an execution command, so that the multi-angle display and the problem solution of the virtual scene subsystem are completed. Gesture changes and voice questions of the mobile terminal can be converted into execution commands, so that changes of the virtual scene and information display are caused, and interaction with the scene is achieved. The virtual scene can be a virtual model or a series of virtual models;
the interface of the augmented reality subsystem is shown in fig. 3. In the figure: the gesture interaction is used for selecting a gesture interaction mode, in the mode, a user can freely rotate and zoom a virtual object superposed on a picture, and the displayed object or scene is observed in 360 degrees; "Voice interaction" is shown for selecting a voice interaction mode in which a user can ask a simple question; the '360-degree panoramic display' is used for observing a displayed object or scene in 360 degrees; the prompt is used for prompting the current recognition state, including prompting the prompt character of the current state, changing the virtual key of the current recognition state and recognizing the touch gesture; the "button" shown is a virtual key for changing the current recognition state and the "triangle symbol" shown is a touch recognition for pausing the playing of the multimedia/3D virtual scene.
The man-machine interaction subsystem is mainly completed through C # language programming, and the main interaction mode comprises the following steps:
1. and gesture interaction for completing rotation, translation and scaling of the model. The operation is completed by an operator through a touch screen or gesture operation, and the operation process is the same as the interactive operation mode of the existing mobile phone;
2. selecting a model: an operator selects a model to be displayed through a touch screen, and a highlight frame is displayed after the model is selected;
3. voice interaction: after the model is selected, the operator selects the voice interaction mode and asks questions about the model, such as "what is the property of the displayed item? "the building age of the building", etc., the voice recognition API adopted by the functional module, after the system successfully recognizes, the text, voice and video introduction corresponding to the model can be automatically called out.
In the embodiment, the Unity3 d-based augmented reality subsystem is developed in the C # language in the design and production stage of the display board. The project engineering files are packaged into APK files, the corresponding model files are stored in FBX format resource packages, the corresponding video files are in MP4 format, and the corresponding picture format is in JPEG format. The APK file is installed at the end of the smart phone, and the identification and enhancement operation of the identification map can be completed under the condition that a cellular mobile network or WiFi is not needed. In the using process of a user, the gesture change and the voice question can be converted into an execution command, so that the change of the virtual scene of the augmented reality subsystem and the information display can be caused, and the interaction with the scene is realized.
Claims (1)
1. An interactive augmented reality exhibition board system, comprising: the system comprises mobile intelligent equipment, an augmented reality subsystem and a display board device;
the mobile intelligent device comprises an intelligent mobile phone or an intelligent tablet serving as an input end and a processor of the system, and is used for scanning an actual image and carrying out image recognition;
the augmented reality subsystem is used for importing a model of an object or scene to be displayed into virtual simulation software, associating video, audio and text information related to the model, setting gesture interaction and voice interaction modes of a three-dimensional model to trigger information interesting for a user, compiling the information into App to be installed on a smart phone or a smart tablet, and completing recognition and enhancement operations of a recognition graph without a cellular mobile network or WiFi;
the augmented reality subsystem comprises a virtual scene subsystem and a man-machine interaction subsystem;
the virtual scene subsystem comprises various 3D models established by taking an actual exhibited object as reference and processing of three-dimensional virtual scene data corresponding to the exhibited object; the three-dimensional virtual scene is modeled through 3D MAX software or a three-dimensional laser scanner, and various types of data such as introductory videos, characters, pictures, audios and the like related to the model are automatically linked in the 3D model;
the human-computer interaction subsystem is used for identifying the problems input by actions and voices of exhibitors through touching a screen or sliding the screen or clicking the screen, converting the identification result into an execution command and completing multi-angle display and problem solution of the virtual scene subsystem; as a spectator of the mobile terminal, the gesture change and the voice question can also be converted into an execution command, so that the change of the virtual scene and the information display are caused, and the interaction with the scene is realized; the virtual scene can be a virtual model or a series of virtual models;
the exhibition board device comprises an identification picture for triggering the virtual scene subsystem, theme content to be displayed and a download special link of the augmented reality subsystem in an embedded manner; the download special link is positioned at the lower left corner of the display board; the identification picture is positioned at the lower right corner of the display board;
the system can realize the combination of the traditional propaganda mode and the augmented reality technology by a method of installing an App and scanning an identification picture through a smart phone or a smart tablet;
the exhibition board device completes data association and action setting of the model through 3D simulation software, and the augmented reality subsystem is developed by adopting C # language; project engineering files of the system are packaged into APK files, corresponding model files are stored in FBX format resource packages, corresponding video files are in MP4 format, and corresponding picture formats are in JPEG format.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811311765.1A CN109508090B (en) | 2018-11-06 | 2018-11-06 | Augmented reality panel system with interchangeability |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811311765.1A CN109508090B (en) | 2018-11-06 | 2018-11-06 | Augmented reality panel system with interchangeability |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109508090A CN109508090A (en) | 2019-03-22 |
| CN109508090B true CN109508090B (en) | 2021-01-08 |
Family
ID=65747662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811311765.1A Active CN109508090B (en) | 2018-11-06 | 2018-11-06 | Augmented reality panel system with interchangeability |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109508090B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110047148B (en) * | 2019-04-10 | 2024-01-26 | 珠海惠中智能技术有限公司 | Simulation interaction visualization system of virtual robot workstation and implementation method |
| CN111949904B (en) * | 2019-05-16 | 2024-06-11 | 深圳市雅阅科技有限公司 | Data processing method and device based on browser and terminal |
| CN111679742A (en) * | 2020-06-10 | 2020-09-18 | 浙江商汤科技开发有限公司 | Interaction control method and device based on AR, electronic equipment and storage medium |
| CN112750167B (en) * | 2020-12-30 | 2022-11-04 | 燕山大学 | Robot vision positioning simulation method and device based on virtual reality |
| CN114185279A (en) * | 2021-12-30 | 2022-03-15 | 深圳思悦创新有限公司 | Intelligent household scene and hardware equipment display device and display method thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461006A (en) * | 2014-12-17 | 2015-03-25 | 卢晨华 | Internet intelligent mirror based on natural user interface |
| CN105929945A (en) * | 2016-04-18 | 2016-09-07 | 展视网（北京）科技有限公司 | Augmented reality interaction method and device, mobile terminal and mini-computer |
| CN106126014A (en) * | 2016-06-16 | 2016-11-16 | 北京小米移动软件有限公司 | The generation method and device of interactive interface |
| CN106485779A (en) * | 2016-03-22 | 2017-03-08 | 智合新天（北京）传媒广告股份有限公司 | A kind of 3D virtual interacting display platform and the method for showing 3D animation |
| CN107608649A (en) * | 2017-11-02 | 2018-01-19 | 泉州创景视迅数字科技有限公司 | A kind of AR augmented realities intelligent image identification displaying content system and application method |
| CN108022543A (en) * | 2017-11-27 | 2018-05-11 | 深圳中科呼图电子商务有限公司 | A kind of advertisement autonomous demenstration method, system and advertisement machine and application |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130293580A1 (en) * | 2012-05-01 | 2013-11-07 | Zambala Lllp | System and method for selecting targets in an augmented reality environment |
| US10228242B2 (en) * | 2013-07-12 | 2019-03-12 | Magic Leap, Inc. | Method and system for determining user input based on gesture |
-
2018
- 2018-11-06 CN CN201811311765.1A patent/CN109508090B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461006A (en) * | 2014-12-17 | 2015-03-25 | 卢晨华 | Internet intelligent mirror based on natural user interface |
| CN106485779A (en) * | 2016-03-22 | 2017-03-08 | 智合新天（北京）传媒广告股份有限公司 | A kind of 3D virtual interacting display platform and the method for showing 3D animation |
| CN105929945A (en) * | 2016-04-18 | 2016-09-07 | 展视网（北京）科技有限公司 | Augmented reality interaction method and device, mobile terminal and mini-computer |
| CN106126014A (en) * | 2016-06-16 | 2016-11-16 | 北京小米移动软件有限公司 | The generation method and device of interactive interface |
| CN107608649A (en) * | 2017-11-02 | 2018-01-19 | 泉州创景视迅数字科技有限公司 | A kind of AR augmented realities intelligent image identification displaying content system and application method |
| CN108022543A (en) * | 2017-11-27 | 2018-05-11 | 深圳中科呼图电子商务有限公司 | A kind of advertisement autonomous demenstration method, system and advertisement machine and application |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109508090A (en) | 2019-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109508090B (en) | Augmented reality panel system with interchangeability | |
| WO2022001593A1 (en) | Video generation method and apparatus, storage medium and computer device | |
| WO2020083021A1 (en) | Video recording method and apparatus, video playback method and apparatus, device, and storage medium | |
| US20130076788A1 (en) | Apparatus, method and software products for dynamic content management | |
| US20160041981A1 (en) | Enhanced cascaded object-related content provision system and method | |
| CN105306861A (en) | Online teaching recording and playing method and system | |
| WO2023030010A1 (en) | Interaction method, and electronic device and storage medium | |
| CN110930220A (en) | Display method, display device, terminal equipment and medium | |
| TW202141446A (en) | Multimedia interactive method, device and computer readable storage medium | |
| US11758217B2 (en) | Integrating overlaid digital content into displayed data via graphics processing circuitry | |
| KR101679791B1 (en) | Display opperation system using mobile apparatus and method thereof | |
| CN105760420B (en) | Realize the method and device with multimedia file content interaction | |
| CN111741321A (en) | Live broadcast control method, device, equipment and computer storage medium | |
| CN109788327A (en) | Multi-screen interaction method, device and electronic equipment | |
| CN112988008A (en) | Information display method and device, computer equipment and storage medium | |
| CN114846808A (en) | Content distribution system, content distribution method, and content distribution program | |
| CN104485052B (en) | A kind of product demonstration method and system based on mobile terminal | |
| US20110167346A1 (en) | Method and system for creating a multi-media output for presentation to and interaction with a live audience | |
| CN103543829A (en) | Intelligent 3D (three-dimensional) visual presenter capable of realizing projection touch | |
| CN114449355B (en) | Live interaction method, device, equipment and storage medium | |
| CN112929685B (en) | Interaction method and device for VR live broadcast room, electronic device and storage medium | |
| CN211506772U (en) | Virtual book page turning display device | |
| Zhu et al. | Design and application of project-based teaching of convergence media smart classroom based on VR+ AR technology | |
| US20210133220A1 (en) | Information processing apparatus and information processing method | |
| CN102999244A (en) | Realization method, realization system and manufacturing method for exhibition of electronic house type |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |