CN109508090A - A kind of augmented reality display board system having interactivity - Google Patents

A kind of augmented reality display board system having interactivity Download PDF

Info

Publication number
CN109508090A
CN109508090A CN201811311765.1A CN201811311765A CN109508090A CN 109508090 A CN109508090 A CN 109508090A CN 201811311765 A CN201811311765 A CN 201811311765A CN 109508090 A CN109508090 A CN 109508090A
Authority
CN
China
Prior art keywords
subsystem
augmented reality
identification
virtual scene
exhibition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811311765.1A
Other languages
Chinese (zh)
Other versions
CN109508090B (en
Inventor
丁伟利
刘鑫
曲春雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yanshan University
Original Assignee
Yanshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yanshan University filed Critical Yanshan University
Priority to CN201811311765.1A priority Critical patent/CN109508090B/en
Publication of CN109508090A publication Critical patent/CN109508090A/en
Application granted granted Critical
Publication of CN109508090B publication Critical patent/CN109508090B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text

Abstract

The invention discloses a kind of augmented reality display board systems of interactivity, including intelligent movable equipment, augmented reality subsystem and exhibition panel assembly;The intelligent movable equipment includes the input terminal and processor of smart phone or Intelligent flat as the system, for scanning real image, and carries out image recognition;The augmented reality subsystem is will to import virtual emulation software by the model of exhibiting objects or scene;The augmented reality subsystem includes virtual scene subsystem and human-machine interaction subsystem;The exhibition panel assembly includes the downloading private link of the identification figure of the embedded triggering virtual scene subsystem, the subject content and augmented reality subsystem to be shown;The downloading private link is located at the panel lower left corner;The identification figure is located at the panel lower right corner.Present system is not necessarily to mobile network or WiFi, can be completed to by the comprehensive displaying of exhibiting objects, it can enhances the interactivity and artistry of exhibition, and cheap, convenient for promoting.

Description

A kind of augmented reality display board system having interactivity
Technical field
The invention belongs to display and demonstration and augmented reality field more particularly to a kind of augmented reality exhibitions for having interactivity Plate system.
Background technique
Augmented reality (Augmented Reality, AR) technology is that one kind can be by real scene and virtual scene " nothing The integrated technology of seam ".In recent years, the Chinese government and scientific circles all pay close attention, augmented reality house property, education and The fields such as medium publication are widely used.However currently on the market towards the exhibition augmented reality product in field and few.It is existing Meeting, exhibition, fair, trade fair, fair and exhibition etc. used in most of panel be using light Equipment splices or is welded.Such as existing collapsible exhibition version, Portable panel and with turnover device or setting Sliding rail can expansion type panel etc..China Patent Publication No. CN201410672551.2 discloses a kind of Magnetic adhesive tape panel, Panel is made of magnetic small pieces steel plate, is rolled up in rotating box, be can freely adjust and is pulled out length, is easily installed.It is insufficient Place is excessively dull, shortage interest and interactivity.In order to increase the interest of panel, somebody, which has invented, has interaction function The panel of energy, if Chinese patent CN201310039639.6 discloses a kind of fingerprint panel, it includes touching panel, finger scan Device and microprocessor can use fingerprint and establish the information that personal account storage onlooker needs.Chinese patent 201510081904.6 disclose a kind of interaction publicity exhibition board based on multiselect answering mode, pass through the multiselect based on more knowledge points " dynamic " that answering mode carries out more sense organ interactions is shown, human-centred to be shown design, can be transferred spectators and actively be joined With show in the interacting of content, interest of the excitation spectators to open up content, promotion promotional display effect.The deficiency of foregoing invention The plate being in finger touch class should not be promoted in public, need timely to safeguard;Answering type panel may not apply to Non- answering type application.
For the promotional display effect for promoting panel, increase Interactive Experience, interest of the excitation spectators to opened up content.In intelligence It needs to develop a kind of new interaction panel today that the smart machines such as mobile phone, tablet computer generally use.
Exhibition system established by the present invention based on augmented reality is by establishing 3D by exhibiting objects in reality Scene constructs virtual environment true to nature, and is added on true object, can be carried out visiting by scanning true object Browsing.AR panel increases interactive function by C# language programming, can play extension and supplementary function to real world object, increases Strong interactivity and artistry.The research of the technology has been increasingly becoming a kind of trend of current social.
Summary of the invention
The present invention overcomes the disadvantages of the prior art, provides a kind of augmented reality display board system for having interactivity.
In order to solve above-mentioned technical problem, the present invention is achieved by the following technical solutions:
A kind of augmented reality display board system of interactivity, including intelligent movable equipment, augmented reality subsystem and exhibition Panel assembly;
The intelligent movable equipment includes the input terminal and processor of smart phone or Intelligent flat as the system, is used In scanning real image, and carry out image recognition;
The augmented reality subsystem is will to import virtual emulation software by the model of exhibiting objects or scene, and be associated with The gesture interaction and speech interaction mode of threedimensional model is arranged, to trigger in video relevant to model, audio, text information User interest information, and be compiled into App and be installed to smart phone or Intelligent flat, without cellular mobile network or WiFi In the case of can complete identification to identification figure and enhancing operation;
The augmented reality subsystem includes virtual scene subsystem and human-machine interaction subsystem;
The virtual scene subsystem includes the dummy model established with reality by spectacular for reference, by spectacular phase Corresponding database and data attribute information;
Exhibitor slides screen/by touching screen/either or clicks screen the human-machine interaction subsystem for identification The problem of curtain is to act with voice input, and convert recognition result to and execute order, it completes to the more of virtual scene subsystem Angle is shown and answer;As the exhibition visitor of mobile terminal, gesture variation and voice enquirement can also be converted to execution Order, and then the variation of virtual scene and information is caused to show, realization is interacted with scene;The virtual scene can be one Dummy model is also possible to a series of dummy model compositions;
In the theme that the exhibition panel assembly includes the identification figure of the embedded triggering virtual scene subsystem, to be shown The downloading private link of appearance and augmented reality subsystem;The downloading private link is located at the panel lower left corner;The identification figure Positioned at the panel lower right corner;
The system can be realized traditional by the method for smart phone or Intelligent flat installation App and scanning recognition figure The combination of publicity mode and augmented reality.
Due to the adoption of the above technical scheme, augmented reality display board system provided by the invention, has compared with prior art It is such the utility model has the advantages that
(1) On-site Experience sense is promoted.The present invention can both complete to enhance the interactivity of exhibition to by the introduction of exhibiting objects And artistry, and bandwagon effect can be enhanced, so that user's multi-angle, comprehensive understanding is demonstrated content;
(2) at low cost, a wide range of can be used with large-scale promotion.Exhibition panel assembly of the invention and common panel difference are not Greatly, the identification figure of link augmented reality subsystem has only been embedded, therefore at low cost.The Intelligent mobile equipment is general at present And do not increase user cost, therefore be suitble to a wide range of, large-scale promotion;
(3) sense of reality of shown content is promoted.The present invention using three-dimensional virtual scene, gesture interaction and with text, sound The mode of sound interaction provides displaying, strong sense of reality for user.
Present system is not necessarily to mobile network or WiFi, can be completed to by the comprehensive displaying of exhibiting objects, it can Enhance the interactivity and artistry of exhibition, and cheap, convenient for promoting.
Figure of description
Fig. 1 is present system entire block diagram;
Fig. 2 is exhibition panel assembly schematic diagram;
Fig. 3 is augmented reality subsystem interfaces schematic diagram;
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.
The augmented reality display board system of a kind of interactivity of the invention, as shown in Figure 1, the system includes that intelligent movable is set Standby, exhibition panel assembly and augmented reality subsystem;
The intelligent movable equipment is using the smart phone for the included camera sold on the market or the intelligence of included camera Energy plate, the present embodiment use Android smartphone, for scanning panel actual content, and carry out image recognition.
In the theme that the exhibition panel assembly includes the identification figure of the embedded triggering virtual scene subsystem, to be shown The downloading private link of appearance and augmented reality subsystem;The downloading private link is located at the panel lower left corner;The identification figure Positioned at the panel lower right corner.The composition of panel assembly is puted on display described in the present embodiment as shown in Fig. 2, shown by exhibiting objects brief introduction is normal It advises in exhibition device to by the introduction of exhibiting objects, including picture and text class introduction;It is identification at shown " target recognition chart " The two-dimentional code mark of this exhibiting objects.The link of augmented reality subsystem is placed on the panel lower left corner, and target recognition chart places position It is set to the lower right corner.
The augmented reality subsystem is will to import virtual emulation software by the model of exhibiting objects or scene, and be associated with The gesture interaction and speech interaction mode of threedimensional model is arranged, to trigger in video relevant to model, audio, text information User interest information, and be compiled into App and be installed to Android smartphone, in the case where being not necessarily to cellular mobile network or WiFi It can complete the identification and enhancing operation to identification figure.
The augmented reality subsystem includes virtual scene subsystem and human-machine interaction subsystem.
The virtual scene subsystem include it is various with reality by spectacular referring to the 3D model established and to be puted on display The processing of the corresponding three-dimensional virtual scene data of object;The three-dimensional virtual scene passes through 3D MAX software or three-dimensional laser scanner Modeling, the various forms data such as the introductory video relevant to model of AutoLink built in 3D model, text, picture, audio, with Abundant 3D model content.Three-dimensional virtual scene is mainly the three-dimensional enhancing display of real scene, is needed in panel manufacturing process Model creation is completed, and completes data correlation and the movement of model by 3D simulation software (the present embodiment uses Unity3D) Setting.The three-dimensional virtual scene be not it is single, can be and be made of multiple virtual scenes or three-dimension object, these scenes Or three-dimension object has substantial consistency with the text and picture introduction look in panel assembly.The human-machine interaction subsystem is used for The touch screen of analysis identification exhibitor/either slide the movement such as screen/or click screen, and be related to showing content Related voice problem;And convert recognition result to and execute order, complete to show the multi-angle of virtual scene subsystem and Answer.The gesture variation and voice enquirement of its mobile terminal can be converted to and execute order, and then lead to the virtual scene Variation and information show, realize and interacted with scene.The virtual scene can be a dummy model, be also possible to a system Column dummy model composition;
The interface of the augmented reality subsystem is as shown in Figure 3.In figure: shown " gesture interaction " is for selecting gesture interaction Mode, in such a mode, user can rotate freely, scale the dummy object being superimposed upon on picture, 360 degree of observation institute display materials Body or scene;Shown " interactive voice " for selecting speech interaction mode, in such a mode, user can propose simple problem; Shown " 360 degree of panoramas are shown " is for the shown object of 360 degree of observations or scene;Shown " prompt " is for prompting currently to identify State includes the prompting character for prompting current state, is known for changing the virtual key and touch gestures of current identification state Not;Shown " button " is virtual key for changing current identification state, and shown " triangle symbol " is touch recognition for temporary Stop the broadcasting of multimedia/3D virtual scene.
Human-machine interaction subsystem described in the present embodiment is mainly programmed by C# language and is completed, and main interactive mode includes:
1, gesture interaction, for completing rotation, the Pan and Zoom of model.These need operator to pass through touch screen or gesture To complete, operating process is identical as existing mobile phone interaction operation mode for operation;
2, model is selected: display high-brightness frame after operator selects the model to be shown, model selected by touch screen;
3, interactive voice: after model is selected, operator selects speech interaction mode, and voice puts question to ask relevant to model Topic, such as " what display articles attribute is? ", " the building up the age of the building " etc., the speech recognition that this functional module uses API after system successfully identifies, can recall the corresponding text of model, voice, video introduction automatically.
In the present embodiment, it is completed based on Unity3d in panel design and production phase, augmented reality subsystem uses C# Language development.Projects file is packaged as APK file, and corresponding model file is with the storage of FBX format resource packet, corresponding view Frequency file is MP4 format, and corresponding picture format is jpeg format.APK file is installed to mobile phone end, can be not necessarily to honeycomb It can complete the identification to identification figure and enhance to operate in the case where mobile network or WiFi.User in use, hand Gesture variation and voice enquirement can be converted to the variation and information for executing order, and then can lead to augmented reality subsystem virtual scene It shows, realization is interacted with scene.

Claims (1)

1. a kind of augmented reality display board system of interactivity, it is characterised in that: the system comprises intelligent movable equipment, enhancing Real subsystem and exhibition panel assembly;
The intelligent movable equipment includes the input terminal and processor of smart phone or Intelligent flat as the system, for sweeping Real image is retouched, and carries out image recognition;
The augmented reality subsystem, be will by the model of exhibiting objects or scene import virtual emulation software, and be associated with and mould The gesture interaction and speech interaction mode of threedimensional model is arranged, to trigger user in the relevant video of type, audio, text information Information of interest, and be compiled into App and be installed to smart phone or Intelligent flat, the case where being not necessarily to cellular mobile network or WiFi The identification and enhancing operation to identification figure can be completed down;
The augmented reality subsystem includes virtual scene subsystem and human-machine interaction subsystem;
The virtual scene subsystem includes by spectacular with reality for referring to the dummy model, corresponding by spectacular established Database and data attribute information;
The human-machine interaction subsystem for identification exhibitor slide screen/by touching screen/either or click screen with The problem of movement and voice input, and convert recognition result to and execute order, complete the multi-angle to virtual scene subsystem Displaying and answer;As the exhibition visitor of mobile terminal, gesture variation and voice enquirement can also be converted to and execute order, And then the variation of virtual scene and information is caused to show, realization is interacted with scene;The virtual scene can be one virtually Model is also possible to a series of dummy model compositions;
The subject content that the exhibition panel assembly includes the identification figure of the embedded triggering virtual scene subsystem, to be shown with And the downloading private link of augmented reality subsystem;The downloading private link is located at the panel lower left corner;The identification figure is located at The panel lower right corner;
The system can realize traditional publicity by the method that smart phone or Intelligent flat install App and scanning recognition figure The combination of mode and augmented reality.
CN201811311765.1A 2018-11-06 2018-11-06 Augmented reality panel system with interchangeability Active CN109508090B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811311765.1A CN109508090B (en) 2018-11-06 2018-11-06 Augmented reality panel system with interchangeability

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811311765.1A CN109508090B (en) 2018-11-06 2018-11-06 Augmented reality panel system with interchangeability

Publications (2)

Publication Number Publication Date
CN109508090A true CN109508090A (en) 2019-03-22
CN109508090B CN109508090B (en) 2021-01-08

Family

ID=65747662

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811311765.1A Active CN109508090B (en) 2018-11-06 2018-11-06 Augmented reality panel system with interchangeability

Country Status (1)

Country Link
CN (1) CN109508090B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110047148A (en) * 2019-04-10 2019-07-23 珠海梅西互动技术有限公司 A kind of the emulation interactive visual system and implementation method of virtual robot work station
CN111679742A (en) * 2020-06-10 2020-09-18 浙江商汤科技开发有限公司 Interaction control method and device based on AR, electronic equipment and storage medium
CN111949904A (en) * 2019-05-16 2020-11-17 腾讯科技(深圳)有限公司 Data processing method and device based on browser and terminal
CN112750167A (en) * 2020-12-30 2021-05-04 燕山大学 Robot vision positioning simulation method and device based on virtual reality

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104461006A (en) * 2014-12-17 2015-03-25 卢晨华 Internet intelligent mirror based on natural user interface
US20150234477A1 (en) * 2013-07-12 2015-08-20 Magic Leap, Inc. Method and system for determining user input based on gesture
CN105929945A (en) * 2016-04-18 2016-09-07 展视网(北京)科技有限公司 Augmented reality interaction method and device, mobile terminal and mini-computer
CN106126014A (en) * 2016-06-16 2016-11-16 北京小米移动软件有限公司 The generation method and device of interactive interface
CN106485779A (en) * 2016-03-22 2017-03-08 智合新天(北京)传媒广告股份有限公司 A kind of 3D virtual interacting display platform and the method for showing 3D animation
CN107608649A (en) * 2017-11-02 2018-01-19 泉州创景视迅数字科技有限公司 A kind of AR augmented realities intelligent image identification displaying content system and application method
CN108022543A (en) * 2017-11-27 2018-05-11 深圳中科呼图电子商务有限公司 A kind of advertisement autonomous demenstration method, system and advertisement machine and application
US20180158248A1 (en) * 2012-05-01 2018-06-07 Augmented Reality Holdings, Llc System, Method and Apparatus Of Eye Tracking or Gaze Detection Applications Including Facilitating Action on or Interaction with a Simulated Object

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180158248A1 (en) * 2012-05-01 2018-06-07 Augmented Reality Holdings, Llc System, Method and Apparatus Of Eye Tracking or Gaze Detection Applications Including Facilitating Action on or Interaction with a Simulated Object
US20150234477A1 (en) * 2013-07-12 2015-08-20 Magic Leap, Inc. Method and system for determining user input based on gesture
CN104461006A (en) * 2014-12-17 2015-03-25 卢晨华 Internet intelligent mirror based on natural user interface
CN106485779A (en) * 2016-03-22 2017-03-08 智合新天(北京)传媒广告股份有限公司 A kind of 3D virtual interacting display platform and the method for showing 3D animation
CN105929945A (en) * 2016-04-18 2016-09-07 展视网(北京)科技有限公司 Augmented reality interaction method and device, mobile terminal and mini-computer
CN106126014A (en) * 2016-06-16 2016-11-16 北京小米移动软件有限公司 The generation method and device of interactive interface
CN107608649A (en) * 2017-11-02 2018-01-19 泉州创景视迅数字科技有限公司 A kind of AR augmented realities intelligent image identification displaying content system and application method
CN108022543A (en) * 2017-11-27 2018-05-11 深圳中科呼图电子商务有限公司 A kind of advertisement autonomous demenstration method, system and advertisement machine and application

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110047148A (en) * 2019-04-10 2019-07-23 珠海梅西互动技术有限公司 A kind of the emulation interactive visual system and implementation method of virtual robot work station
CN110047148B (en) * 2019-04-10 2024-01-26 珠海惠中智能技术有限公司 Simulation interaction visualization system of virtual robot workstation and implementation method
CN111949904A (en) * 2019-05-16 2020-11-17 腾讯科技(深圳)有限公司 Data processing method and device based on browser and terminal
CN111679742A (en) * 2020-06-10 2020-09-18 浙江商汤科技开发有限公司 Interaction control method and device based on AR, electronic equipment and storage medium
CN112750167A (en) * 2020-12-30 2021-05-04 燕山大学 Robot vision positioning simulation method and device based on virtual reality

Also Published As

Publication number Publication date
CN109508090B (en) 2021-01-08

Similar Documents

Publication Publication Date Title
Skublewska-Paszkowska et al. 3D technologies for intangible cultural heritage preservation—literature review for selected databases
CN109508090A (en) A kind of augmented reality display board system having interactivity
Ardito et al. Interaction with large displays: A survey
Ishii Tangible bits: beyond pixels
Manovich The poetics of augmented space
CN103034395B (en) Equipment and method for Asynchronous Communication
CN109710206A (en) Show method, apparatus, terminal and the storage medium of information
WO2006025468A1 (en) Presentation system
CN107870711A (en) Page navigation method, the method and client that user interface is provided
CN104504576A (en) Method for achieving three-dimensional display interaction of clothes and platform
WO2023030010A1 (en) Interaction method, and electronic device and storage medium
Binder et al. Supporting configurability in a mixed-media environment for design students
CN104616342A (en) Interconversion method of sequence frame and panorama
WO2013182142A1 (en) Terminal device desktop implementation method, system and terminal device
KR101679791B1 (en) Display opperation system using mobile apparatus and method thereof
Gehring et al. Interaction with media façades
Luo et al. Research on the application of environmental art design based on digital media technology
CN110456901A (en) Control method, system, electronic equipment and the storage medium that object is shown in exhibition
CN114449355B (en) Live interaction method, device, equipment and storage medium
TW201101077A (en) Constructing method and system for 3D environmental appearance simulation
Xie et al. The application of augmented reality technology in digital display for intangible cultural heritage: The case of cantonese furniture
CN109947247B (en) Somatosensory interaction display system and method
Hu et al. Development based on Unity3D virtual campus roaming system
Li et al. Exploring the implementation of web, mobile, and VR technology for cultural heritage presentation
Wang et al. A case study of interactive technology applied to the thematic exhibition activities in city and rural areas

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant