CN114063512B - Maintenance service guiding and monitoring method, cloud platform, AR glasses and system - Google Patents

Maintenance service guiding and monitoring method, cloud platform, AR glasses and system Download PDF

Info

Publication number
CN114063512B
CN114063512B CN202111347948.0A CN202111347948A CN114063512B CN 114063512 B CN114063512 B CN 114063512B CN 202111347948 A CN202111347948 A CN 202111347948A CN 114063512 B CN114063512 B CN 114063512B
Authority
CN
China
Prior art keywords
maintenance
equipment
glasses
maintenance service
receiving
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111347948.0A
Other languages
Chinese (zh)
Other versions
CN114063512A (en
Inventor
王文哲
赵文东
安岗
王源野
钱红赫
范天伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China United Network Communications Group Co Ltd
Original Assignee
China United Network Communications Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China United Network Communications Group Co Ltd filed Critical China United Network Communications Group Co Ltd
Priority to CN202111347948.0A priority Critical patent/CN114063512B/en
Publication of CN114063512A publication Critical patent/CN114063512A/en
Application granted granted Critical
Publication of CN114063512B publication Critical patent/CN114063512B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/042Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
    • G05B19/0428Safety, monitoring
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/20Pc systems
    • G05B2219/24Pc safety
    • G05B2219/24024Safety, surveillance

Abstract

The invention provides a maintenance service guiding and monitoring method, a cloud platform, AR glasses and a system, wherein the method comprises the following steps: receiving basic data of maintenance service work orders and equipment to be maintained, wherein the basic data are created and sent by a manufacturing execution system MES; acquiring corresponding Augmented Reality (AR) virtual scene content according to the basic data; sending the AR virtual scene content to front-end AR glasses according to the maintenance service work order, so that the front-end AR glasses guide maintenance personnel to execute maintenance service according to the AR virtual scene content; receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses; receiving maintenance task ending information sent by front-end AR glasses when the maintenance task is detected to be ended; and sending the maintenance task ending information to the MES to trigger the MES to receive the execution result data of the maintenance service, closing the maintenance service work order and informing the front-end AR glasses. The invention solves the problems that maintenance service is not integrated with MES in the prior art, and closed-loop informationized monitoring from production arrangement to maintenance execution is not formed.

Description

Maintenance service guiding and monitoring method, cloud platform, AR glasses and system
Technical Field
The invention relates to the technical field of intelligent manufacturing and augmented reality, in particular to a maintenance service guiding and monitoring method, a cloud platform, AR glasses and a system.
Background
With the rapid development of industrial internet, artificial intelligence and 5G technologies and the change of future economic structures, the augmented reality (Augmented Reality, AR) technology which seamlessly connects the real world with the virtual world, and the equipment maintenance service combined with 5G, cloud computing, big data and artificial intelligence (artificial intelligence, AI) are also becoming important application directions of many enterprises to improve the operation and maintenance efficiency and reduce the operation and maintenance cost.
The AR technology is applied to the prior art of equipment maintenance, and no maintenance service AR guiding method is integrated with an actual production and manufacturing execution system (Manufacturing Execution System, MES) and is integrated, and no informationized maintenance service AR monitoring method is formed from the service perspective to form a real closed loop in the maintenance execution process.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a maintenance service guiding and monitoring method, a cloud platform, AR glasses and a system aiming at the defects of the prior art, so as to solve the problems that maintenance service is not integrated and fused with MES in the prior art, and closed-loop informationized monitoring from production scheduling to maintenance execution is not formed from a service angle.
In a first aspect, the present invention provides a method for guiding and monitoring a maintenance service, which is applied to a maintenance cloud platform, and the method includes:
receiving basic data of maintenance service work orders and equipment to be maintained, wherein the basic data are created and sent by a manufacturing execution system MES;
acquiring corresponding Augmented Reality (AR) virtual scene content according to the basic data of the equipment to be maintained;
sending the AR virtual scene content to front-end AR glasses according to the maintenance service work order, so that the front-end AR glasses guide maintenance personnel to execute maintenance service according to the AR virtual scene content;
receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses;
receiving maintenance task ending information sent by the front-end AR glasses when the front-end AR glasses detect that the maintenance task is ended;
and sending the maintenance task ending information to the MES to trigger the MES to receive the execution result data of the maintenance service, closing the maintenance service work order and informing the front-end AR glasses.
Preferably, the AR virtual scene content includes a shop layout map, location information, and a maintenance guidance virtual content;
before receiving the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the MES, the method further comprises:
receiving workshop environment and equipment acquisition information sent by a front-end environment scanning tool;
generating a workshop layout map according to the workshop environment and the equipment acquisition information;
the obtaining the corresponding augmented reality AR virtual scene content according to the basic data of the equipment to be maintained specifically includes:
reading prestored maintenance instruction virtual content according to the basic data of the equipment to be maintained;
determining equipment aimed by the current maintenance service according to the basic data of the equipment to be maintained;
and determining the position information of the equipment aimed at by the local maintenance service according to the workshop layout map, wherein the position information comprises the position information of the equipment and the position information of an operating part of the equipment.
Preferably, before receiving the basic data of the maintenance service worksheet and the equipment to be maintained, which are created and sent by the manufacturing execution system MES, the method further includes:
receiving model materials and maintenance operation outline;
and generating AR animation maintenance guidance virtual contents aiming at the model materials according to the maintenance operation outline.
Preferably, after the AR virtual scene content is sent to front-end AR glasses according to the maintenance service work order, the method further includes:
receiving environment identification data sent by the front-end AR glasses;
positioning equipment aimed at the maintenance service according to the equipment position information and the environment identification data, and generating corresponding equipment positioning guide information;
and positioning the operation part corresponding to the maintenance instruction virtual content according to the operation part position information and the environment identification information, and generating corresponding operation part positioning instruction information.
Preferably, the equipment for the present maintenance service is specifically Printed Circuit Board (PCB) drilling equipment;
the model material comprises a three-dimensional model of PCB drilling equipment;
the maintenance operation outline comprises the operation steps of guide rail dismantling dust removal, drill bit equal height debugging and drill bit torsion deflection test of PCB drilling equipment;
the virtual contents of the AR animation maintenance instruction comprise guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
In a second aspect, the present invention provides a method for guiding and monitoring a maintenance service, which is applied to front-end AR glasses, and the method includes:
receiving Augmented Reality (AR) scene virtual contents sent by a maintenance cloud platform according to a maintenance service work order, wherein the AR scene virtual contents are sent after the maintenance cloud platform receives basic data of the maintenance service work order and equipment to be maintained, which are created and sent by a Manufacturing Execution System (MES), and acquires corresponding AR virtual scene contents according to the basic data of the equipment to be maintained;
guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content;
monitoring and transmitting the process of executing the maintenance service by the maintenance personnel;
when the maintenance task is detected to be ended, transmitting maintenance task ending information to the maintenance cloud platform;
and sending the execution result data of the maintenance service to the MES, and receiving the notification of closing the maintenance service work order sent by the MES.
Preferably, the AR virtual scene content includes a shop layout map, positioning information, and maintenance guidance virtual content;
the guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content specifically comprises the following steps:
starting an environment recognition engine to acquire environment recognition data, and transmitting the environment recognition data to the maintenance cloud platform;
receiving equipment positioning guide information sent by the maintenance cloud platform, and guiding maintenance personnel to position equipment aimed at the maintenance service;
and receiving the operation part positioning guide information sent by the maintenance cloud platform, guiding a maintenance person to position an operation part corresponding to the maintenance guide virtual content, and operating the corresponding operation part according to the maintenance guide virtual content.
In a third aspect, the present invention provides a maintenance cloud platform, including: the device comprises a first receiving module, an acquisition module and a first sending module;
the first receiving module is used for receiving the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the manufacturing execution system MES;
the acquisition module is connected with the first receiving module and is used for acquiring corresponding Augmented Reality (AR) virtual scene content according to the basic data of the equipment to be maintained;
the first sending module is connected with the obtaining module and is used for sending the AR virtual scene content to front-end AR glasses according to the maintenance service work order, so that the front-end AR glasses guide maintenance personnel to execute maintenance service according to the AR virtual scene content;
the first receiving module is connected with the first sending module and is also used for receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses; the front-end AR glasses are also used for receiving maintenance task ending information sent when the front-end AR glasses detect that the maintenance task is ended;
the first sending module is connected with the first receiving module and is further configured to send the maintenance task end information to the MES, so as to trigger the MES to receive execution result data of the maintenance service, close the maintenance service worksheet and notify the front-end AR glasses.
In a fourth aspect, the present invention provides front AR glasses comprising: the system comprises a second receiving module, a guiding module, a monitoring module and a second sending module;
the second receiving module is used for receiving the augmented reality AR scene virtual content sent by the maintenance cloud platform according to the maintenance service work order, wherein the AR scene virtual content is sent after the maintenance cloud platform receives the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the manufacturing execution system MES, and obtains the corresponding AR virtual scene content according to the basic data of the equipment to be maintained;
the guiding module is connected with the second receiving module and is used for guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content;
the monitoring module is connected with the guiding module and is used for monitoring and transmitting the process of executing the maintenance service by the maintenance personnel;
the second sending module is connected with the monitoring module and is used for sending maintenance task ending information to the maintenance cloud platform when the maintenance task ending is detected; and the system is also used for sending the execution result data of the maintenance service to the MES and receiving the notification of closing the maintenance service work order sent by the MES.
In a fifth aspect, the present invention provides a maintenance service guiding and monitoring system, including: manufacturing an execution system MES, a maintenance cloud platform and front-end AR glasses;
the maintenance cloud platform is used for executing the maintenance service guiding and monitoring method;
the front-end AR glasses are used for executing the maintenance service guiding and monitoring method.
The invention provides a maintenance service guiding and monitoring method, a cloud platform, AR glasses and a system, wherein a maintenance cloud platform is established, a manufacturing execution system MES and front-end AR glasses are connected in a butt joint mode, maintenance tasks are automatically and synchronously acquired from the manufacturing execution system MES, the maintenance cloud platform is used for sending AR virtual scene content corresponding to the maintenance tasks to the front-end AR glasses, the maintenance tasks are guided by an AR technology to be executed by maintenance personnel, the maintenance task execution process is synchronously monitored, after the maintenance task execution is finished, the manufacturing execution system MES is notified to receive execution result data of maintenance service, a maintenance service work order is closed, meanwhile, the front-end AR glasses are notified, maintenance service and the manufacturing execution system MES are integrated and communicated, and closed-loop informatization monitoring from arrangement to the maintenance execution process is formed from a service angle.
Drawings
FIG. 1 is a schematic diagram of a system for guiding and monitoring maintenance business according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for guiding and monitoring maintenance business according to embodiment 1 of the present invention;
FIG. 3 is a flow chart of another method for guiding and monitoring maintenance business according to embodiment 2 of the present invention;
fig. 4 is a schematic structural diagram of a maintenance cloud platform according to embodiment 3 of the present invention;
fig. 5 is a schematic structural diagram of front AR glasses according to embodiment 4 of the present invention.
Detailed Description
In order to make the technical scheme of the present invention better understood by those skilled in the art, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings.
It is to be understood that the specific embodiments and figures described herein are merely illustrative of the invention, and are not limiting of the invention.
It is to be understood that the various embodiments of the invention and the features of the embodiments may be combined with each other without conflict.
It is to be understood that only the portions relevant to the present invention are shown in the drawings for convenience of description, and the portions irrelevant to the present invention are not shown in the drawings.
It should be understood that each unit and module in the embodiments of the present invention may correspond to only one physical structure, may be formed by a plurality of physical structures, or may be integrated into one physical structure.
It will be appreciated that, without conflict, the functions and steps noted in the flowcharts and block diagrams of the present invention may occur out of the order noted in the figures.
It is to be understood that the flowcharts and block diagrams of the present invention illustrate the architecture, functionality, and operation of possible implementations of systems, apparatuses, devices, methods according to various embodiments of the present invention. Where each block in the flowchart or block diagrams may represent a unit, module, segment, code, or the like, which comprises executable instructions for implementing the specified functions. Moreover, each block or combination of blocks in the block diagrams and flowchart illustrations can be implemented by hardware-based systems that perform the specified functions, or by combinations of hardware and computer instructions.
It should be understood that the units and modules related in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, for example, the units and modules may be located in a processor.
Example 1:
the embodiment 1 of the invention provides a maintenance service guiding and monitoring method, which is applied to a maintenance cloud platform 2 as shown in fig. 1.
Specifically, the maintenance cloud platform 2 is located in a maintenance service guiding and monitoring system shown in fig. 1, and the maintenance service guiding and monitoring system further includes a manufacturing execution system MES1 and front-end AR glasses 3.
As shown in fig. 2, the method includes:
s21, receiving the basic data of the maintenance service work order and the equipment to be maintained, which are created and transmitted by the manufacturing execution system MES 1.
Specifically, in this embodiment, the maintenance cloud platform 2 is connected to the manufacturing execution system MES1, and is configured to automatically obtain a maintenance task from the manufacturing execution system MES1, when the manufacturing execution system MES1 issues the maintenance task, issue the maintenance task in the form of a maintenance service work order, and send basic data of a device to be maintained at the same time, where the maintenance task includes two forms of an instant task and a planned task, the planned task may specify a task execution time point and an execution period, the maintenance service work order may specify a task pushing recipient, that is, a maintenance person executing a maintenance service, and the basic data of the device to be maintained is obtained from the manufacturing execution system MES1 according to the maintenance service work order.
In this embodiment, before receiving the basic data of the maintenance service work order and the equipment to be maintained created and sent by the manufacturing execution system MES1, the method further includes: receiving model materials and maintenance operation outline; and generating AR animation maintenance guidance virtual contents aiming at the model materials according to the maintenance operation outline.
Specifically, in this embodiment, the maintenance cloud platform 2 includes a maintenance editor, the maintenance editor receives a device three-dimensional model made by a modeler, and further receives a maintenance operation outline, according to the maintenance operation outline, on the basis of an existing three-dimensional model, AR animation maintenance guidance virtual content is generated, and may also include text and audio guidance, the maintenance editor is provided with a user interface, and a person with corresponding authority may log in the user interface to perform operations such as creating, modifying, viewing, and the like on the AR animation maintenance guidance virtual content.
In this embodiment, before receiving the basic data of the maintenance service work order and the equipment to be maintained created and sent by the manufacturing execution system MES1, the method further includes: receiving workshop environment and equipment acquisition information sent by a front-end environment scanning tool; and generating a workshop layout map according to the workshop environment and the equipment acquisition information.
Specifically, in this embodiment, the layout of the collection device in the workshop and the appearance structure of the device are scanned and collected in the workshop in the device workshop in advance by using an environment scanning tool (including a mobile phone or a tablet computer or a professional camera or a camera of the front-end AR glasses 3, etc.), and the layout map of the workshop is generated and stored in the maintenance cloud platform 2.
S22, obtaining the corresponding augmented reality AR virtual scene content according to the basic data of the equipment to be maintained.
In this embodiment, the AR virtual scene content includes a shop layout map, location information, and maintenance instruction virtual content; the obtaining the corresponding augmented reality AR virtual scene content according to the basic data of the equipment to be maintained specifically includes: reading prestored maintenance instruction virtual content according to the basic data of the equipment to be maintained; determining equipment aimed by the current maintenance service according to the basic data of the equipment to be maintained; and determining the position information of the equipment aimed at by the local maintenance service according to the workshop layout map, wherein the position information comprises the position information of the equipment and the position information of an operating part of the equipment.
Specifically, in this embodiment, the maintenance editor of the maintenance cloud platform 2 outputs the maintenance guidance virtual content in the ARPackage format, after the maintenance cloud platform 2 receives the maintenance task, it needs to read the maintenance guidance virtual content and the shop layout map according to the basic data of the equipment to be maintained in the maintenance task, determine the equipment to which the maintenance service is to be performed at this time, and determine the equipment position information to be operated by the maintenance service and the operating component position information on the equipment on the shop layout map, so that the front-end AR glasses 3 can position the equipment and position the operating component to be operated by each step of the maintenance guidance virtual content to be superimposed on the equipment through the environment recognition engine when identifying the equipment object, and the maintenance personnel can find the equipment to be maintained and operate according to the glasses display data.
S23, sending the AR virtual scene content to front-end AR glasses 3 according to the maintenance service work order, so that the front-end AR glasses 3 guide maintenance personnel to execute maintenance service according to the AR virtual scene content.
In this embodiment, after the AR virtual scene content is sent to the front-end AR glasses 3 according to the maintenance service worksheet, the method further includes: receiving environment identification data sent by the front-end AR glasses 3; positioning equipment aimed at the maintenance service according to the equipment position information and the environment identification data, and generating corresponding equipment positioning guide information; and positioning the operation part corresponding to the maintenance instruction virtual content according to the operation part position information and the environment identification information, and generating corresponding operation part positioning instruction information.
Specifically, in this embodiment, the front-end AR glasses 3 support a 5G function, have high-precision SLAM (instant localization and mapping, simultaneous Localization And Mapping) capability, enable 3D object/space recognition, implement AR virtual scene content and physical equipment superposition display through an AR space localization technology, anchor virtual content in real space, support 360-degree observation and operation of virtual content, support execution of all operation steps defined by a maintenance task, view virtual operation hints, hint content include text, pictures, videos, models, animations, etc. anchored on a device, view device information data AR holographic display, and display content includes a line drawing, a histogram, a web page, etc. anchored on the device.
And S24, receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses 3.
Specifically, in this embodiment, after the maintenance personnel clicks the start task, the front-end AR glasses 3 scan the equipment and then start to execute the maintenance guidance step, and synchronously record the process video in the process of guiding and executing each step, and manually trigger shooting to collect the pictures of any step as required to return to the maintenance cloud platform 2 for storage, so as to realize informationized monitoring of the maintenance process.
And S25, receiving maintenance task ending information sent by the front-end AR glasses 3 when the maintenance task is detected to be ended.
Specifically, in this embodiment, after all the maintenance steps are completed, the front-end AR glasses 3 submits the maintenance task end information, and the maintenance cloud platform 2 returns the maintenance task end information to the manufacturing execution system MES1, triggers the manufacturing execution system MES1 to receive maintenance execution result data, and the manufacturing execution system MES1 further closes the maintenance service work order.
S26, sending the maintenance task ending information to the MES1 to trigger the MES1 to receive execution result data of the maintenance service, closing the maintenance service work order and informing the front-end AR glasses 3.
In the embodiment, the equipment for which the present maintenance service is aimed is specifically a printed circuit board (Printed Circuit board) PCB drilling machine equipment; the model material comprises a three-dimensional model of PCB drilling equipment; the maintenance operation outline comprises the operation steps of guide rail dismantling dust removal, drill bit equal height debugging and drill bit torsion deflection test of PCB drilling equipment; the virtual contents of the AR animation maintenance instruction comprise guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
Specifically, in this embodiment, the virtual contents of the AR animation maintenance instruction include visual electronic instruction book of the maintenance operation of the PCB drilling machine equipment, which includes the instruction contents of removing dust from the guide rail of the PCB drilling machine equipment, debugging the height of the drill bit, testing the torsion deflection of the drill bit, and the like, and the AR technology is used as a carrier, so that the maintenance operation flow of the PCB drilling machine equipment is converted into a more visual AR operation instruction manual through the forms of text, video, model animation, and the like, which is easier to be accepted by an operator, and the purposes of monitoring the full life cycle of the maintenance of the PCB drilling machine equipment, depositing a knowledge base, developing staff training, and the like are finally realized in an augmented reality manner, so that the understanding threshold is reduced, the efficiency of work and study is improved.
The embodiment 1 of the invention provides a maintenance service guiding and monitoring method, which comprises the steps of establishing a maintenance cloud platform 2, butting a manufacturing execution system MES1 and front-end AR glasses 3, automatically and synchronously acquiring maintenance tasks from the manufacturing execution system MES1, sending AR virtual scene content corresponding to the maintenance tasks to the front-end AR glasses 3 through the maintenance cloud platform 2 so as to guide maintenance personnel to execute the maintenance tasks by using an AR technology, synchronously monitoring the execution process of the maintenance tasks, notifying the manufacturing execution system MES1 to receive execution result data of the maintenance service through the maintenance cloud platform 2 after the execution of the maintenance tasks is finished, closing a maintenance service work order, notifying the front-end AR glasses 3, realizing integrated fusion and communication of the maintenance service and the manufacturing execution system MES1, and forming closed-loop informationized monitoring from the production to the maintenance execution process from a service angle.
Example 2:
as shown in fig. 3, embodiment 2 of the present invention provides a second aspect, and the present invention provides a method for guiding and monitoring a maintenance service, which is applied to front-end AR glasses 3, and the method includes:
s31, receiving Augmented Reality (AR) scene virtual contents sent by a maintenance cloud platform according to a maintenance service work order, wherein the AR scene virtual contents are sent after the maintenance cloud platform 2 receives basic data of the maintenance service work order and equipment to be maintained, which are created and sent by a manufacturing execution system (MES 1), and acquires corresponding AR virtual scene contents according to the basic data of the equipment to be maintained;
s32, guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content;
s33, monitoring and transmitting a process of executing the maintenance service by the maintenance personnel;
s34, when the maintenance task is detected to be ended, transmitting maintenance task ending information to the maintenance cloud platform;
s35, sending the execution result data of the maintenance service to the MES1, and receiving the notification of closing the maintenance service work order sent by the MES 1.
Optionally, the AR virtual scene content includes a shop layout map, positioning information, and maintenance guidance virtual content;
the guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content specifically comprises the following steps:
starting an environment recognition engine to acquire environment recognition data, and transmitting the environment recognition data to the maintenance cloud platform 2;
receiving equipment positioning guide information sent by the maintenance cloud platform 2, and guiding maintenance personnel to position equipment aimed at the maintenance service;
and receiving the operation part positioning guide information sent by the maintenance cloud platform 2, guiding a maintenance person to position an operation part corresponding to the maintenance guide virtual content, and operating the corresponding operation part according to the maintenance guide virtual content.
Optionally, the equipment for the present maintenance service is specifically a printed circuit board PCB drilling machine equipment;
the virtual content of the maintenance instruction comprises guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
Example 3:
as shown in fig. 1 and fig. 4, embodiment 3 of the present invention provides a third aspect, and the present invention provides a maintenance cloud platform 2, including: a first receiving module 21, an acquiring module 22 and a first transmitting module 23;
the first receiving module 21 is used for receiving the maintenance service work order and the basic data of the equipment to be maintained, which are created and sent by the manufacturing execution system MES 1;
the acquiring module 22 is connected with the first receiving module 21, and is configured to acquire corresponding augmented reality AR virtual scene content according to the basic data of the device to be maintained;
the first sending module 23 is connected to the obtaining module 22, and is configured to send the AR virtual scene content to the front-end AR glasses 3 according to the maintenance service worksheet, so that the front-end AR glasses 3 instruct a maintenance person to execute maintenance service according to the AR virtual scene content;
the first receiving module 21 is connected to the first transmitting module 23, and is further configured to receive and store monitoring information in the executing process of the maintenance service sent by the front-end AR glasses 3; and is further configured to receive maintenance task end information sent when the front-end AR glasses 3 detect that a maintenance task ends;
the first sending module 23 is connected to the first receiving module 21, and is further configured to send the maintenance task end information to the MES1, so as to trigger the MES1 to receive execution result data of the maintenance service, close the maintenance service work order, and notify the front-end AR glasses 3.
Optionally, the AR virtual scene content includes a shop layout map, location information, and maintenance guidance virtual content;
the maintenance cloud platform 2 further includes:
the workshop layout map generation module is used for receiving the workshop environment and equipment acquisition information sent by the front-end environment scanning tool and generating a workshop layout map according to the workshop environment and equipment acquisition information;
the acquiring module 22 specifically includes:
the reading unit is used for reading the prestored maintenance instruction virtual content according to the basic data of the equipment to be maintained;
the first determining unit is used for determining equipment aimed at by the current maintenance service according to the basic data of the equipment to be maintained;
and the second determining unit is used for determining the position information of the equipment aimed at by the current maintenance service according to the workshop layout map, wherein the position information comprises the equipment position information and the position information of an operating part of the equipment.
Optionally, the maintenance cloud platform 2 further includes:
and the maintenance editor is used for receiving the model materials and the maintenance operation outline and generating AR animation maintenance guidance virtual contents for the model materials according to the maintenance operation outline.
Optionally, the maintenance cloud platform 2 further includes:
the positioning module is used for receiving the environment identification data sent by the front-end AR glasses; positioning equipment aimed at the maintenance service according to the equipment position information and the environment identification data, and generating corresponding equipment positioning guide information; and positioning the operation part corresponding to the maintenance instruction virtual content according to the operation part position information and the environment identification information, and generating corresponding operation part positioning instruction information.
Optionally, the equipment for the present maintenance service is specifically a printed circuit board PCB drilling machine equipment;
the model material comprises a three-dimensional model of PCB drilling equipment;
the maintenance operation outline comprises the operation steps of guide rail dismantling dust removal, drill bit equal height debugging and drill bit torsion deflection test of PCB drilling equipment;
the virtual contents of the AR animation maintenance instruction comprise guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
Example 4:
as shown in fig. 1 and 5, embodiment 4 of the present invention provides front AR glasses 3, including: a second receiving module 31, a guiding module 32, a monitoring module 33 and a second transmitting module 34;
the second receiving module 31 is configured to receive the augmented reality AR scene virtual content sent by the maintenance cloud platform 2 according to the maintenance service work order, where the AR scene virtual content is sent after the maintenance cloud platform 2 receives the basic data of the maintenance service work order and the equipment to be maintained created and sent by the manufacturing execution system MES1, and obtains the corresponding AR virtual scene content according to the basic data of the equipment to be maintained;
the guiding module 32 is connected to the second receiving module 31, and is configured to guide a maintenance person to execute a maintenance service according to the AR virtual scene content;
the monitoring module 33 is connected with the guiding module 32 and is used for monitoring and sending the process of executing the maintenance service by the maintenance personnel;
the second sending module 34 is connected with the monitoring module 33, and is configured to send maintenance task end information to the maintenance cloud platform 2 when it is detected that the maintenance task ends; and the system is also used for sending the execution result data of the maintenance service to the MES1 and receiving the notification of closing the maintenance service work order sent by the MES 1.
Optionally, the AR virtual scene content includes a shop layout map, positioning information, and maintenance guidance virtual content;
the instruction module 32 specifically includes:
the environment recognition unit is used for starting an environment recognition engine to acquire environment recognition data and transmitting the environment recognition data to the maintenance cloud platform;
the first positioning unit is used for receiving the equipment positioning guide information sent by the maintenance cloud platform and guiding maintenance personnel to position equipment for the current maintenance service;
the second positioning unit is used for receiving the operation part positioning guide information sent by the maintenance cloud platform, guiding maintenance personnel to position the operation part corresponding to the maintenance guide virtual content, and operating the corresponding operation part according to the maintenance guide virtual content.
Optionally, the equipment for the present maintenance service is specifically a printed circuit board PCB drilling machine equipment;
the virtual content of the maintenance instruction comprises guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
Example 5:
as shown in fig. 1, embodiment 5 of the present invention provides a maintenance service guiding and monitoring system, including: manufacturing execution system MES1, maintenance cloud platform 2 and front-end AR glasses 3;
the maintenance cloud platform 2 is configured to execute the maintenance service guiding and monitoring method described in embodiment 1;
the front-end AR glasses 3 are used to perform the maintenance business guiding and monitoring method as described in embodiment 2.
The embodiments 1-5 of the present invention provide a guiding and monitoring method for maintenance service, a cloud platform, AR glasses and a system, by establishing the maintenance cloud platform 2, docking the manufacturing execution system MES1 with the front-end AR glasses 3, automatically and synchronously acquiring maintenance tasks from the manufacturing execution system MES1, sending AR virtual scene contents corresponding to the maintenance tasks to the front-end AR glasses 3 through the maintenance cloud platform 2, so as to achieve the purpose of guiding maintenance personnel to execute the maintenance tasks by using AR technology, synchronously monitoring the execution process of the maintenance tasks, notifying the manufacturing execution system MES1 to receive the execution result data of the maintenance service through the maintenance cloud platform 2 after the execution of the maintenance tasks is finished, closing the maintenance service work order, notifying the front-end AR glasses 3, realizing the integration and the completion of the maintenance service and the manufacturing execution system MES1, and forming closed-loop informationized monitoring from the production to the maintenance execution process from the service angle.
The embodiment of the invention discloses a closed-loop augmented reality AR guiding and monitoring method aiming at maintenance service of Printed Circuit Board (PCB) drilling machine equipment; meanwhile, the AR content universal editing function of the service scene is provided, and the AR content universal editing function can be expanded, popularized and copied to equipment maintenance AR informationized monitoring scenes in other fields.
It is to be understood that the above embodiments are merely illustrative of the application of the principles of the present invention, but not in limitation thereof. Various modifications and improvements may be made by those skilled in the art without departing from the spirit and substance of the invention, and are also considered to be within the scope of the invention.

Claims (10)

1. The utility model provides a dimension guaranty business guidance and monitoring method which is characterized in that is applied to dimension guaranty cloud platform, the method includes:
receiving basic data of maintenance service work orders and equipment to be maintained, wherein the basic data are created and sent by a manufacturing execution system MES;
acquiring corresponding Augmented Reality (AR) virtual scene content according to the basic data of the equipment to be maintained;
sending the AR virtual scene content to front-end AR glasses according to the maintenance service work order, so that the front-end AR glasses guide maintenance personnel to execute maintenance service according to the AR virtual scene content;
receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses;
receiving maintenance task ending information sent by the front-end AR glasses when the front-end AR glasses detect that the maintenance task is ended;
and sending the maintenance task ending information to the MES to trigger the MES to receive the execution result data of the maintenance service, closing the maintenance service work order and informing the front-end AR glasses.
2. The method of claim 1, wherein the AR virtual scene content comprises a shop floor map, location information, and maintenance guidance virtual content;
before receiving the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the MES, the method further comprises:
receiving workshop environment and equipment acquisition information sent by a front-end environment scanning tool;
generating a workshop layout map according to the workshop environment and the equipment acquisition information;
the obtaining the corresponding augmented reality AR virtual scene content according to the basic data of the equipment to be maintained specifically includes:
reading prestored maintenance instruction virtual content according to the basic data of the equipment to be maintained;
determining equipment aimed by the current maintenance service according to the basic data of the equipment to be maintained;
and determining the position information of the equipment aimed at by the local maintenance service according to the workshop layout map, wherein the position information comprises the position information of the equipment and the position information of an operating part of the equipment.
3. The method of claim 2, wherein prior to receiving the basic data of the maintenance service worksheet and the equipment to be maintained created and transmitted by the manufacturing execution system MES, the method further comprises:
receiving model materials and maintenance operation outline;
and generating AR animation maintenance guidance virtual contents aiming at the model materials according to the maintenance operation outline.
4. The method of claim 2, wherein after the sending the AR virtual scene content to front-end AR glasses according to the maintenance service worksheet, the method further comprises:
receiving environment identification data sent by the front-end AR glasses;
positioning equipment aimed at the maintenance service according to the equipment position information and the environment identification data, and generating corresponding equipment positioning guide information;
and positioning the operation part corresponding to the maintenance instruction virtual content according to the operation part position information and the environment identification information, and generating corresponding operation part positioning instruction information.
5. A method according to claim 3, characterized in that the equipment for which the present maintenance service is directed is in particular printed circuit board, PCB, drilling equipment;
the model material comprises a three-dimensional model of PCB drilling equipment;
the maintenance operation outline comprises the operation steps of guide rail dismantling dust removal, drill bit equal height debugging and drill bit torsion deflection test of PCB drilling equipment;
the virtual contents of the AR animation maintenance instruction comprise guide rail dismantling dust removal, drill bit equal height debugging and digital operation workflow step-by-step instruction of drill bit torsion deflection test.
6. The utility model provides a maintenance business guidance and monitoring method, which is characterized in that the method is applied to front-end AR glasses, and comprises the following steps:
receiving Augmented Reality (AR) scene virtual contents sent by a maintenance cloud platform according to a maintenance service work order, wherein the AR scene virtual contents are sent after the maintenance cloud platform receives basic data of the maintenance service work order and equipment to be maintained, which are created and sent by a Manufacturing Execution System (MES), and acquires corresponding AR virtual scene contents according to the basic data of the equipment to be maintained;
guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content;
monitoring and transmitting the process of executing the maintenance service by the maintenance personnel;
when the maintenance task is detected to be ended, transmitting maintenance task ending information to the maintenance cloud platform;
and sending the execution result data of the maintenance service to the MES, and receiving the notification of closing the maintenance service work order sent by the MES.
7. The method of claim 6, wherein the AR virtual scene content includes a shop floor map, positioning information, and maintenance guidance virtual content;
the guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content specifically comprises the following steps:
starting an environment recognition engine to acquire environment recognition data, and transmitting the environment recognition data to the maintenance cloud platform;
receiving equipment positioning guide information sent by the maintenance cloud platform, and guiding maintenance personnel to position equipment aimed at the maintenance service;
and receiving the operation part positioning guide information sent by the maintenance cloud platform, guiding a maintenance person to position an operation part corresponding to the maintenance guide virtual content, and operating the corresponding operation part according to the maintenance guide virtual content.
8. The utility model provides a dimension protects cloud platform which characterized in that includes: the device comprises a first receiving module, an acquisition module and a first sending module;
the first receiving module is used for receiving the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the manufacturing execution system MES;
the acquisition module is connected with the first receiving module and is used for acquiring corresponding Augmented Reality (AR) virtual scene content according to the basic data of the equipment to be maintained;
the first sending module is connected with the obtaining module and is used for sending the AR virtual scene content to front-end AR glasses according to the maintenance service work order, so that the front-end AR glasses guide maintenance personnel to execute maintenance service according to the AR virtual scene content;
the first receiving module is connected with the first sending module and is also used for receiving and storing monitoring information in the executing process of the maintenance service sent by the front-end AR glasses; the front-end AR glasses are also used for receiving maintenance task ending information sent when the front-end AR glasses detect that the maintenance task is ended;
the first sending module is connected with the first receiving module and is further configured to send the maintenance task end information to the MES, so as to trigger the MES to receive execution result data of the maintenance service, close the maintenance service worksheet and notify the front-end AR glasses.
9. Front-end AR glasses, characterized by comprising: the system comprises a second receiving module, a guiding module, a monitoring module and a second sending module;
the second receiving module is used for receiving the augmented reality AR scene virtual content sent by the maintenance cloud platform according to the maintenance service work order, wherein the AR scene virtual content is sent after the maintenance cloud platform receives the basic data of the maintenance service work order and the equipment to be maintained, which are created and sent by the manufacturing execution system MES, and obtains the corresponding AR virtual scene content according to the basic data of the equipment to be maintained;
the guiding module is connected with the second receiving module and is used for guiding the maintenance personnel to execute maintenance service according to the AR virtual scene content;
the monitoring module is connected with the guiding module and is used for monitoring and transmitting the process of executing the maintenance service by the maintenance personnel;
the second sending module is connected with the monitoring module and is used for sending maintenance task ending information to the maintenance cloud platform when the maintenance task is detected to be ended, sending execution result data of the maintenance service to the MES and receiving a notification of closing the maintenance service work order sent by the MES.
10. A maintenance business guidance and monitoring system, comprising: manufacturing an execution system MES, a maintenance cloud platform and front-end AR glasses;
the maintenance cloud platform is used for executing the maintenance business guiding and monitoring method according to any one of claims 1-5;
the front-end AR glasses are used for executing the maintenance business guiding and monitoring method according to claim 6 or 7.
CN202111347948.0A 2021-11-15 2021-11-15 Maintenance service guiding and monitoring method, cloud platform, AR glasses and system Active CN114063512B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111347948.0A CN114063512B (en) 2021-11-15 2021-11-15 Maintenance service guiding and monitoring method, cloud platform, AR glasses and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111347948.0A CN114063512B (en) 2021-11-15 2021-11-15 Maintenance service guiding and monitoring method, cloud platform, AR glasses and system

Publications (2)

Publication Number Publication Date
CN114063512A CN114063512A (en) 2022-02-18
CN114063512B true CN114063512B (en) 2023-09-19

Family

ID=80272487

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111347948.0A Active CN114063512B (en) 2021-11-15 2021-11-15 Maintenance service guiding and monitoring method, cloud platform, AR glasses and system

Country Status (1)

Country Link
CN (1) CN114063512B (en)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6057159A (en) * 1997-12-12 2000-05-02 Vertex Pharmaceuticals Incorporated Processes for identifying a solvent condition suitable for determining a biophysical property of a protein
FR2860170A1 (en) * 2003-09-29 2005-04-01 Atofina Res IONIC LIQUIDS FOR OLEFIN POLYMERIZATION CATALYSTS
TW201135381A (en) * 2010-04-13 2011-10-16 Univ Nat Cheng Kung Manufacturing execution system with virtual-metrology capabilities and manufacturing system including the same
CN106920291A (en) * 2017-02-27 2017-07-04 上海大学 Body structure surface based on augmented reality is patrolled and examined and analysis system
CN107330143A (en) * 2017-05-23 2017-11-07 深圳增强现实技术有限公司 Using AR check-up PCBA method and device
CN108415386A (en) * 2018-03-12 2018-08-17 范业鹏 Augmented reality system and its working method for intelligent workshop
CN109032348A (en) * 2018-06-26 2018-12-18 亮风台(上海)信息科技有限公司 Intelligence manufacture method and apparatus based on augmented reality
KR20190003214A (en) * 2017-06-30 2019-01-09 주식회사 한비세트론 Warehouse Management System using smart glasses having bar code or QR code scan function and capable of communicating with RFID reader
KR20190005006A (en) * 2017-07-05 2019-01-15 주식회사 한비세트론 Warehouse Management System using smart glasses having bar code or QR code scan function and capable of communicating with RFID reader, and providing location management based on Beacon ID
CN109947052A (en) * 2017-12-20 2019-06-28 中国科学院沈阳自动化研究所 A kind of virtual factory manufacturing execution system based on digitlization AR technology
CN111351789A (en) * 2020-04-07 2020-06-30 中国联合网络通信集团有限公司 Equipment detection/maintenance method, system and electronic device
WO2020212523A1 (en) * 2019-04-18 2020-10-22 Volkswagen Aktiengesellschaft System and method for the simulation of industrial processes
CN112085232A (en) * 2020-09-14 2020-12-15 武汉瑞莱保能源技术有限公司 Operation inspection system and method based on augmented reality technology
CN112292186A (en) * 2018-06-22 2021-01-29 3M创新有限公司 Personal protective equipment security system using contextual information from an industrial control system
CN112949457A (en) * 2021-02-26 2021-06-11 中国国际航空股份有限公司 Maintenance method, device and system based on augmented reality technology
KR20210121796A (en) * 2020-03-31 2021-10-08 엑스퍼트아이엔씨 주식회사 Solution for remote service and supporting of maintenance-repairment based in cloud employing smart glass
CN113497832A (en) * 2021-07-14 2021-10-12 中国联合网络通信集团有限公司 Remote maintenance system, method and server

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015006334A1 (en) * 2013-07-08 2015-01-15 Ops Solutions Llc Eyewear operational guide system and method
WO2018164852A1 (en) * 2017-02-22 2018-09-13 Mine One Gmbh Image reconstruction for virtual 3d
US10866631B2 (en) * 2016-11-09 2020-12-15 Rockwell Automation Technologies, Inc. Methods, systems, apparatuses, and techniques for employing augmented reality and virtual reality
US10499021B2 (en) * 2017-04-11 2019-12-03 Microsoft Technology Licensing, Llc Foveated MEMS scanning display
US11747800B2 (en) * 2017-05-25 2023-09-05 Johnson Controls Tyco IP Holdings LLP Model predictive maintenance system with automatic service work order generation
EP3841442A1 (en) * 2018-08-24 2021-06-30 Cytiva Sweden AB Monitoring system and method for biopharmaceutical products

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6057159A (en) * 1997-12-12 2000-05-02 Vertex Pharmaceuticals Incorporated Processes for identifying a solvent condition suitable for determining a biophysical property of a protein
FR2860170A1 (en) * 2003-09-29 2005-04-01 Atofina Res IONIC LIQUIDS FOR OLEFIN POLYMERIZATION CATALYSTS
TW201135381A (en) * 2010-04-13 2011-10-16 Univ Nat Cheng Kung Manufacturing execution system with virtual-metrology capabilities and manufacturing system including the same
CN106920291A (en) * 2017-02-27 2017-07-04 上海大学 Body structure surface based on augmented reality is patrolled and examined and analysis system
CN107330143A (en) * 2017-05-23 2017-11-07 深圳增强现实技术有限公司 Using AR check-up PCBA method and device
KR20190003214A (en) * 2017-06-30 2019-01-09 주식회사 한비세트론 Warehouse Management System using smart glasses having bar code or QR code scan function and capable of communicating with RFID reader
KR20190005006A (en) * 2017-07-05 2019-01-15 주식회사 한비세트론 Warehouse Management System using smart glasses having bar code or QR code scan function and capable of communicating with RFID reader, and providing location management based on Beacon ID
CN109947052A (en) * 2017-12-20 2019-06-28 中国科学院沈阳自动化研究所 A kind of virtual factory manufacturing execution system based on digitlization AR technology
CN108415386A (en) * 2018-03-12 2018-08-17 范业鹏 Augmented reality system and its working method for intelligent workshop
CN112292186A (en) * 2018-06-22 2021-01-29 3M创新有限公司 Personal protective equipment security system using contextual information from an industrial control system
CN109032348A (en) * 2018-06-26 2018-12-18 亮风台(上海)信息科技有限公司 Intelligence manufacture method and apparatus based on augmented reality
WO2020212523A1 (en) * 2019-04-18 2020-10-22 Volkswagen Aktiengesellschaft System and method for the simulation of industrial processes
KR20210121796A (en) * 2020-03-31 2021-10-08 엑스퍼트아이엔씨 주식회사 Solution for remote service and supporting of maintenance-repairment based in cloud employing smart glass
CN111351789A (en) * 2020-04-07 2020-06-30 中国联合网络通信集团有限公司 Equipment detection/maintenance method, system and electronic device
CN112085232A (en) * 2020-09-14 2020-12-15 武汉瑞莱保能源技术有限公司 Operation inspection system and method based on augmented reality technology
CN112949457A (en) * 2021-02-26 2021-06-11 中国国际航空股份有限公司 Maintenance method, device and system based on augmented reality technology
CN113497832A (en) * 2021-07-14 2021-10-12 中国联合网络通信集团有限公司 Remote maintenance system, method and server

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"黑科技"在设备维保系统中的应用;李科;;电气自动化(01);22-24 *
Efficient integration process of production data into augmented reality based maintenance of machine tools;Kallatsch, C 等;《PRODUCTION ENGINEERING-RESEARCH AND DEVELOPMENT》;第15卷(第3-4期);311-319 *
基于AR的钢铁只能运维单兵研究;任昱杰;《中国优秀硕士学位论文全文数据库工程科技I辑》(第1期);B023-158 *
面向空天地一体化场景的5G卫星双模终端需求及应用探讨;王文哲 等;《电信科学》;第38卷(第S1期);221-230 *

Also Published As

Publication number Publication date
CN114063512A (en) 2022-02-18

Similar Documents

Publication Publication Date Title
US10977868B2 (en) Remote collaboration methods and systems
Palmarini et al. A systematic review of augmented reality applications in maintenance
US10360729B2 (en) Methods and apparatus for augmented reality applications
US11417091B2 (en) Use of coordinated local user devices during a shared augmented reality session
EP3690815B1 (en) Method, medium and apparatus for automatically labeling target object within image
US20220044019A1 (en) Augmented reality smartglasses for use at cultural sites
CN102509330B (en) Application of virtual three-dimensional system of transformer substation on the basis of electric power geographic information system (GIS)
US20160284079A1 (en) System for inspecting objects using augmented reality
CN104504497A (en) BIM engineering information management system and method
CN103473720A (en) Shield tunnel construction management system based on BIM
CN107610238B (en) AR dynamic model system of power equipment and working method thereof
CN106200983A (en) A kind of combined with virtual reality and BIM realize the system of virtual reality scenario architectural design
CN110738895A (en) Criminal scene investigation real-scene VR comprehensive training system
CN111300416A (en) Modularized reconfigurable robot planning simulation method and system based on augmented reality
Benbelkacem et al. Augmented reality platform for collaborative E-maintenance systems
CN115661412A (en) Aero-engine auxiliary assembly system and method based on mixed reality
Salem et al. BIM and VR/AR technologies: From project development to lifecycle asset management
CN114063512B (en) Maintenance service guiding and monitoring method, cloud platform, AR glasses and system
CN110163580B (en) Creation method of multitask VR training scene, VR training system and storage medium
CN110660124A (en) Three-dimensional model modeling and editing system and method
Gimeno et al. An easy-to-use AR authoring tool for industrial applications
CN114360087A (en) Mining AR system of patrolling and examining
KR101958199B1 (en) Configuration management system by the 3D model, The system for maintenance of small and mediun-sized plant
CN116402497B (en) AR-based equipment overhaul and maintenance method and system
Sung et al. Assembly planning and motion study using virtual reality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant