CN113823129A - Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality - Google Patents
Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality Download PDFInfo
- Publication number
- CN113823129A CN113823129A CN202111076705.8A CN202111076705A CN113823129A CN 113823129 A CN113823129 A CN 113823129A CN 202111076705 A CN202111076705 A CN 202111076705A CN 113823129 A CN113823129 A CN 113823129A
- Authority
- CN
- China
- Prior art keywords
- disassembly
- assembly
- state information
- mixed reality
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 95
- 230000008569 process Effects 0.000 claims abstract description 46
- 238000005516 engineering process Methods 0.000 claims abstract description 18
- 238000004891 communication Methods 0.000 claims description 17
- 239000011521 glass Substances 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 230000000007 visual effect Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 abstract description 15
- 238000004422 calculation algorithm Methods 0.000 abstract description 3
- 238000012423 maintenance Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000002452 interceptive effect Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 238000009434 installation Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 235000019504 cigarettes Nutrition 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005553 drilling Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
Abstract
The invention provides a method and a device for guiding the disassembly and assembly of a U-turn wheel device based on mixed reality, wherein the method for guiding the disassembly and assembly comprises the following steps: acquiring current state information of target equipment; identifying target equipment and current operation state information thereof according to the obtained current state information; according to the method, disassembly and assembly guidance is carried out through a mixed reality technology according to current operation state information, the type of the target equipment and the current operation state information of the equipment can be judged through obtaining the current state information of the target equipment, and corresponding disassembly and assembly guidance is further carried out according to the current operation state information, namely, even if the current operation state information is an operation state with incomplete history, subsequent disassembly and assembly guidance can be carried out quickly and effectively, mixed reality guidance is achieved through machine vision and a space algorithm, operators can finish disassembly or assembly of the turning wheel independently, normalized knowledge is integrated into the disassembly and assembly operation process, and disassembly and assembly training guidance efficiency is improved.
Description
Technical Field
The invention belongs to the technical field of mechanical equipment maintenance, and particularly relates to a method and a device for guiding the disassembly and assembly of a U-turn wheel device based on mixed reality.
Background
The premise and basis for the use of mechanical equipment is the routine maintenance and upkeep of the equipment. During the use of the equipment in long-term and different environments (including severe environments), the stability, reliability and use benefit of the equipment are reduced to a considerable extent, and even the mechanical equipment loses the inherent basic performance and cannot operate normally. Along with the development of the industry, more and more advanced mechanical equipment is applied to daily production, the industry puts higher requirements on the disassembly, assembly and maintenance of the mechanical equipment, a three-stage method of training teaching, operation practice and periodic assessment is adopted on a training guidance process, the relevant standards of equipment maintenance are solidified, and the operation process of operators is standardized.
Virtual reality, augmented reality, and mixed reality are popular visual interaction technologies in recent years. The virtual reality technology enables a user to be immersed in the virtual three-dimensional scene through a head-mounted display by repeatedly carving the three-dimensional scene, and can simulate an environment which cannot be simulated easily in reality. The augmented reality technology identifies the identification object through machine vision, generates business logic, and then utilizes an equipment screen for interaction. The mixed reality is the upgrading of the virtual interactive content and the real space, the virtual interactive content and the real space are further integrated, the sensing equipment is used for scanning and reconstructing the real three-dimensional space, the machine vision is combined to identify a real object, the interactive content is placed in the real space, and a user directly interacts with the real space. Virtual interactive content generated by mixed reality, which actually exists in a real space position, can be controlled by a user.
The case that utilizes virtual reality to simulate mechanical dismouting maintenance is more for the number, through three-dimensional modeling scene reduction maintenance operation environment, utilizes interactive handle after the user wears the helmet, can experience the operation by the immersive, because carry out interactive operation through the handle, feels unable accurate simulation of accomplishing to the use of instrument, experiences the reduction degree limited, can only accomplish the accurate simulation of operating procedure and equipment structure.
The traditional disassembly, assembly and maintenance guidance mode is summarized into three stages. In the first stage, systematic standard training is carried out through classroom declaration, and the aim is to form a standardized thinking mode, to be familiar with the equipment structure and to know key points and difficulties in disassembly and assembly. And in the second stage, practical operation drilling is carried out, knowledge in a classroom is used for practice, and students are helped to understand the key points of the knowledge from practice. And the third stage of periodic assessment is performed, the disassembly and assembly knowledge of related equipment is consolidated, a memory habit is formed, and a standardized behavior mode is developed. The conventional method has the following limitations:
1. standard standard floor difficulty
The traditional training mode is prepared in an earlier stage by giving lessons in a centralized mode, the knowledge density is high, certain requirements are placed on the basic knowledge level of the audience, the knowledge absorption capacity of corresponding students is different from person to person, muscle memory is formed through continuous practice to cultivate standard habits, and the training is slow to take effect and the standard is difficult to fall on the ground.
2. High time cost
The traditional training mode adopts a three-section type training method due to the limitation of technical conditions, the essence of the traditional training method is that the training method is repeated continuously, audiences and trainees are required to perform disassembly, assembly and simulation and practical operation repeatedly to achieve the training effect, and a large amount of time cost is consumed in the training period.
3. Unstable training effect
Because the time cost is higher, the receptivity of students is different, although the process and the method are standardized flows, the final effect has stronger dependence on the practical time, the more times of repetition, the better the effect, and the longer the time, the better the effect.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a turning wheel equipment dismounting guidance method and device based on mixed reality, mixed reality guidance is realized through machine vision and a space algorithm, a user can independently complete the dismounting of the turning wheel equipment through the system prompt, normalized knowledge is integrated into the dismounting operation process, and the dismounting training guidance efficiency is improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
a mixed reality-based turning wheel equipment dismounting guidance method comprises the following steps:
acquiring current state information of target equipment;
identifying the target equipment and the current operation state information thereof according to the obtained current state information;
and carrying out disassembly and assembly guidance according to the current operation state information and by a mixed reality technology.
Preferably, before the step of obtaining the current state information, the method further includes: and determining the step of disassembling or assembling the target equipment.
Preferably, before the step of confirming the detachment or installation of the target device, the method further comprises: and confirming the identity validity of the operator.
Preferably, identifying the target device comprises:
and identifying the model information of the target equipment by comparing the equipment appearance information in the current state information with prestored appearance reference information, wherein the appearance reference information is obtained by performing high-precision industrial modeling in a laser scanning mode.
Preferably, identifying the current operating state information of the target device comprises:
comparing the current state information with a plurality of pieces of operation state reference information stored in advance so as to judge the validity of the current state information;
and further when the judgment result is valid, obtaining the matched current operation state information, wherein the current operation state information is used for representing the disassembly or assembly progress of the target equipment.
Preferably, the disassembly and assembly guide comprises at least one of the following guide contents: and playing voice prompt information, playing dismounting video and playing dismounting picture information with position prompt.
Preferably, the disassembly and assembly guidance method further comprises the following steps in the disassembly and assembly guidance process:
collecting current disassembly and assembly process data in real time;
and sending the disassembly and assembly process data to a cloud server in real time for storage.
The utility model provides a turn round wheel equipment dismouting guides device based on mixed reality, and this dismouting guides device includes:
the visual acquisition module is used for acquiring the current state information of the target equipment;
the identification processing module identifies the target equipment and the current operation state information thereof according to the current state information;
and the guidance display module is used for carrying out dismounting guidance according to the current operation state information and by adopting a mixed reality technology.
Preferably, the disassembly and assembly guide device is provided as a wearable glasses device so as to facilitate the disassembly and assembly operation of an operator.
Preferably, the disassembly and assembly instruction device further comprises a communication module, and the communication module is in communication connection with a cloud server through a communication network, so that data transmission is performed between the communication module and the cloud server, and the cloud server can perform data processing and storage operations conveniently.
Compared with the prior art, the invention has the beneficial effects that: according to the method and the device for guiding the disassembly and assembly of the U-turn wheel equipment based on the mixed reality, the type of the target equipment and the current operation state information of the equipment can be judged by acquiring the current state information of the target equipment, and corresponding disassembly and assembly guidance is further carried out according to the current operation state information, namely, even if the current operation state information is an operation state with incomplete history, subsequent disassembly and assembly guidance can be rapidly and effectively carried out, so that an operator can independently complete the disassembly or assembly of the U-turn wheel, and does not need to carry out the disassembly and assembly operation from the beginning; compared with the prior dismounting guidance which only provides complete dismounting video guidance to enable operators to watch, study or simulate, the invention can effectively improve the guidance efficiency and ensure that the operators can efficiently finish the dismounting operation.
Drawings
Fig. 1 is a flowchart of a method for guiding the disassembly and assembly of a U-turn wheel device based on mixed reality in an embodiment of the present invention.
Fig. 2 is a process diagram of instant positioning and spatial reconstruction of a method for guiding the disassembly and assembly of a U-wheel device based on mixed reality in an embodiment of the present invention.
Fig. 3 is a flowchart of a method for guiding the disassembly and assembly of the U-turn wheel device based on mixed reality according to another embodiment of the present invention.
Fig. 4 is a block diagram of a mixed reality-based turning wheel device dismounting guidance device in an embodiment of the present invention.
Fig. 5 is a stereoscopic vision establishing process diagram of the mixed reality-based turning wheel device dismounting guide device in the embodiment of the invention.
Detailed Description
The invention will be further described with reference to examples of embodiments shown in the drawings.
As shown in fig. 1, the present embodiment provides a method for guiding the disassembly and assembly of a turning wheel device based on mixed reality, which guides the disassembly or assembly of the turning wheel by using a mixed reality technology, and in the present embodiment, the turning wheel is a component installed in a cigarette making machine device. The disassembly and assembly guiding method specifically comprises the following steps:
and step S1, acquiring the current state information of the target device. In this embodiment, the target device is a cigarette maker device for mounting the turning wheel. In this step, as a specific implementation manner, the current state information of the target device may be obtained through an image acquisition manner (such as taking a picture, etc.).
And step S2, identifying the target device and the current operation state information thereof according to the obtained current state information.
Specifically, in step S2, the process of identifying the target device according to the obtained current state information of the target device includes:
and comparing the device appearance information in the current state information with the pre-stored appearance reference information so as to identify the model information of the target device. The method specifically comprises the following steps: firstly, shooting a picture, detecting the natural features of the image, matching the current video image with a reference image with a specified appearance, and judging whether the current scene image is the same as the model image according to the matching result. If not, the identification process continues.
The appearance reference information is obtained by performing high-precision industrial modeling in a laser scanning mode and is pre-stored in a designated device. As the smoke machine equipment has the characteristics of high precision and high density, high-precision industrial modeling is carried out by adopting a laser scanning mode of 0.1MM level in a modeling stage, and twin equipment with part level equal-size full reduction is topologically arranged on the basis.
In addition, the process of identifying the current operation state information of the target device according to the obtained current state information of the target device includes:
and comparing the current state information with a plurality of pieces of operation state reference information stored in advance so as to judge the validity of the current state information. The validity here means whether the current operating state corresponding to the current target device is any valid state information in the process of detachment or installation. The method specifically comprises the following steps: and finding the 3D point coordinates of the object model corresponding to the current image according to the mapping table, recovering the pose matrix of the current image frame according to the projection matrix and the known internal reference matrix, and then performing geometric verification and back projection error analysis to judge whether the image frame is in an effective flow state.
And further acquiring the matched current operation state information when the judgment result is valid. That is, the process state corresponding to the disassembly or assembly corresponding to the operation state reference information obtained by matching is used as the current operation state information of the target device, and the disassembly or assembly progress of the target device is represented by the current operation state information. Namely, the guidance method of the present embodiment deeply combines with the SOP (standard operation program) flow, performs dynamic management and loading of operation steps, and performs step optimization and management scoring through data analysis. In addition, after the effective flow state is judged, a coordinate system of the virtual object can be defined, and the three-dimensional virtual object is overlaid for rendering. And then, starting to enter a tracking stage to track and calculate a new pose matrix, and when the tracked points influence the precision of calculating the pose matrix, re-identifying the appearance of the equipment and the effective flow state.
In another embodiment, in order to improve the accuracy of the current operation state information identification, the process of identifying the current operation state information may include: the method comprises the steps of identifying and comparing the current state information of target equipment with a pre-stored equipment model in the depth direction, identifying and comparing the current state information with pre-stored operation state reference information, and improving the identification accuracy of the current operation state information through two-level identification and comparison.
And step S3, performing disassembly and assembly guidance through a mixed reality technology according to the current operation state information.
Specifically, in step S3, the specific manner of performing the detachment/attachment instruction includes at least one of the following instructions: and playing voice prompt information, playing dismounting video and playing dismounting picture information with position prompt. The disassembly and assembly that matches is guided through the mixed reality technology, the disassembly and assembly that can efficiently guide the operator to disassemble or assemble, the operation is realized while the disassembly and assembly is guided, and the disassembly and assembly operation can be quickly and accurately carried out by the operator through the position prompt in the form of videos or pictures.
In step S3, guidance of subsequent steps is further performed based on the specific step of SOP located in step S2, and in the guidance process, the guidance method has a function of tool identification, establishes a complete tool placement specification according to the SOP flow, and identifies the model of the tool and gives detailed guidance to tool selection through MR (mixed reality technology). In the guidance process, the position guidance of the tool is realized by an instant positioning and space reconstruction technology, which specifically comprises the following steps:
as shown in fig. 2, a spatial relationship is established through a corresponding camera, other sensors and a laser radar, specifically, an environment sensing camera is adopted to capture a real environment image and extract feature points from the real environment image, then the feature points of adjacent images are matched, and a motion vector is obtained through difference calculation, so that the motion to be executed by an operator is determined; and obtaining space depth information through a depth algorithm in stereoscopic vision, and re-engraving the three-dimensional space in a computer.
Through the guidance method of the steps S1 to S3, the model of the target device and the current operating state information of the device can be determined by obtaining the current state information of the target device, and further, corresponding disassembly and assembly guidance is performed according to the current operating state information, that is, even if the current operating state information is an operating state with incomplete history, subsequent disassembly and assembly guidance can be performed quickly and effectively, so that an operator can finish the disassembly and assembly of the U-turn wheel independently without performing the disassembly and assembly operation from the beginning. Namely, through discerning equipment outward appearance model, rebuild virtual maintenance scene, with dismouting operation suggestion content location to corresponding spatial position in, when the user carries out the dismouting operation, can look over the dismouting suggestion simultaneously, including contents such as part location, equipment structure, dismouting step. The time utilization efficiency in the equipment dismouting training link is improved, the instruction content has been merged into in practice, and the beginner can independently accomplish equipment dismouting through the instrument. Compared with the prior dismounting guidance which only provides complete dismounting video guidance to enable operators to watch, study or simulate, the invention can effectively improve the guidance efficiency and ensure that the operators can efficiently finish the dismounting operation.
In addition, the disassembly and assembly guidance method of the embodiment performs dynamic management and loading of the operation steps by an asynchronous loading scheme with optimized performance and deep combination of the SOP flow, and performs step optimization and management scoring by data analysis and comparison.
In addition, the disassembly and assembly guidance method of the present embodiment further combines an offline rendering technology, performs step guidance using high-quality visual information such as offline rendered video pictures, and performs virtual-real combination of information using low-quality positions based on spatial positioning with high accuracy, and the two are complementary to each other. The method solves the problems that the traditional MR adopts real-time rendering on the displayed content, is mainly limited by the performance of the equipment, and has poor effect, low frame rate and the like on the fine performance of a three-dimensional model.
In another embodiment, as shown in fig. 3, in order to ensure the validity of the detaching or attaching operation performed on the target device, the method for guiding the detaching and attaching of the u-turn wheel device based on the mixed reality may further include the following steps before step S1: and confirming the identity validity of the operator. That is to say, only through the operating personnel of authentication can carry out the turning round to target equipment and dismantle or the installation operation, ensure illegal personnel's maloperation etc. improve equipment security. Specifically, the manner of authenticating the operator includes, but is not limited to, the following ways: user information login and verification modes, fingerprint identification modes, iris identification modes and the like.
Further, as shown in fig. 3, after the identity validity of the operator is confirmed, the method for guiding the disassembly and assembly of the U-turn wheel device based on mixed reality may further include the following steps: and determining the step of disassembling or assembling the target equipment. Therefore, the target equipment to be detached or installed is effectively distinguished, and the overall guidance efficiency is improved.
In another embodiment, in order to implement the traceability query of the disassembly or assembly operation performed by the operator and the record of the disassembly or assembly information of the target equipment, the method for guiding the disassembly and assembly of the U-turn wheel equipment based on the mixed reality further comprises the following steps in the process of guiding the disassembly and assembly: collecting current disassembly and assembly process data in real time; and the disassembly and assembly process data are sent to the cloud server in real time to be stored. Here, the whole course record of the disassembly or assembly process of the operator is realized by collecting the disassembly and assembly process data in real time, and the disassembly and assembly process data collection can be specifically carried out in a mode of shooting videos or pictures at a preset frequency. In addition, the acquired disassembly and assembly process data needs to be stored, specifically, the disassembly and assembly process data can be directly stored in the guiding device body, and the acquired disassembly and assembly process data can be sent to the cloud server for storage, so that the storage space of the guiding device body is saved, and meanwhile, managers can conveniently access the cloud server at any time and any place to inquire the disassembly and assembly records of the target equipment. Therefore, the deep integration of the service is realized by deeply combining the equipment operation and maintenance system and the MES system and confirming the work order of the maintenance result and the diagnosis result.
Based on the foregoing process, the method for guiding the disassembly and assembly of the turning wheel device based on mixed reality further provides a device 100 for guiding the disassembly and assembly of the turning wheel device based on mixed reality, where the device 100 includes: a vision acquisition module 10, a recognition processing module 20, a guidance display module 30, and a control module 40.
The vision acquisition module 10 is used for acquiring current status information of the target device.
The identification processing module 20 identifies the target device and the current operating status information thereof according to the current status information, and for a specific identification process, reference is made to the detailed process of step S2 in the foregoing embodiment, which is not described herein again.
The guidance display module 30 performs dismounting guidance according to the current operation state information and by using a mixed reality technology, and for a specific guidance manner, reference is made to the detailed content of step S3 in the foregoing embodiment, which is not described herein again.
The control module 40 stores therein a control program for controlling the vision capturing module 10, the recognition processing module 20, and the guidance display module 30 to perform specific steps.
According to the specific structure of the turning wheel equipment dismounting guidance device 100 based on mixed reality, the current state information of the target equipment is acquired through the vision acquisition module 10, the model of the target equipment and the current operating state information of the equipment can be judged through the identification processing module 20, and the guidance display module 30 further carries out corresponding dismounting guidance according to the current operating state information, namely, even if the current operating state information is an operating state with incomplete history, subsequent dismounting guidance can be carried out quickly and effectively without carrying out dismounting operation from the beginning by an operator; compared with the prior dismounting guidance which only provides complete dismounting video guidance to enable operators to watch, study or simulate, the invention can effectively improve the guidance efficiency and ensure that the operators can efficiently finish the dismounting operation.
In another embodiment, in order to facilitate the operator to be instructed to perform the detachment or attachment operation, the detachment/attachment instructing device 100 may be configured as a wearable glasses device, so that only the detachment/attachment instructing device 100 needs to be worn at the eye position, thereby facilitating the viewing of the instructing information while performing the detachment or attachment operation.
Based on be set up to the dismouting of wearable glasses equipment and guide device 100, in-service use, can correct the camera on the glasses equipment according to scene demand at that time to thereby follow-up can throw out accurate virtual picture and accurately dismantle or install and guide, specifically as follows:
as shown in fig. 5, the glasses device is provided with a plurality of cameras to facilitate establishment of stereoscopic vision, and the overall stereoscopic vision is processed by performing steps of camera correction, image alignment, left-right matching, visual image, depth image and synchronous positioning and space establishment on all the cameras, so that a picture projected by the glasses end can be synthesized into a space according to a real perspective relationship, and a physical shielding relationship of the real space is reserved.
Based on the disassembly and assembly guide apparatus 100 provided as the wearable eyeglass device, in order to ensure the validity of the disassembly or assembly operation performed on the target device, the identity validity of the operator may be confirmed first after the operator wears the eyeglass device. That is to say, only through the operating personnel of authentication can carry out the turning round to target equipment and dismantle or the installation operation, ensure illegal personnel's maloperation etc. improve equipment security. Specifically, the mode of identity authentication of the operator can adopt an iris recognition mode, and the identity can be quickly confirmed after the device is worn. Of course, other authentication methods may be used, such as a method of logging in and authenticating user information before wearing the glasses device, or a fingerprint recognition method.
Further, after the identity validity of the operator is confirmed, the operator can be made to determine the requirement for detaching or installing the target device. Therefore, the target equipment to be detached or installed is effectively distinguished, and the overall guidance efficiency is improved.
In another embodiment, in order to perform a retroactive query of the disassembly or assembly operation performed by the operator and record the disassembly or assembly information of the target equipment, the disassembly and assembly instruction device 100 can also collect the current disassembly and assembly process data in real time and store the disassembly and assembly process data. Specifically, the disassembly and assembly instruction device 100 further includes a communication module 50, and the communication module 50 is in communication connection with the cloud server through a communication network, so as to perform data transmission with the cloud server, so as to facilitate data processing and storage operations of the cloud server. As a specific embodiment, the communication module 50 may adopt any existing communication means, such as a mobile communication technology such as 2G/3G/4G/5G/6G, a wireless communication technology such as WIFI and ZIGBEE, a bluetooth communication technology, and the like.
Firstly, the visual acquisition module 10 acquires the disassembly and assembly process data in real time, so as to realize the whole-process recording of the disassembly or assembly process of an operator, and specifically, the disassembly and assembly process data acquisition can be performed in a mode of shooting videos or pictures at a preset frequency. In addition, the collected disassembly and assembly process data are sent to the cloud server to be stored, so that the storage space of the guiding device body is saved, and meanwhile, managers can conveniently visit the cloud server at any time and any place to inquire disassembly and assembly records of the target equipment. Here, it is not excluded that the collected disassembly and assembly process data can be saved in the glasses device body as long as the actual requirements are met.
Based on this dismouting of wearable glasses equipment form guides device, the specific guidance process that operating personnel carried out is as follows:
1. the operator wears the eyewear equipment for assembly and disassembly instruction and starts the power supply. In this embodiment, the glasses device may be holtens 2 glasses. HoloLens is a mixed reality headset produced by Microsoft and provides basic functions for a user to interact with a real space.
2. The equipment starts iris recognition so as to authenticate the identity of an operator, and starts an application program of disassembly and assembly guidance after the authentication is successful.
3. And identifying the equipment, starting model matching when the turning wheel is identified, and overlapping the virtual model and the physical model for identification.
4. Whether the disassembly process or the assembly process is to be performed is confirmed by the operator.
5. Judging the current equipment state, and positioning to the step corresponding to the SOP process, namely judging the specific step of dismounting or mounting the current equipment state, namely possibly the step in the middle, namely explaining that the previous dismounting or mounting process of the equipment is not completed, so as to execute guidance of the subsequent dismounting or mounting step according to the current equipment state. In the step, the depth camera of hololens can be compared with the established model, the visual camera is compared with the previously pre-stored state reference picture, the state of the current equipment can be accurately judged by comparing the two layers, and the problem that the judgment of a pure visual camera is inaccurate due to the fact that the visual angles of the currently shot picture and the state reference picture are inconsistent is mainly solved.
6. The method has the advantages that the real space is subjected to three-dimensional reconstruction, the three-dimensional model of the equipment is accurately positioned, the virtual scene and the real scene are fused, the UI is prompted to have the correct spatial position relation, and good interaction experience is provided.
7. The method for starting to disassemble or assemble the electronic device specifically comprises the following steps: and prompting the tool to be adopted and prompting the next operation step until all the disassembly or assembly steps are completed.
8. And recording the disassembly or assembly operation process while disassembling or assembling, and finally uploading the work order information and the disassembly and assembly record data.
The protective scope of the present invention is not limited to the above-described embodiments, and it is apparent that various modifications and variations can be made to the present invention by those skilled in the art without departing from the scope and spirit of the present invention. It is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Claims (10)
1. A mixed reality-based turning wheel equipment dismounting guidance method is characterized by comprising the following steps:
acquiring current state information of target equipment;
identifying the target equipment and the current operation state information thereof according to the obtained current state information;
and carrying out disassembly and assembly guidance according to the current operation state information and by a mixed reality technology.
2. The method for guiding the disassembly and assembly of the mixed reality-based U-turn wheel device according to claim 1, further comprising, before the step of obtaining the current state information: and determining the step of disassembling or assembling the target equipment.
3. The method for guiding the disassembly and assembly of the mixed reality-based U-turn wheel device according to claim 2, further comprising, before the step of confirming the disassembly or assembly of the target device: and confirming the identity validity of the operator.
4. The mixed reality-based turning wheel equipment dismounting and mounting guiding method according to any one of claims 1-3, wherein identifying the target equipment comprises:
and identifying the model information of the target equipment by comparing the equipment appearance information in the current state information with prestored appearance reference information, wherein the appearance reference information is obtained by performing high-precision industrial modeling in a laser scanning mode.
5. The mixed reality-based turning wheel device dismounting guidance method according to claim 4, wherein identifying the current operating state information of the target device comprises:
comparing the current state information with a plurality of pieces of operation state reference information stored in advance so as to judge the validity of the current state information;
and further when the judgment result is valid, obtaining the matched current operation state information, wherein the current operation state information is used for representing the disassembly or assembly progress of the target equipment.
6. The method for guiding the disassembly and assembly of the mixed reality-based U-turn wheel device according to claim 5, wherein the disassembly and assembly guide comprises at least one of the following guide contents: and playing voice prompt information, playing dismounting video and playing dismounting picture information with position prompt.
7. The mixed reality-based turning wheel equipment dismounting and mounting guiding method according to claim 6, wherein the dismounting and mounting guiding method further comprises the following steps in the dismounting and mounting guiding process:
collecting current disassembly and assembly process data in real time;
and sending the disassembly and assembly process data to a cloud server in real time for storage.
8. The utility model provides a device is guided in turn round equipment dismouting based on mixed reality which characterized in that, this dismouting guides the device to include:
the visual acquisition module is used for acquiring the current state information of the target equipment;
the identification processing module identifies the target equipment and the current operation state information thereof according to the current state information;
and the guidance display module is used for carrying out dismounting guidance according to the current operation state information and by adopting a mixed reality technology.
9. The mixed reality-based turning wheel device disassembly and assembly guidance device of claim 8, wherein the disassembly and assembly guidance device is configured as a wearable glasses device to facilitate disassembly and assembly operations by an operator.
10. The mixed reality-based turning wheel equipment dismounting and mounting guiding device according to claim 8, further comprising a communication module, wherein the communication module is in communication connection with a cloud server through a communication network, so as to perform data transmission with the cloud server, so that the cloud server performs data processing and storage operations.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111076705.8A CN113823129A (en) | 2021-09-14 | 2021-09-14 | Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111076705.8A CN113823129A (en) | 2021-09-14 | 2021-09-14 | Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113823129A true CN113823129A (en) | 2021-12-21 |
Family
ID=78922399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111076705.8A Pending CN113823129A (en) | 2021-09-14 | 2021-09-14 | Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113823129A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130044130A1 (en) * | 2011-08-17 | 2013-02-21 | Kevin A. Geisner | Providing contextual personal information by a mixed reality device |
| CN104484523A (en) * | 2014-12-12 | 2015-04-01 | 西安交通大学 | Equipment and method for realizing augmented reality induced maintenance system |
| CN105741643A (en) * | 2016-04-20 | 2016-07-06 | 北京百度网讯科技有限公司 | Method, device and system for realizing automatic driving skill training |
| CN108806364A (en) * | 2018-06-08 | 2018-11-13 | 潍坊护理职业学院 | A kind of wisdom simulation type nursing teaching system and construction method based on double net interconnections |
| CN110322024A (en) * | 2019-06-21 | 2019-10-11 | 上海翊视皓瞳信息科技有限公司 | A kind of job guide system and method based on wearable device |
| CN110491233A (en) * | 2019-08-23 | 2019-11-22 | 北京枭龙科技有限公司 | A kind of new-energy automobile disassembly system and method based on mixed reality |
| CN111915943A (en) * | 2020-08-18 | 2020-11-10 | 营口巨成教学科技开发有限公司 | Unmanned public nursing and first-aid training system and training method |
| CN111951636A (en) * | 2020-08-20 | 2020-11-17 | 广州驰创科技有限公司 | Training operation equipment for big data interaction-based teaching |
-
2021
- 2021-09-14 CN CN202111076705.8A patent/CN113823129A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130044130A1 (en) * | 2011-08-17 | 2013-02-21 | Kevin A. Geisner | Providing contextual personal information by a mixed reality device |
| CN104484523A (en) * | 2014-12-12 | 2015-04-01 | 西安交通大学 | Equipment and method for realizing augmented reality induced maintenance system |
| CN105741643A (en) * | 2016-04-20 | 2016-07-06 | 北京百度网讯科技有限公司 | Method, device and system for realizing automatic driving skill training |
| CN108806364A (en) * | 2018-06-08 | 2018-11-13 | 潍坊护理职业学院 | A kind of wisdom simulation type nursing teaching system and construction method based on double net interconnections |
| CN110322024A (en) * | 2019-06-21 | 2019-10-11 | 上海翊视皓瞳信息科技有限公司 | A kind of job guide system and method based on wearable device |
| CN110491233A (en) * | 2019-08-23 | 2019-11-22 | 北京枭龙科技有限公司 | A kind of new-energy automobile disassembly system and method based on mixed reality |
| CN111915943A (en) * | 2020-08-18 | 2020-11-10 | 营口巨成教学科技开发有限公司 | Unmanned public nursing and first-aid training system and training method |
| CN111951636A (en) * | 2020-08-20 | 2020-11-17 | 广州驰创科技有限公司 | Training operation equipment for big data interaction-based teaching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8374395B2 (en) | Augmented reality method and devices using a real time automatic tracking of marker-free textured planar geometrical objects in a video stream | |
| CN111526118A (en) | Remote operation guiding system and method based on mixed reality | |
| CN100426198C (en) | Calibration method and apparatus | |
| CN105426827A (en) | Living body verification method, device and system | |
| CN105635669B (en) | The movement comparison system and method for data and real scene shooting video are captured based on three-dimensional motion | |
| KR101399655B1 (en) | Somatotype swing analysis apparatus, system and method for managing golf lesson thereof | |
| JP2017191490A (en) | Skill transmission system and method | |
| US20110109628A1 (en) | Method for producing an effect on virtual objects | |
| CN103354761A (en) | Virtual golf simulation apparatus and method | |
| US11945125B2 (en) | Auxiliary photographing device for dyskinesia analysis, and control method and apparatus for auxiliary photographing device for dyskinesia analysis | |
| CN102801994A (en) | Physical image information fusion device and method | |
| CN112562056A (en) | Control method, device, medium and equipment for virtual light in virtual studio | |
| CN113627005B (en) | Intelligent vision monitoring method | |
| CN117333644A (en) | Virtual reality display picture generation method, device, equipment and medium | |
| CN112288876A (en) | Long-distance AR identification server and system | |
| Lugrin et al. | Usability benchmarks for motion tracking systems | |
| CN111881807A (en) | VR conference control system and method based on face modeling and expression tracking | |
| CN113823129A (en) | Method and device for guiding disassembly and assembly of turning wheel equipment based on mixed reality | |
| US20230162458A1 (en) | Information processing apparatus, information processing method, and program | |
| CN113836722B (en) | Assembly state perception method based on wearable device, scene and digital-analog registration | |
| CN111913572B (en) | Human-computer interaction system and method for user labor learning | |
| CN116402497B (en) | AR-based equipment overhaul and maintenance method and system | |
| CN115576434B (en) | AR-based equipment operation guiding method, AR-based equipment operation guiding system and related equipment | |
| JP6729952B1 (en) | Railway simulator system, display control method and program | |
| US11562543B2 (en) | Method and system for augmented reality visualisation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |