CN117150587A - Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality - Google Patents
Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality Download PDFInfo
- Publication number
- CN117150587A CN117150587A CN202310992082.1A CN202310992082A CN117150587A CN 117150587 A CN117150587 A CN 117150587A CN 202310992082 A CN202310992082 A CN 202310992082A CN 117150587 A CN117150587 A CN 117150587A
- Authority
- CN
- China
- Prior art keywords
- additive manufacturing
- arc fuse
- information
- module
- fuse additive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 145
- 239000000654 additive Substances 0.000 title claims abstract description 139
- 230000000996 additive effect Effects 0.000 title claims abstract description 139
- 238000010276 construction Methods 0.000 title claims abstract description 8
- 238000003466 welding Methods 0.000 claims abstract description 64
- 238000000151 deposition Methods 0.000 claims abstract description 30
- 230000008021 deposition Effects 0.000 claims abstract description 28
- 238000004891 communication Methods 0.000 claims abstract description 16
- 230000000694 effects Effects 0.000 claims abstract description 9
- 239000011324 bead Substances 0.000 claims abstract description 7
- 238000000034 method Methods 0.000 claims description 42
- 238000007639 printing Methods 0.000 claims description 40
- 238000012545 processing Methods 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 25
- 230000033001 locomotion Effects 0.000 claims description 18
- 239000000463 material Substances 0.000 claims description 9
- 230000007547 defect Effects 0.000 claims description 7
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000005137 deposition process Methods 0.000 claims description 4
- 230000003993 interaction Effects 0.000 abstract description 9
- 238000011161 development Methods 0.000 abstract description 3
- 230000001737 promoting effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 11
- 230000002452 interceptive effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000004927 fusion Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 239000002184 metal Substances 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008846 dynamic interplay Effects 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/18—Details relating to CAD techniques using virtual or augmented reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2113/00—Details relating to the application field
- G06F2113/10—Additive manufacturing, e.g. 3D printing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Evolutionary Computation (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a mixed reality-based arc fuse additive manufacturing digital twin system construction method and system, comprising the following steps: acquiring real model scene information of arc fuse additive manufacturing; establishing a virtual model; acquiring an arc fuse additive manufacturing physical entity data signal information chain, and establishing communication between the arc fuse additive manufacturing virtual model through a UDP communication protocol; displaying a welding bead deposition morphology effect of arc fuse additive manufacturing welding bead by using a welding piece deposition display module; the model operation module is used for enabling a user to interact with the system through different gestures; and importing the arc fuse additive manufacturing virtual model, equipment information and operation information into a mixed reality scene, and visually superposing the mixed reality display model and the three-dimensional configuration. The application intuitively guides the manufacturing process of the arc fuse additive, has strong man-machine interaction and convenient operation, is not influenced by external factors, and plays a great role in promoting the development of the arc fuse additive manufacturing.
Description
Technical Field
The application relates to the technical field of intelligent engineering manufacturing, in particular to a method and a system for constructing an arc fuse additive manufacturing digital twin system based on mixed reality.
Background
Arc fuse additive manufacturing is an important development direction of additive manufacturing, and is a process for stacking and forming molten liquid drops layer by operating an arc melting metal wire, a computer planning deposition path, a numerical control machine tool or a robot. The method has the characteristics of high processing efficiency, short manufacturing period, excellent product performance and the like, and is suitable for the rapid integral forming manufacture of large-size metal parts. However, in order to obtain optimal process conditions during additive manufacturing, a great deal of experimentation is required to optimize parameters in a given machine and process, and producing components with good structural and mechanical properties by experimental methods would take a great deal of time and cost and expensive economic costs.
Digital Twinning (DT) is defined as a key way to realize bidirectional mapping, dynamic interaction and real-time connection between virtual and real, and can map the attributes, structures, states, performances, functions and behaviors of physical entities and systems to a virtual world to form a high-fidelity dynamic multidimensional, multiscale and multiscale physical quantity model, so that an effective means is provided for observing the physical world, knowing the physical world, understanding the physical world, controlling the physical world and modifying the physical world. They are potential solutions that help overcome many of the problems in manufacturing to improve part quality and shorten the time of acceptable products.
The mixed reality technology is a new technology based on the field of computer graphics, and is combined on the basis of virtual reality and mixed reality by taking the virtual reality technology as a basis, so that a new environment is constructed, and real world and virtual world objects are presented and interacted together. It reflects both physical information of the real world and digital information of the virtual world. An interactive feedback information loop among the virtual world, the real world and the user is constructed, and enough live feeling, mixed feeling and technological feeling are given to the user in the real world.
Aiming at large-scale metal structures, the arc additive manufacturing currently adopts one-dimensional path planning geometric continuous forming difference, the processing process lacks a forming real-time monitoring and defect in-situ identification prediction means, the structural deformation is large, and the comprehensive performance is poor. The method for planning the intelligent path of the complex structure, the deformation regulation and prediction in the manufacturing process, the intelligent monitoring, identification and prediction of forming precision and defects, and a great deal of time cost and expensive economic cost are consumed when parts with good structure and mechanical properties are produced based on an additive manufacturing method are needed to break through the performance cross-scale uniform regulation and prediction of metal parts.
Patent document CN114564880a (application number: CN202210094566. X) discloses a method for constructing a digital twin module in an additive manufacturing process, comprising the steps of 1: performing an additive manufacturing experiment, and collecting the size of a molten pool under corresponding technological parameters as experimental data to form a physical entity in the additive manufacturing process; step 2: establishing a multi-physical field coupling mechanism model in the additive manufacturing process, verifying the confidence coefficient of the mechanism model, performing digital platform virtual printing based on the high-confidence coefficient mechanism model, and obtaining a printing result and data to form a digital twin body in the additive manufacturing physical process; step 3: forward and backward prediction is carried out on the technological parameters and the deep pool size based on machine learning, so that virtual-real fusion and data intercommunication of the additive manufacturing physical entity and the digital twin body are realized; step 4: the obtained optimal machine learning model is used for new data prediction of experiments and mechanism models, so that additive manufacturing experiments, fusion of a high-confidence mechanism model and machine learning, and efficient and accurate prediction of corresponding technological parameters and molten pool sizes are realized. However, this patent does not solve the technical problems existing at present.
Disclosure of Invention
Aiming at the defects in the prior art, the application aims to provide a method and a system for constructing a digital twin system for arc fuse additive manufacturing based on mixed reality.
The application provides a mixed reality-based arc fuse additive manufacturing digital twin system construction method, which comprises the following steps:
step S1: acquiring real model scene information of arc fuse additive manufacturing;
step S2: establishing an arc fuse additive manufacturing virtual model by adopting 3 Dmax;
step S3: the mixed reality display module is used for comparing the arc fuse additive manufacturing real model with the arc fuse additive manufacturing virtual model in real time to obtain an arc fuse additive manufacturing physical entity data signal information chain;
step S4: establishing communication between an arc fuse additive manufacturing physical entity data signal information chain and an arc fuse additive manufacturing virtual model through a UDP communication protocol;
step S5: displaying arc fuse additive manufacturing welding bead deposition morphology effects in a virtual model through different welding process parameters by using a welding piece deposition display module;
step S6: the method comprises the steps of using a model operation module for a user to interact with the system through different gestures so as to observe an arc fuse additive manufacturing deposition process;
step S7: importing the arc fuse additive manufacturing virtual model, equipment information and operation information into a mixed reality scene, visually superposing the mixed reality display model and the three-dimensional configuration, and monitoring and printing by an operator according to the operation information until the error value of the three-dimensional configuration and the arc fuse additive manufacturing virtual model is within a preset range;
the mixed reality display module displays three-dimensional configuration mixed reality projection of the arc fuse additive manufacturing virtual model through a Hollolens 2 display device.
Preferably, the step S3 includes:
the signal information chain comprises the current equipment position state, the deposition progress, the current processing and deposition technological parameters, the processing progress and the geometric errors;
step S31: identifying and comparing the positions of a mechanical arm and a positioner of the equipment, and acquiring matched position characteristic information of the mechanical arm and the positioner;
step S32: analyzing the image information of the processing progress to obtain the characteristic information of the image, wherein the characteristic information comprises one or more of the completion degree of printing the workpiece, the temperature of the workpiece and the number of printing layers;
step S33: and analyzing the image information of the technological parameters to obtain characteristic information of the image, wherein the characteristic information comprises one or more of welding speed, wire feeding speed and welding current.
Preferably, the step S5 includes:
step S51: analyzing the actual printing processing process information of the arc fuse additive manufacturing to obtain characteristic information, wherein the characteristic information comprises one or more of material types, welding speed, wire feeding speed, welding current and shielding gas flow;
step S52: analyzing the obtained characteristic information, and processing the characteristic information by a machine learning method to obtain correct molten drop and molten pool information;
step S53: and analyzing the obtained information of the molten drops and the molten pool to obtain the deposition morphology information of different welding process parameters of the arc fuse additive manufacturing.
Preferably, the model operation module comprises a menu calling sub-module and a model control sub-module, wherein the menu calling sub-module is used for calling a system menu through palm gestures; the model control submodule is used for controlling the virtual model by selecting different operation instructions, including single-hand, double-hand or click button control;
and the menu calling sub-module calls a system menu by the following modes: the palm is away from the user's sight and away from the user, and a menu is displayed; the palm is opposite to the sight of the user and far away from the user, so that the menu is hidden;
the operation instructions in the model control sub-module comprise one or more of movement, rotation, scaling, printing model selection, printing material selection, arc information display, printing state display, workpiece completion display and defect display.
Preferably, the step S7 includes:
step S71: respectively calculating image and motion information, generating a virtual object by using digital virtual information calculated and output in an arc fuse additive manufacturing virtual model, and acquiring a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model;
step S72: acquiring real scene information, acquiring current space coordinates, and projecting a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model in an arc fuse additive manufacturing virtual model into the real space coordinates;
step S73: final image projection information is generated by fusing digital data in the arc fuse additive manufacturing virtual model with digital information in the mixed reality display module.
The application provides a mixed reality-based arc fuse additive manufacturing digital twin system construction system, which comprises:
module M1: acquiring real model scene information of arc fuse additive manufacturing;
module M2: establishing an arc fuse additive manufacturing virtual model by adopting 3 Dmax;
module M3: the mixed reality display module is used for comparing the arc fuse additive manufacturing real model with the arc fuse additive manufacturing virtual model in real time to obtain an arc fuse additive manufacturing physical entity data signal information chain;
module M4: establishing communication between an arc fuse additive manufacturing physical entity data signal information chain and an arc fuse additive manufacturing virtual model through a UDP communication protocol;
module M5: displaying arc fuse additive manufacturing welding bead deposition morphology effects in a virtual model through different welding process parameters by using a welding piece deposition display module;
module M6: the method comprises the steps of using a model operation module for a user to interact with the system through different gestures so as to observe an arc fuse additive manufacturing deposition process;
module M7: importing the arc fuse additive manufacturing virtual model, equipment information and operation information into a mixed reality scene, visually superposing the mixed reality display model and the three-dimensional configuration, and monitoring and printing by an operator according to the operation information until the error value of the three-dimensional configuration and the arc fuse additive manufacturing virtual model is within a preset range;
the mixed reality display module displays three-dimensional configuration mixed reality projection of the arc fuse additive manufacturing virtual model through a Hollolens 2 display device.
Preferably, the module M3 includes:
the signal information chain comprises the current equipment position state, the deposition progress, the current processing and deposition technological parameters, the processing progress and the geometric errors;
module M31: identifying and comparing the positions of a mechanical arm and a positioner of the equipment, and acquiring matched position characteristic information of the mechanical arm and the positioner;
module M32: analyzing the image information of the processing progress to obtain the characteristic information of the image, wherein the characteristic information comprises one or more of the completion degree of printing the workpiece, the temperature of the workpiece and the number of printing layers;
module M33: and analyzing the image information of the technological parameters to obtain characteristic information of the image, wherein the characteristic information comprises one or more of welding speed, wire feeding speed and welding current.
Preferably, the module M5 includes:
module M51: analyzing the actual printing processing process information of the arc fuse additive manufacturing to obtain characteristic information, wherein the characteristic information comprises one or more of material types, welding speed, wire feeding speed, welding current and shielding gas flow;
module M52: analyzing the obtained characteristic information, and processing the characteristic information by a machine learning method to obtain correct molten drop and molten pool information;
module M53: and analyzing the obtained information of the molten drops and the molten pool to obtain the deposition morphology information of different welding process parameters of the arc fuse additive manufacturing.
Preferably, the model operation module comprises a menu calling sub-module and a model control sub-module, wherein the menu calling sub-module is used for calling a system menu through palm gestures; the model control submodule is used for controlling the virtual model by selecting different operation instructions, including single-hand, double-hand or click button control;
and the menu calling sub-module calls a system menu by the following modes: the palm is away from the user's sight and away from the user, and a menu is displayed; the palm is opposite to the sight of the user and far away from the user, so that the menu is hidden;
the operation instructions in the model control sub-module comprise one or more of movement, rotation, scaling, printing model selection, printing material selection, arc information display, printing state display, workpiece completion display and defect display.
Preferably, the module M7 includes:
module M71: respectively calculating image and motion information, generating a virtual object by using digital virtual information calculated and output in an arc fuse additive manufacturing virtual model, and acquiring a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model;
module M72: acquiring real scene information, acquiring current space coordinates, and projecting a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model in an arc fuse additive manufacturing virtual model into the real space coordinates;
module M73: final image projection information is generated by fusing digital data in the arc fuse additive manufacturing virtual model with digital information in the mixed reality display module.
Compared with the prior art, the application has the following beneficial effects:
(1) The equipment based on digital twinning can evaluate the state of an additive manufacturing process through geometric physical behavior and rule modeling, monitor and predict the manufacturing process of arc fuse additive manufacturing, and realize the state sensing and control functions of physical objects through measuring and controlling the data flow and information flow transmission between physical entities corresponding to the virtual model and the digital twinning;
(2) According to the application, the physical entity information and the virtual model information are integrated together through the mixed reality technology, the corresponding instruction is determined according to the acquired data, and the corresponding feedback result is given, so that the arc fuse additive manufacturing process is intuitively guided, the man-machine interaction is strong, the operation is convenient, the influence of external factors is avoided, and a great effect is exerted on promoting the development of the arc fuse additive manufacturing;
(3) The application is beneficial to solving the key problems of limiting the formation and quality control of the arc fuse additive manufacturing structure, shortening the manufacturing period, providing the most effective solution for the additive manufacturing process, expanding the application range of the arc fuse additive manufacturing structure, contributing to the manufacture of large-scale high-performance metal parts, providing a new theory, a new thought and a new strategy of optimizing the arc fuse additive process parameters and controlling the quality of the arc fuse additive manufacturing structure, and having important theoretical significance and engineering application value.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, given with reference to the accompanying drawings in which:
FIG. 1 is an overall architecture of a build arc fuse additive manufacturing digital twin system;
FIG. 2 is a block diagram of a hybrid reality technology based arc fuse additive manufacturing digital twin system;
FIG. 3 is a flow chart of a method of constructing an arc fuse additive manufacturing digital twin system based on mixed reality technology;
FIG. 4 is a flow diagram of a model operation module for constructing an arc fuse additive manufacturing digital twin system based on mixed reality technology;
FIG. 5 is a flow diagram of a model operation module for constructing an arc fuse additive manufacturing digital twin system based on mixed reality technology;
wherein: the device comprises a 1-real model, a 2-virtual model, a 3-virtual environment communication module, a 4-weldment deposition display module, a 5-mixed reality display module and a 6-model operation module;
31-mechanical arm, 32-position changing machine, 33-welding machine, 34-wire feeder and 35-welding table;
41-position tracker, 42-motion capture device, 43-head-mounted interaction device, 44-force feedback device.
Detailed Description
The present application will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the present application, but are not intended to limit the application in any way. It should be noted that variations and modifications could be made by those skilled in the art without departing from the inventive concept. These are all within the scope of the present application.
Example 1:
referring to fig. 1 to 5, an implementation method for constructing a digital twin system for manufacturing an arc fuse additive based on a mixed reality technology according to the present application will be described with reference to an embodiment, and specific implementation steps are as follows:
s1: the scene information of the arc fuse additive manufacturing real model 2 is obtained, and the scene information comprises specific data of the equipment such as a mechanical arm 31, a positioner 32, a welding machine 33, a wire feeder 34, a welding table 35 and the like of the real model 1, and physical entity environment information of the arc fuse additive manufacturing.
S2: the arc fuse additive manufacturing virtual model 2 is constructed by adopting 3Dmax, and specifically comprises the following steps: models such as a mechanical arm 31, a positioner 32, a welding machine 33, a wire feeder 34, a welding table 35 and the like in the virtual model 2 are assembled, rendered, optimized, subjected to format conversion and the like, and then are imported into a Unity3D platform, so that an arc fuse additive manufacturing digital twin environment is built.
S3: the virtual-real environment communication module 3 is configured to establish communication between the collected arc fuse additive manufacturing physical entity data signal information chain and the arc fuse additive manufacturing digital twin environment through a UDP communication protocol, receive data by the mechanical arm 31, and execute related instructions, that is, implement accurate parameter and path adaptive matching in the Unity3D, so as to establish a dynamic arc fuse additive manufacturing digital twin environment.
S4: and the weldment deposition display module 4 is used for accumulating the small spheres to construct an additive component, and observing the appearance effect of welding process parameters on arc fuse additive manufacturing forming weld beads in a virtual environment of the digital twin system.
S5: a mixed reality display module 5 for displaying holographic projections of the arc fuse additive manufacturing digital twin model by a hollens 2 display device; the model operation module is used for enabling a user to interact with the system through different gestures so as to visually observe the arc fuse additive manufacturing and depositing process;
the head-mounted interaction device 43 in the mixed reality display module 5 processes the models of the mechanical arm 31, the positioner 32, the welding machine 33, the wire feeder 34, the welding table 35 and the like in the virtual model 2 and the real model 1, and solves the problems of consistency, coexistence and interactivity of the real scene and the virtual object in geometric and time by utilizing a virtual-real fusion display technology to generate a three-dimensional configuration; acquiring the position information of models such as a mechanical arm 31, a positioner 32, a welding machine 33, a wire feeder 34, a welding table 35 and the like of the current machine equipment through head-mounted interactive equipment 43 in the mixed reality display model 5, and capturing gesture motion and equipment position information by a position tracker 41 and a motion capture equipment 42 in the mixed reality display model 5; confirming whether printing processing can be performed or not, enabling a palm to be opposite to the sight line of a user and far away from the user, displaying a menu, enabling the palm to be opposite to the sight line of the user and far away from the user, and hiding the menu;
s6: comparing the arc fuse additive manufacturing real model with the arc fuse additive manufacturing virtual model and the workbench model in real time to obtain operation information, wherein the operation information comprises the current equipment position state, the processing progress, arc parameters, and the processing parameters, the processing progress and the geometric errors of current processing printing;
the palm is away from the user's line of sight and far away from the user, a menu is displayed, the processing printing material is selected, the printing processing is started, the current printing processing state is obtained in real time, the characteristic information of the workpiece processing progress, the technological parameters, the number of printing layers, geometric errors and the like is included, the characteristic information of the processing progress includes one or more of the degree of completion of the printing workpiece, the temperature of the workpiece and the number of printing layers, and the characteristic information of the technological parameters includes one or more of welding speed, wire feeding speed and welding current; the palm is opposite to the user's line of sight and away from the user, hiding the menu.
S7: the real model for manufacturing the arc fuse additive, the virtual model for manufacturing the arc fuse additive and the workbench model are compared in real time, and operation information is obtained, wherein the operation information comprises the current equipment position state, the deposition progress, the current processing and deposition technological parameters, the processing progress and the geometric errors.
The operator knows the real-time situation of the printed matter and knows whether the current geometrical error meets the design requirement, whether the printing process needs to be terminated, whether the current moving speed of the robot arm 31 needs to be adjusted, how much the adjustment value is, whether the printed matter generates dead weight deformation, and the like. And an operator judges whether to interfere the arc fuse additive manufacturing and printing process or not according to the information provided by the mixed reality display module 5, and adjusts the process parameters.
The physical entity is arc fuse additive manufacturing printing experimental equipment, the virtual model is a digital twin system, the real model 1 and the virtual model 2 are connected through a sensor or changed sensor information flow, data acquired in real time in the real model 1 are transmitted to the virtual model 2 through the sensor, and the virtual environment communication module 3 is a concept of amplifying the sensor information flow and is used as a module for explanation. The real model 1 and the virtual model 2 are connected through the virtual environment communication module 3, the weldment deposition display module 4 is a part of the virtual model 2, only a part printed in real time, namely a part printed by us, is displayed, and the part is generated in real time. The mixed reality display module 5 represents what the mixed reality projects, namely the three-dimensional configuration of the projection, and also comprises an operating system capable of performing virtual clicking on the blank, and then performs certain operation through the model operation module 6 to obtain certain feedback, and the mixed reality display module 5 performs corresponding embodiment. That is, the mixed reality display module 5 completes the whole digital twin visualization operation, and then the model operation module 6 performs certain operation.
Example 2:
referring to fig. 1-5, step S5 in the implementation method of constructing the arc fuse additive manufacturing digital twin system based on the mixed reality technology is described in detail, and is used for explaining another function of the mixed reality display module 5 and the model operation module 6 implemented in the method, that is, capturing hand gesture actions of an operator by using the mixed reality display module 5, and adjusting the angle and the relative position relationship between the virtual model module and the real model scene 1 in the combined image, so as to achieve the interaction effect.
S5: the mixed reality display module is used for displaying holographic projection of the arc fuse additive manufacturing digital twin model through Hollolens 2 display equipment; and the model operation module is used for enabling a user to interact with the system through different gestures so as to visually observe the arc fuse additive manufacturing and depositing process.
The position tracker 41 in the mixed reality display device 5 is used for capturing the virtual model information and the position information of the real model scene 1, processing and analyzing the obtained images, and outputting a combined image containing the virtual model and the real model scene 1.
The operator wears the data glove in the mixed reality display module 5, the data glove has a motion capture device 42, and can capture and recognize hand gesture motions, a sensor component is arranged in the data glove, angular velocity and acceleration information of the hand gesture motions are obtained, the information is transmitted to a head-mounted interaction device 43, and the head-mounted interaction device 43 binds the 3D virtual model with the real model scene 1, namely, adds the virtual model into the real scene 1 to obtain a combined image, so that virtual-real fusion is realized.
The operator wears the data glove in the mixed reality display module 5 and performs lifting, pulling down, rotating, translating and folding opening actions, namely, an instruction is issued to the mixed reality display module 5, the instruction is further transmitted to the motion capturing device 42, the head-mounted interaction device 43 realizes dragging in different directions through calculation, rotation and scaling of functions of the virtual model and the real model scene 1, and different interaction effects are achieved.
The operator wears the head-mounted interactive device 43 in the mixed reality display device 5, the head-mounted interactive device is provided with the motion capturing device 42, head motions can be captured and identified, a sensor and an eye tracker component are arranged in the head-mounted interactive device, angle and speed information of the head motions and the eye motions are obtained, then the virtual model is bound with the real model scene 1, namely the virtual model is added into the real model scene 1 to obtain a combined image, and virtual-real fusion is achieved.
The operator wears the head-mounted interactive device 43 in the mixed reality display device 5, and mainly realizes three interactive functions, namely lifting, pulling down, rotating, translating and folding to open.
In the description of the present application, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present application.
Those skilled in the art will appreciate that the systems, apparatus, and their respective modules provided herein may be implemented entirely by logic programming of method steps such that the systems, apparatus, and their respective modules are implemented as logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., in addition to the systems, apparatus, and their respective modules being implemented as pure computer readable program code. Therefore, the system, the apparatus, and the respective modules thereof provided by the present application may be regarded as one hardware component, and the modules included therein for implementing various programs may also be regarded as structures within the hardware component; modules for implementing various functions may also be regarded as being either software programs for implementing the methods or structures within hardware components.
The foregoing describes specific embodiments of the present application. It is to be understood that the application is not limited to the particular embodiments described above, and that various changes or modifications may be made by those skilled in the art within the scope of the appended claims without affecting the spirit of the application. The embodiments of the application and the features of the embodiments may be combined with each other arbitrarily without conflict.
Claims (10)
1. The method for constructing the digital twin system for manufacturing the arc fuse additive based on mixed reality is characterized by comprising the following steps of:
step S1: acquiring real model scene information of arc fuse additive manufacturing;
step S2: establishing an arc fuse additive manufacturing virtual model by adopting 3 Dmax;
step S3: the mixed reality display module is used for comparing the arc fuse additive manufacturing real model with the arc fuse additive manufacturing virtual model in real time to obtain an arc fuse additive manufacturing physical entity data signal information chain;
step S4: establishing communication between an arc fuse additive manufacturing physical entity data signal information chain and an arc fuse additive manufacturing virtual model through a UDP communication protocol;
step S5: displaying arc fuse additive manufacturing welding bead deposition morphology effects in a virtual model through different welding process parameters by using a welding piece deposition display module;
step S6: the method comprises the steps of using a model operation module for a user to interact with the system through different gestures so as to observe an arc fuse additive manufacturing deposition process;
step S7: importing the arc fuse additive manufacturing virtual model, equipment information and operation information into a mixed reality scene, visually superposing the mixed reality display model and the three-dimensional configuration, and monitoring and printing by an operator according to the operation information until the error value of the three-dimensional configuration and the arc fuse additive manufacturing virtual model is within a preset range;
the mixed reality display module displays three-dimensional configuration mixed reality projection of the arc fuse additive manufacturing virtual model through a Hollolens 2 display device.
2. The mixed reality based arc fuse additive manufacturing digital twin system construction method of claim 1, wherein the step S3 comprises:
the signal information chain comprises the current equipment position state, the deposition progress, the current processing and deposition technological parameters, the processing progress and the geometric errors;
step S31: identifying and comparing the positions of a mechanical arm and a positioner of the equipment, and acquiring matched position characteristic information of the mechanical arm and the positioner;
step S32: analyzing the image information of the processing progress to obtain the characteristic information of the image, wherein the characteristic information comprises one or more of the completion degree of printing the workpiece, the temperature of the workpiece and the number of printing layers;
step S33: and analyzing the image information of the technological parameters to obtain characteristic information of the image, wherein the characteristic information comprises one or more of welding speed, wire feeding speed and welding current.
3. The mixed reality based arc fuse additive manufacturing digital twin system construction method of claim 1, wherein the step S5 comprises:
step S51: analyzing the actual printing processing process information of the arc fuse additive manufacturing to obtain characteristic information, wherein the characteristic information comprises one or more of material types, welding speed, wire feeding speed, welding current and shielding gas flow;
step S52: analyzing the obtained characteristic information, and processing the characteristic information by a machine learning method to obtain correct molten drop and molten pool information;
step S53: and analyzing the obtained information of the molten drops and the molten pool to obtain the deposition morphology information of different welding process parameters of the arc fuse additive manufacturing.
4. The method for constructing the digital twin system for manufacturing the arc fuse additive based on mixed reality according to claim 1, wherein the model operation module comprises a menu calling sub-module and a model control sub-module, and the menu calling sub-module is used for calling a system menu through palm gestures; the model control submodule is used for controlling the virtual model by selecting different operation instructions, including single-hand, double-hand or click button control;
and the menu calling sub-module calls a system menu by the following modes: the palm is away from the user's sight and away from the user, and a menu is displayed; the palm is opposite to the sight of the user and far away from the user, so that the menu is hidden;
the operation instructions in the model control sub-module comprise one or more of movement, rotation, scaling, printing model selection, printing material selection, arc information display, printing state display, workpiece completion display and defect display.
5. The mixed reality based arc fuse additive manufacturing digital twin system construction method of claim 1, wherein the step S7 comprises:
step S71: respectively calculating image and motion information, generating a virtual object by using digital virtual information calculated and output in an arc fuse additive manufacturing virtual model, and acquiring a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model;
step S72: acquiring real scene information, acquiring current space coordinates, and projecting a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model in an arc fuse additive manufacturing virtual model into the real space coordinates;
step S73: final image projection information is generated by fusing digital data in the arc fuse additive manufacturing virtual model with digital information in the mixed reality display module.
6. A mixed reality-based arc fuse additive manufacturing digital twin system building system, comprising:
module M1: acquiring real model scene information of arc fuse additive manufacturing;
module M2: establishing an arc fuse additive manufacturing virtual model by adopting 3 Dmax;
module M3: the mixed reality display module is used for comparing the arc fuse additive manufacturing real model with the arc fuse additive manufacturing virtual model in real time to obtain an arc fuse additive manufacturing physical entity data signal information chain;
module M4: establishing communication between an arc fuse additive manufacturing physical entity data signal information chain and an arc fuse additive manufacturing virtual model through a UDP communication protocol;
module M5: displaying arc fuse additive manufacturing welding bead deposition morphology effects in a virtual model through different welding process parameters by using a welding piece deposition display module;
module M6: the method comprises the steps of using a model operation module for a user to interact with the system through different gestures so as to observe an arc fuse additive manufacturing deposition process;
module M7: importing the arc fuse additive manufacturing virtual model, equipment information and operation information into a mixed reality scene, visually superposing the mixed reality display model and the three-dimensional configuration, and monitoring and printing by an operator according to the operation information until the error value of the three-dimensional configuration and the arc fuse additive manufacturing virtual model is within a preset range;
the mixed reality display module displays three-dimensional configuration mixed reality projection of the arc fuse additive manufacturing virtual model through a Hollolens 2 display device.
7. The mixed reality based arc fuse additive manufacturing digital twin system building system of claim 6, wherein the module M3 comprises:
the signal information chain comprises the current equipment position state, the deposition progress, the current processing and deposition technological parameters, the processing progress and the geometric errors;
module M31: identifying and comparing the positions of a mechanical arm and a positioner of the equipment, and acquiring matched position characteristic information of the mechanical arm and the positioner;
module M32: analyzing the image information of the processing progress to obtain the characteristic information of the image, wherein the characteristic information comprises one or more of the completion degree of printing the workpiece, the temperature of the workpiece and the number of printing layers;
module M33: and analyzing the image information of the technological parameters to obtain characteristic information of the image, wherein the characteristic information comprises one or more of welding speed, wire feeding speed and welding current.
8. The mixed reality based arc fuse additive manufacturing digital twin system building system of claim 6, wherein the module M5 comprises:
module M51: analyzing the actual printing processing process information of the arc fuse additive manufacturing to obtain characteristic information, wherein the characteristic information comprises one or more of material types, welding speed, wire feeding speed, welding current and shielding gas flow;
module M52: analyzing the obtained characteristic information, and processing the characteristic information by a machine learning method to obtain correct molten drop and molten pool information;
module M53: and analyzing the obtained information of the molten drops and the molten pool to obtain the deposition morphology information of different welding process parameters of the arc fuse additive manufacturing.
9. The mixed reality based arc fuse additive manufacturing digital twin system building system of claim 6, wherein the model operation module comprises a menu calling sub-module and a model manipulation sub-module, the menu calling sub-module being configured to call a system menu by palm gestures; the model control submodule is used for controlling the virtual model by selecting different operation instructions, including single-hand, double-hand or click button control;
and the menu calling sub-module calls a system menu by the following modes: the palm is away from the user's sight and away from the user, and a menu is displayed; the palm is opposite to the sight of the user and far away from the user, so that the menu is hidden;
the operation instructions in the model control sub-module comprise one or more of movement, rotation, scaling, printing model selection, printing material selection, arc information display, printing state display, workpiece completion display and defect display.
10. The mixed reality based arc fuse additive manufacturing digital twin system building system of claim 6, wherein the module M7 comprises:
module M71: respectively calculating image and motion information, generating a virtual object by using digital virtual information calculated and output in an arc fuse additive manufacturing virtual model, and acquiring a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model;
module M72: acquiring real scene information, acquiring current space coordinates, and projecting a mechanical arm, a welding machine, a welding gun, a positioner and a workbench model in an arc fuse additive manufacturing virtual model into the real space coordinates;
module M73: final image projection information is generated by fusing digital data in the arc fuse additive manufacturing virtual model with digital information in the mixed reality display module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310992082.1A CN117150587B (en) | 2023-08-08 | 2023-08-08 | Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310992082.1A CN117150587B (en) | 2023-08-08 | 2023-08-08 | Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117150587A true CN117150587A (en) | 2023-12-01 |
| CN117150587B CN117150587B (en) | 2024-07-02 |
Family
ID=88905261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310992082.1A Active CN117150587B (en) | 2023-08-08 | 2023-08-08 | Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117150587B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014140719A1 (en) * | 2013-03-11 | 2014-09-18 | Lincoln Global, Inc. | Systems and methods providing an enhanced user experience in a real-time simulated virtual reality welding environment |
| CN111161410A (en) * | 2019-12-30 | 2020-05-15 | 中国矿业大学(北京) | Mine digital twinning model and construction method thereof |
| CN112287489A (en) * | 2020-11-06 | 2021-01-29 | 中国石油大学(北京) | Method for determining parameters of arc fuse additive manufacturing process based on multi-physical-field simulation |
| US20210201584A1 (en) * | 2019-12-31 | 2021-07-01 | VIRNECT inc. | System and method for monitoring field based augmented reality using digital twin |
| WO2022090983A1 (en) * | 2020-10-30 | 2022-05-05 | Universidade De Coimbra | Methods and apparatus for cognitive robotic additive manufacturing system based on the directed energy deposition (ded) technology |
| CN114564880A (en) * | 2022-01-26 | 2022-05-31 | 南京理工大学 | Method for constructing digital twin module in additive manufacturing process |
| US20230158595A1 (en) * | 2021-11-22 | 2023-05-25 | University Of Southern California | Prediction and control of product shape quality in wire and arc additive manufacturing through machine learning |
| CN116339517A (en) * | 2023-04-25 | 2023-06-27 | 同济大学 | Closed-loop bidirectional interaction digital twin system based on robot and construction method thereof |
-
2023
- 2023-08-08 CN CN202310992082.1A patent/CN117150587B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014140719A1 (en) * | 2013-03-11 | 2014-09-18 | Lincoln Global, Inc. | Systems and methods providing an enhanced user experience in a real-time simulated virtual reality welding environment |
| CN111161410A (en) * | 2019-12-30 | 2020-05-15 | 中国矿业大学(北京) | Mine digital twinning model and construction method thereof |
| US20210201584A1 (en) * | 2019-12-31 | 2021-07-01 | VIRNECT inc. | System and method for monitoring field based augmented reality using digital twin |
| WO2022090983A1 (en) * | 2020-10-30 | 2022-05-05 | Universidade De Coimbra | Methods and apparatus for cognitive robotic additive manufacturing system based on the directed energy deposition (ded) technology |
| CN112287489A (en) * | 2020-11-06 | 2021-01-29 | 中国石油大学(北京) | Method for determining parameters of arc fuse additive manufacturing process based on multi-physical-field simulation |
| US20230158595A1 (en) * | 2021-11-22 | 2023-05-25 | University Of Southern California | Prediction and control of product shape quality in wire and arc additive manufacturing through machine learning |
| CN114564880A (en) * | 2022-01-26 | 2022-05-31 | 南京理工大学 | Method for constructing digital twin module in additive manufacturing process |
| CN116339517A (en) * | 2023-04-25 | 2023-06-27 | 同济大学 | Closed-loop bidirectional interaction digital twin system based on robot and construction method thereof |
Non-Patent Citations (4)
| Title |
|---|
| BINTAO WU: "Enhanced corrosion performance in Ti-6Al-4V alloy produced via wire-arc directed energy deposition with magnetic arc oscillation", 《ADDITIVE MANUFACTURING》, vol. 66, no. 25, 26 February 2023 (2023-02-26), pages 1 - 12 * |
| HAOCHEN MUA: "Toward a smart wire arc additive manufacturing system: A review on current developments and a framework of digital twin", JOURNAL OF MANUFACTURING SYSTEMS, vol. 67, 3 February 2023 (2023-02-03), pages 174 - 189 * |
| 丁东红: "数字孪生驱动增材制造的研究进展", 《电焊机》, vol. 53, no. 2, 28 February 2023 (2023-02-28), pages 24 - 40 * |
| 刘理想: "电弧增材制造形貌研究及堆积过程数字孪生模拟", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 1, 15 January 2021 (2021-01-15), pages 8 - 9 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117150587B (en) | 2024-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dianatfar et al. | Review on existing VR/AR solutions in human–robot collaboration | |
| CN110238831B (en) | Robot teaching system and method based on RGB-D image and teaching device | |
| JP6725727B2 (en) | Three-dimensional robot work cell data display system, display method, and display device | |
| US9415509B2 (en) | Robot teach device with 3-D display | |
| Liu et al. | Augmented reality-assisted intelligent window for cyber-physical machine tools | |
| CN105665970B (en) | For the path point automatic creation system and method for welding robot | |
| CN110599603A (en) | Mechanical equipment visual interaction and equipment state monitoring system and method based on augmented reality | |
| JP2017094406A (en) | Simulation device, simulation method, and simulation program | |
| JP2019188530A (en) | Simulation device of robot | |
| US20030120391A1 (en) | Robot operation teaching method and apparatus | |
| EP1435737A1 (en) | An augmented reality system and method | |
| CA2713700A1 (en) | Robot off-line teaching method | |
| US10606241B2 (en) | Process planning apparatus based on augmented reality | |
| JPH09212219A (en) | Three-dimensional virtual model preparing device and monitor controller for controlled object | |
| JP2006190228A (en) | Operation program creating method | |
| US20060287772A1 (en) | Device and method for programming an industrial robot | |
| KR20170024769A (en) | Robot control apparatus | |
| CN112958974A (en) | Interactive automatic welding system based on three-dimensional vision | |
| CN117150587B (en) | Arc fuse additive manufacturing digital twin system construction method and system based on mixed reality | |
| JP2003256025A (en) | Robot motion teaching method and device | |
| WO2022032688A1 (en) | Method for remote assistance and device | |
| CN105955180A (en) | Intelligent manufacturing adaptive dynamic generation robot real-time automatic programming method | |
| Fang et al. | A survey on HoloLens AR in support of human-centric intelligent manufacturing | |
| CN115401696A (en) | Data/model hybrid-driven robot remote driving method | |
| JPH1158276A (en) | Off-line simulation system of robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |