WO2024020937A1

WO2024020937A1 - Handicraft path guidance learning system based on mixed reality technology

Info

Publication number: WO2024020937A1
Application number: PCT/CN2022/108639
Authority: WO
Inventors: 王冠云; 王舒弘; 陶冶; 姬俊哲; 汪芷淇; 蔡琳琳; 孙凌云
Original assignee: 浙江大学
Priority date: 2022-07-27
Filing date: 2022-07-28
Publication date: 2024-02-01
Also published as: CN115170776A

Abstract

A handicraft path guidance learning system based on mixed reality technology, the system providing to a user during the process of handicraft manufacturing and on the basis of mixed reality technology a drawing line guidance path on an MR display interface, allowing the user to use a 3D printing tool to perform drawing line printing according to the drawing line guidance path. The system comprises: a model import selection module, a parameter adjustment and control module, and a path planning module. The system can be applied to common daily-life mobile terminals, such as mobile phones, tablet computers, etc., and does not require an additional mixed reality terminal. The system can be applied to handicraft manufacturing which utilizes drawn lines as a basic implementation mode, for instance: hand-drawn lines, jewelry made by winding metal wire, hand drawing printing utilizing a 3D printing pen, etc. The system does not require an additional auxiliary shaping mold, lowers the threshold for manual operation, and can simultaneously help a beginner to carry out a design preview via a visual guidance system, aiding in the design and creation of handicrafts.

Description

Craft path guidance learning system based on mixed reality technology

Technical field

The present invention relates to the field of mixed reality. Specifically, it relates to a craft path guidance learning system and method based on mixed reality technology. The system and method can be used on mobile terminals such as mobile phones and tablet computers, using existing screens, cameras, etc. Components to complete the guidance and auxiliary learning functions of related handicrafts by reducing costs.

Background technique

Handicraft refers to arts and crafts with a unique artistic style made by manual labor. Arts and crafts are different from the mass production of standardized daily handicrafts through large-scale industrial mechanization.

However, for beginners, learning crafts often has a high threshold. Learning through pictures, videos and other methods is highly intuitive and can quickly understand the process and production methods of handicrafts. However, the three-dimensional sense of space and sense of experience are weak, which is not suitable for beginners who do not have strong spatial imagination and hands-on ability. It is said that it is difficult to complete the practice by yourself. Therefore, with the help of mixed reality technology, beginners can be effectively helped to feel and experience the production process immersively, thereby improving learning efficiency and creation quality. However, traditional virtual reality glasses and other devices can only present users with images on the screen, but cannot provide information about the surrounding environment.

Contents of the invention

The purpose of the present invention is to provide a handicraft path guidance learning system based on mixed reality technology in view of the shortcomings of the existing technology, which can be used for spatial path planning guidance in handicraft learning and practice.

The technical solutions adopted by the present invention are as follows:

A handicraft path guidance learning system based on mixed reality technology. During the handicraft production process, the system provides users with a line drawing guidance path on the MR display interface based on mixed reality technology, so that the user can use 3D printing tools to draw according to the description. Line guide path for line drawing and printing.

In the above technical solution, further, the system has the following functional modules:

A model import selection module, which is used for users to import required models into the system or select required models from the sample model library;

The parameter control module adjusts the outline and size parameters of single components according to the required target handicraft structure, combined with the selected or imported model, to determine a design model that meets the requirements of the target handicraft structure;

The path planning display module is used to layer the design model and plan the printing paths of each layer to form a line drawing path guide map of each layer, and present the line drawing path guide maps of each layer on the MR in sequence according to the printing order. In the display interface, the user performs line drawing printing with the 3D printing tool according to the line drawing path guide map presented in the MR display interface to obtain the target handicraft structure.

Further, the model is a solid model or a vector path.

Furthermore, the system builds an interactive interface based on the Grasshopper tool, performs calculations and modeling based on user selection and input; previews the model based on the three-dimensional modeling software Rhinoceros; and implements the interface and model in the mobile terminal virtual world based on the plug-in Fologram. Synchronize to achieve mixed reality effects.

Further, the path planning method is as follows: according to the user's selection of the desired effect, parameters such as the width, spacing, and angle between the actual drawn lines and the edge of the model are obtained, and then a series of points are taken on the edge of the model, and based on the path parameters Connect the corresponding points to obtain several line segments, and connect these line segments end to end in order to obtain the final path.

Furthermore, the line drawing and printing guide map is a simulation of the path. Using the Time Trigger in Grasshopper, based on the settings of parameters such as speed, it is set to perform a cumulative calculation of the distance that the line should move every 10 ms. Calculate Find the length of the path expected to be drawn at the current moment, mark the current drawing line position in the path, and color the corresponding path to achieve the animation guidance effect.

The handicraft path-guided learning and production method based on mixed reality technology is implemented based on the above system, including the following steps:

Step 1: Obtain the model: Select a sample model in the system interface or import the model yourself;

Step 2: Design model parameters: According to the required goals, combined with the selected or imported model in the MR system, adjust its outline, size and other parameters to determine the basic shape of the model;

According to the required goals, the final design model is determined by mirroring, rotating, array and other transformations on the model;

Step 3: Layering and path planning

According to the demand goals, multiple models with different functional requirements can be set hierarchically. On this basis, the system plans the line drawing path of the model according to the user's selection. The user's selection includes setting the speed parameters of the path simulation, Set the order of model path simulation when there are multiple complex models;

According to the above path planning parameters, the system obtains real-time images through the camera of the mobile terminal, and displays the simulated animation guidance interface of the planned path through the display screen of the mobile terminal. The animation guidance interface can be unified with the real-time scene images to achieve a mixed reality effect;

Step 4: Manual operation guided by MR

In the mixed reality learning guidance practice, users do not need to wear additional mixed reality accessories, they can watch directly on the display terminal, and at the same time use 3D printing tools to complete handicraft production along the guidance path. Users can pause and continue at any time when using the guidance system. Loop playback and end animations achieve a more humane learning effect.

After obtaining the printed model, you can also combine external effects such as heat, water and other external factors to induce the deformation of the model material to achieve the desired effect.

The beneficial effects of the present invention are:

The present invention is a handicraft path guidance learning system based on mixed reality technology. With the help of hardware sensors of mobile terminals such as mobile phones and tablet computers, users do not need to purchase a complete set of virtual reality equipment and can use the MR system to complete related handicrafts with less cost. of learning. Mixed reality technology improves the spatial three-dimensionality of the handicraft process display and the user's sense of participation, provides users with real model previews and guidance effects, helps users complete learning and design more efficiently, and lowers the learning threshold.

Description of drawings

Figure 1 is a schematic flow chart of a specific method of the present invention;

Figure 2 is a schematic diagram of actual operation in a specific example of the present invention.

Figure 3 is a schematic diagram of actual operation in a specific example of the present invention.

Figure 4 is a schematic interface diagram of the system of the present invention.

Figure 5 is a schematic interface diagram of the system of the present invention.

Figure 6 is a test chart of line drawing speed and layering performance of the present invention.

Figures 7-10 are diagrams of application examples of the present invention respectively.

Detailed ways

In order to better understand the present invention, the solutions of the present invention will be further described below in conjunction with specific embodiments and drawings, but the content of the present invention is not limited only to the following embodiments.

During the handicraft production process, the handicraft path guidance learning system based on mixed reality technology of the present invention can provide the user with a line drawing guidance path on the MR display interface based on the mixed reality technology, so that the user can use the 3D printing tool to follow the line drawing guidance path. Perform line drawing printing. The system has the following functional modules:

The parameter control module, according to the target handicraft structure required, combined with the selected or imported model, can select an example model or import the model by yourself in the system interface. The model is generally an entity model (stl, obj, step, etc. format) or a vector path ( ai, dwg, dwf, svg and other formats). Adjust the outline and size parameters of the single component, and transform the input model through a series of methods provided by the system, such as mirroring, rotation, array, etc., to ultimately determine the design model that meets the requirements of the target handicraft structure;

The path planning module is used to layer the design model and plan the printing paths of each layer to form a line drawing path guide map of each layer, and present the line drawing path guide maps of each layer on the MR display in sequence according to the printing order. In the interface, the user uses the 3D printing tool to trace and print according to the line drawing path guide map presented on the MR display interface to obtain the target handicraft structure. The path planning method in this module is as follows: according to the user's selection of desired effects, such as metal wire winding patterns, deformation effects of printed objects, etc., parameters such as the width, spacing, and angle between the actual drawn lines and the edge of the model are obtained, and then A series of points are taken on the edge of the model and the corresponding points are connected according to the path parameters to obtain several line segments. These line segments are connected end to end in order to obtain the final path. The line drawing and printing guide map is a simulation of the path. You can use the Time Trigger in Grasshopper, based on the settings of parameters such as speed, and set each period of time, such as 10ms, to perform a cumulative calculation of the distance that the line should move to calculate the current The length of the path expected to be drawn at any time, and the current drawing line position is marked in the path, and the corresponding path is colored to achieve the animation guidance effect.

In the process of guided learning and production of handicraft paths based on the above system, the final design model can be determined in the system first; then according to the demand goals, multiple models with different functional requirements can be set in layers, such as hand-painted line cases Complete filling parts and spacer filling parts can be set in it, deformable parts and non-deformable parts can be set in hand-painted 4D printing, and different winding patterns can be set in metal wire winding. On this basis, the system plans the line drawing path of the model according to the user's choice. For example, the speed parameters of path simulation can be set in hand-drawn line drawing and hand-drawn 4D printing. When there are multiple complex models, you can also set the order of model path simulation to simplify the handicraft production process.

According to the above path planning parameters, the system obtains real-time images through the camera of the mobile terminal, and displays the simulated animation guidance interface of the planned path through the display screen of the mobile terminal. The animation guidance interface can be unified with the real-time scene images to achieve a mixed reality effect. In mixed reality learning guidance practice, users can watch directly on tablets and other terminals without wearing additional mixed reality accessories (such as helmets and glasses), and at the same time use relevant tools (such as 3D printing pens) to complete handicrafts along the guidance path. . Users can pause, continue, loop and end animations at any time when using the guidance system to achieve a more humane learning effect. Furthermore, the MR system of the present invention can simulate corresponding effects in real time based on model design and parameter adjustment, display them in the virtual world, provide design previews, and be applied to mobile terminals such as mobile phones and tablet computers. It uses cameras to collect the real environment and capture what the user sees. The design model is integrated with reality through the terminal screen.

Further, the MR system of the present invention can use the Grasshopper tool to build an interactive interface, and perform calculations and modeling based on user selection and input; use the three-dimensional modeling software Rhinoceros to preview the model; use the plug-in Fologram to realize the interface and model on the mobile terminal Synchronization in the virtual world can achieve mixed reality effects through the Sync component in Fologram to complete model synchronization; filtering operations can be performed through the Filter and Gate components to complete model layering and parameter setting functions with multiple choices. Figures 2 and 3 are printing examples of a self-twisting model in the present invention. The specific process includes the following:

1. Model acquisition. According to the required goals and the model library, select a model in the MR system interface or import the model yourself. The corresponding model appears in the virtual world, is integrated with the real world captured by the mobile terminal camera, and is displayed on the screen. Import a self-twisting (twist) model into the system, and the model is an stl file.

2. Model parameter design

According to the required goals and combined with the model in the MR system, parameters such as its outline, size, degree of deformation, and outer contour curve shape are adjusted to determine the basic shape of the single component. Adjust to wide width: 2cm, length 10cm, deformation degree: level 3.

According to the required goals, the model is transformed through a series of methods provided by the system, such as mirroring, rotation, array, etc., to determine the final design model. Each parameter adjustment operation in this step will be transmitted to the program in Grasshopper in real time through the Fologram plug-in, and the adjusted model effect will be calculated and updated in the mixed reality system interface.

3. Layering and path planning

For the selection of multiple models, the models are layered and combined so that the selected models are arranged in a certain order. The order of angles between the filling lines and edges of the selected self-twisting model is: 45° for the first layer, -45° for the second layer, and the filling lines of the upper and lower layers intersect at right angles. Based on this, the system draws two sets of line segments arranged in parallel in the input model, and connects them end to end in order to obtain the final path. Set the speed parameter of the path simulation to "simple and easy-to-operate mode", and the path is planned for the user into a continuous and easy-to-operate trajectory.

According to the layering sequence and according to the parameters of the selected model, the "line drawing position" guided in the system is moved along the planned path. At the same time, the completed parts are colored to achieve animation effects and carried out in mixed reality. simulation.

4. Manual operation guided by MR

The user uses a 3D printing pen to complete the line drawing and printing of the design model in the real world according to the trajectory and speed guidance in the MR system interface. Figures 2 and 3 show the use of a tablet computer for path-guided 4D printing in the MR system. schematic diagram.

Among them, the printing material is a thermoplastic material PLA (Polymaker Polymax) with a diameter of 1.75 mm, and the printing temperature of the printing pen is 170°C.

Users can pause, continue, loop and end the 4D printing guidance animation at any time when using the guidance system. After the user completes manual 4D printing and drawing lines according to the system guidance, the printed model is placed in 80°C hot water for heating. After a few seconds, the model can deform, thereby obtaining the deformation effect set in the system.

When printing multiple models, users can use the high-temperature adhesion of thermoplastic materials to assemble multiple models to obtain a complete model.

In addition, the system and method of the present invention can be combined with post-processing to enrich the structure of personalized handicrafts and produce complex products. Figures 7-10 show some application examples of the present invention.

Claims

A handicraft path guidance learning system based on mixed reality technology, characterized in that during the handicraft production process, the system provides users with a line drawing guidance path on the MR display interface based on mixed reality technology for the user to use 3D printing tools Follow the line drawing guidance path to perform line drawing and printing.
The handicraft path guidance learning system based on mixed reality technology according to claim 1, characterized in that the system has the following functional modules:

A model import selection module, which is used for users to import required models into the system or select required models from the sample model library;

The parameter control module adjusts the outline and size parameters of the single component according to the required target handicraft structure, combined with the selected or imported model, to determine a design model that meets the requirements of the target handicraft structure;

The path planning display module is used to layer the design model and plan the printing paths of each layer to form a line drawing path guide map of each layer, and present the line drawing path guide maps of each layer on the MR in sequence according to the printing order. In the display interface, the user performs line drawing printing with the 3D printing tool according to the line drawing path guide map presented in the MR display interface to obtain the target handicraft structure.
The handicraft path guidance learning system based on mixed reality technology according to claim 2, characterized in that the model is a solid model or a vector path.
The handicraft path guidance learning system based on mixed reality technology according to claim 2, characterized in that the system builds an interactive interface based on the Grasshopper tool, performs calculation and modeling based on user selection and input; and is based on the three-dimensional modeling software Rhinoceros Preview the model; based on the plug-in Fologram, the interface and model can be synchronized in the mobile terminal virtual world to achieve a mixed reality effect.
The handicraft path guidance learning system based on mixed reality technology according to claim 2, characterized in that the path planning method is as follows: according to the user's selection of the desired effect, the width, interval, and model edge of the actual drawn line are obtained parameters such as the included angle, and then select a series of points on the edge of the model, and connect the corresponding points according to the path parameters to obtain several line segments, and connect these line segments end to end in order to obtain the final path.
The handicraft path guidance learning system based on mixed reality technology according to claim 2, characterized in that the line drawing and printing guidance map is a simulation of the path, using Time Trigger in Grasshopper, based on the setting of parameters such as speed , set the cumulative calculation of the distance that the line should move every period of time, calculate the length of the path expected to be drawn at the current moment, mark the current line drawing position in the path, and color the corresponding path to achieve Animation guidance effect.
The handicraft path guided learning and production method based on mixed reality technology is characterized in that, based on the system implementation according to any one of claims 1-6, it includes the following steps:

Step 1: Obtain the model: Select a sample model in the system interface or import the model yourself;

Step 2: Design model parameters: According to the required goals, combined with the selected or imported model in the MR system, adjust its outline, size and other parameters to determine the basic shape of the model;

According to the required goals, the final design model is determined by mirroring, rotating, array and other transformations on the model;

Step 3: Layering and path planning

According to the demand goals, multiple models with different functional requirements can be set hierarchically. On this basis, the system plans the line drawing path of the model according to the user's selection. The user's selection includes setting the speed parameters of the path simulation, Set the order of model path simulation when there are multiple complex models;

According to the above path planning parameters, the system obtains real-time images through the camera of the mobile terminal, and displays the simulated animation guidance interface of the planned path through the display screen of the mobile terminal. The animation guidance interface can be unified with the real-time scene images to achieve a mixed reality effect;

Step 4: Manual operation guided by MR

In the mixed reality learning guidance practice, users do not need to wear additional mixed reality accessories, they can watch directly on the display terminal, and at the same time use 3D printing tools to complete handicraft production along the guidance path. Users can pause and continue at any time when using the guidance system. Loop playback and end animations achieve a more humane learning effect.