TW202329058A - Virtual reality system with an inspecting function of assembling and disassembling and inspection method thereof - Google Patents

Abstract

A virtual reality system with an inspecting function of assembling and disassembling and an inspection method of assembling and disassembling based on virtual reality are presented. A learning-end acquires an inspection data and a teaching assembling disassembling record being set a plurality of checkpoints, plays the teaching assembling disassembling record, modifies a learning assembling disassembling status based on user operations of assembling or disassembling a plurality of virtual objects. The learning-end issues an assembling disassembling error reminder when the learnin gassembling and disassembling status is inconsistent with a teaching assembling disassembling status at any of the checkpoints.

Description

Virtual Reality System with Disassembly and Inspection Function and Disassembly and Inspection Method

The present invention is related to a virtual reality system, in particular to a virtual reality system with a disassembly inspection function and a virtual reality-based disassembly inspection method.

Currently, taking the virtual reality teaching system as an example, teachers usually conduct teaching demonstrations (such as disassembling and assembling mechanical equipment, assembling models, assembling furniture, etc.) in virtual reality by operating virtual reality devices.

Moreover, the virtual reality teaching system can record the demonstration process to obtain and store the VR video of the demonstration process. Then, students can play the VR video through the virtual reality device at any time to reproduce the demonstration process in the virtual reality for learning.

However, when students practice (such as disassembling and assembling mechanical equipment, assembling models, assembling furniture, etc.) following the playing VR videos, without the immediate supervision of teachers, students cannot immediately confirm whether the exercise results are correct, which greatly reduces learning efficiency . Therefore, the existing virtual reality teaching system can only provide a one-way learning function, and cannot verify the students' real-time learning status.

Therefore, the existing virtual reality teaching system has the above-mentioned problems, and more effective solutions are urgently needed to be proposed.

The main purpose of the present invention is to provide a virtual reality system with a disassembly inspection function and a disassembly inspection method based on virtual reality, which can check the disassembly state of the learning terminal.

In one embodiment, a virtual reality system with a disassembly and inspection function includes a server end and a learning end. The server side is used to provide teaching disassembly records and inspection data. The check data includes the taught disassembly status of multiple virtual objects at each check point. The learning terminal includes a learning terminal module and a learning terminal virtual reality device. The virtual reality device at the learning end is used to play teaching disassembly records and accept operations. The learning terminal module is used to disassemble multiple virtual objects based on operations to change the learning disassembly status of multiple virtual objects, and when the teaching disassembly status of any detection point does not match the learning disassembly status, a disassembly will be issued. Error message.

In one embodiment, a virtual reality-based disassembly and inspection method includes: a) Obtaining a teaching disassembly record and a verification data set with a plurality of verification points at a learning terminal, wherein the verification data Including a teaching disassembly state of multiple virtual objects at each checkpoint; b) Play teaching disassembly records on the learning end; c) Accept operations on the learning end to disassemble and assemble multiple virtual objects to change multiple virtual objects and, d) when the teaching disassembly state at any detection point does not match the learning disassembly state, a disassembly error prompt is issued.

The present invention can immediately confirm whether the disassembly state of the learning terminal is correct, and can greatly improve the learning efficiency and accuracy of students.

A preferred embodiment of the present invention will be described in detail below in conjunction with the drawings.

In order to solve the problem that the existing virtual reality teaching system cannot verify the students' real-time disassembly and assembly status, the present invention proposes a virtual reality system with a disassembly and assembly inspection function and a disassembly and assembly inspection method based on virtual reality. Multiple checkpoints can be set in the teaching disassembly record recorded by the teacher to verify the student's disassembly status in real time when it is reproduced.

Please refer to FIG. 1A and FIG. 1B , FIG. 1A is a schematic diagram of the operation of the teaching terminal of the present invention, and FIG. 1B is a schematic diagram of the operation of the learning terminal of the present invention.

Before starting recording, the teacher 100 can first create a disassembly course (such as disassembly and assembly teaching of a robotic arm) in the virtual reality system, set the basic information of the disassembly course (such as course name, description, browsing rights, etc.), and create A plurality of virtual objects 112 required for the course (such as virtual end effectors, virtual arms, virtual bases, virtual fixtures, etc.).

Next, as shown in FIG. 1A , the teacher 100 can wear the teaching terminal virtual reality device 111 (including a display and a controller) to start recording the assembly and disassembly course.

Specifically, the teacher 100 can wear the teaching terminal virtual reality device 111 to assemble or disassemble a plurality of virtual objects 112 .

Moreover, the virtual reality system can record the above disassembly process of the teacher 100 to generate teaching disassembly records that can be reproduced in virtual reality. The aforementioned teaching disassembly record can be, for example, a VR video.

Furthermore, during the recording process or after the recording is completed, the teacher 100 can set checkpoints for one or more time points or disassembly steps of the teaching disassembly recording. Then, the virtual reality system will obtain the teaching and disassembly statuses of the plurality of virtual objects 112 at each check point and record them as check data. Teaching the state of disassembly and assembly may include the connection relationship, position, rotation and orientation of multiple virtual objects 112, etc.)

Finally, the virtual reality system can store the teaching disassembly records with checkpoints and the check data in the course storage space (such as storage server, network database or cloud hard drive, etc.) for students to download.

Next, as shown in FIG. 1B , taking learning only by watching as an example, the student 120 can download the above-mentioned teaching disassembly record and inspection data, and use the virtual reality device 121 of the teaching terminal to reproduce the teacher 110 in virtual reality. Disassembly process.

In addition, taking learning through asynchronous operation as an example, the student 130 can use the virtual reality device 131 of the teaching terminal to reproduce in virtual reality a plurality of virtual objects 132 that need to be used in teaching disassembly and assembly records.

Then, the student 130 can disassemble and assemble the plurality of virtual objects 132 while watching the reproduced disassembly and assembly process of the teacher 110 .

Moreover, when the teaching disassembly record is played to any inspection point, the virtual reality system of the present invention can automatically compare whether the learning disassembly status of the student 130 is consistent with the teaching disassembly status of the checkpoint, so as to verify in real time Whether the student 130 correctly learns the disassembly process.

Please refer to FIG. 2 . FIG. 2 is a structural diagram of a virtual reality system according to an embodiment of the present invention.

The virtual reality system 2 of this embodiment may include a server end 20 and a learning end 21 connected through a network.

The server 20 is operated by the course administrator, and can be a server, a cloud computing platform, etc., but not limited thereto.

The learning terminal 21 is operated by the student and can be a personal computer, a notebook computer, a tablet computer, a game console, a mobile device, etc., but not limited thereto.

The server end 20 is used to provide teaching disassembly records 201 and inspection data 202 . The aforementioned teaching disassembly record 201 can be set with one or more checkpoints. The check data 200 may include the teaching disassembly status of the multiple virtual objects used in the teaching disassembly record 201 at each check point.

The server 20 can include a server module 200 . The server module 200 is used to control the server 20 to perform subsequent data transmission, control and processing.

The learning terminal 21 may include a learning terminal virtual reality device 210 and a learning terminal module 211 .

The virtual reality device 210 at the learning end is used to play the teaching disassembly record 201 to reproduce the teacher's disassembly process, and accept the operation of the students to disassemble the displayed virtual object.

The learning terminal module 211 is used to control the learning terminal 21 to perform subsequent data transmission, control and processing.

In one embodiment, in the process of playing teaching and disassembling records, the learning terminal module 211 can disassemble and disassemble multiple virtual objects based on the student's operation to change the learning disassembly status of multiple virtual objects (such as multiple virtual objects The relative position, relative connection relationship, etc.), and when the teaching disassembly status of any detection point does not match the learning disassembly status, a disassembly error prompt will be issued to remind students that the current disassembly process is wrong.

Please refer to FIG. 3 . FIG. 3 is a structural diagram of a virtual reality system according to an embodiment of the present invention.

In this embodiment, the virtual reality system 3 may include a server end 20 , a learning end 21 , a network database 30 and a teaching end 31 .

The server end 20 can be connected to the network database 30 through a network (such as an internal network or the Internet), and can be connected to the learning end 21 and the teaching end 31 through the network.

The network database 30 can store teaching and disassembly records 201 , verification data 202 , account information 203 , learning disassembly records 204 and course templates 205 .

In one embodiment, the account data 203 may include the teacher account data used by the teaching terminal 31 and the student account data of the learning terminal 21 . Account information 203 may include basic information of teachers and/or students, account passwords, permissions, course records (such as uploaded courses or watched courses), etc., but not limited thereto.

In one embodiment, the learning disassembly record 204 is uploaded by the learning terminal 21 , and its content is the disassembly process made by the students after watching the teaching disassembly record 201 .

In one embodiment, a plurality of course templates 205 are used for downloading by the teaching terminal 31 . Multiple course templates 205 can respectively correspond to different types of courses, and a plurality of related virtual objects are pre-built for teachers to quickly create courses.

In one embodiment, the learning terminal 21 may include a learning terminal virtual reality device 210 , a learning terminal module 211 , a processing unit 212 , a communication unit 213 , a storage unit 214 and a man-machine interface 215 .

In one embodiment, the learning terminal virtual reality device 210 may include a display 2100 and a controller 2101 .

The teaching terminal 31 can be a computer device similar to the learning terminal 21, the only difference is that it is operated by different users.

In one embodiment, the teaching terminal may include a teaching terminal virtual reality device 310 , a teaching terminal module 311 , a processing unit 312 , a communication unit 313 , a storage unit 314 and a man-machine interface 315 .

In one embodiment, the teaching end virtual reality device 310 may include a display 3100 and a controller 3101 .

In one embodiment, the virtual reality device 210 at the learning end and the virtual reality device 310 at the teaching end can be any existing virtual reality devices on the market. The displays 2100 and 3100 are used to present virtual reality images. The controllers 2101, 3101 (such as a control handle) is used to receive user operations to operate the virtual reality.

In one embodiment, the processing units 211 and 311 may be circuits with processing capabilities such as processors and microcontrollers. The communication units 213 and 214 (such as network interface cards) are used to connect to the network and transmit data through the network. The storage units 214 and 314 may include volatile memory (such as RAM) and non-volatile memory (such as ROM, EEPROM, solid state hard disk, magnetic disk hard disk, flash memory, etc.) for storing data. The man-machine interface 215, 315 may include an input interface (such as a keyboard, a mouse, a touch pad, etc.) and an output interface (such as a display, a speaker, etc.) for accepting user operations.

In one embodiment, the virtual reality system 3 of the present embodiment can provide a teaching material production function (realized by the teaching terminal 31 and the server terminal 20) and an interactive learning function (realized by the server terminal 20 and the learning terminal 21) accomplish).

The data generated by the teaching material production function (such as teaching disassembly records and inspection data) can be used in the interactive learning function. Moreover, the data generated by the interactive learning function (such as learning disassembly records) can be imported into the teaching material production function after being analyzed and processed. Thereby, the present invention can form an organic teaching material dynamic circulation system.

In the aforementioned teaching material production function, the teacher can operate the teaching terminal 31 to edit course information and upload course teaching materials (such as teaching disassembly record 201 and inspection data 202, which can include text data, numerical data, image data and 3D models, etc.) , and set the course collaboration mode (including single-person collaboration, multi-person collaboration, single-person autonomy and multi-person synchronization, etc.).

In addition, the teacher can operate the teaching terminal 31 to set the course execution mode (for example, the teacher can set the instruction mode, guidance mode, and exploration mode according to the dimensions of the strong scaffold to the weak scaffold, and can set tests and evaluations according to the measurement purpose, etc.) 1. Select the course template 205 (such as the teacher can arrange the teaching content according to the structural paragraphs recommended by the system to produce courses with high teaching effectiveness), record the teaching disassembly record 201 (such as recording the teacher’s voice in the virtual environment, all behaviors Actions, use system tools to mark symbols in 3D space, brush notes, key reminders, etc., and can be taught with course materials at the same time), and set check points and check data 202 (for example, the system detects the relationship between virtual objects Automatically generate checkpoints when changing, or set checkpoints manually by the teacher).

Then, after the teaching disassembly record 201 and the verification data 202 are prepared, the teaching terminal 31 can upload the teaching disassembly record 201 and the verification data 202 to the network database 30 through the server 20 to complete the release of the course.

In the aforementioned interactive learning function, students can operate the learning terminal 21 to download courses (teaching and disassembly records 202 and check data), and play the teaching and disassembly records 202 through the virtual reality device 210 of the learning terminal to observe the teacher's disassembly process , and can operate the learning end virtual reality device 210 synchronously or afterwards to simulate the disassembly process.

During the process of simulating the assembly and disassembly process, the learning terminal module 211 can automatically check the learning disassembly and assembly status of each check point, and give immediate feedback (such as notifying that the action is correct or wrong).

Moreover, the entire imitation disassembly process will be recorded as a learning disassembly record 204 and uploaded to the network database 30 through the server 20 for subsequent analysis.

Please refer to FIG. 4 . FIG. 4 is a structural diagram of a teaching terminal module, a server terminal module and a learning terminal module according to an embodiment of the present invention.

In one embodiment, the teaching terminal module 211 may include a course creation module 400 for realizing different functions, a teaching terminal interaction module 401, a teaching terminal recording module 402, a check setting module 403 and a teaching terminal feedback module Group 404.

The course creation module 400 is configured to create courses and set course information.

The teaching-end interaction module 401 is configured to provide teachers to interact with virtual objects through the teaching-end virtual reality device 310 .

The teaching end recording module 402 is set to record the teacher's disassembly process to generate the teaching disassembly record 201 .

The check setting module 403 is configured to set check points in the teaching disassembly record 201 and generate check data 202 of the check points.

The teaching end feedback module 404 is configured for the teacher to provide feedback information (such as created virtual objects, course settings or operating suggestions, etc.) to the server end 20 .

In one embodiment, the server-side module 200 may include an account management module 410 , a teaching management module 411 and a record management module 412 for realizing different functions.

The account management module 410 is configured to manage the account information 203 and verify the login behavior of the learning terminal 21 and the teaching terminal 31 .

The teaching management module 411 is configured to manage the teacher's course-related settings and the learning progress of the students.

The record management module 412 is configured to manage the teaching disassembly record 201 , the check data 303 and the learning disassembly record 204 .

In one embodiment, the teaching terminal module 211 may include a recording and playback module 420 for realizing different functions, a learning terminal interaction module 421, a learning terminal recording module 422, a verification execution module 423, and a learning terminal feedback module. group 424 and synchronous learning module 425 .

The record and playback module 420 is set to play the downloaded teaching and disassembly record 201 , and can also play the recorded learning and disassembly record 204 .

The learning-end interaction module 421 is configured to provide students to interact with virtual objects through the learning-end virtual reality device 210 . The aforementioned interactions may include 3D annotation, notes, content export, recording learning content, whiteboard sharing, data sharing, etc.

The learning end recording module 422 is set to record the disassembly process of the students to generate the learning disassembly record 204 .

The check execution module 423 is set to compare the students' actions, results and virtual values in the virtual reality with the check data at each check point, and provide operation feedback and suggestions.

The learning end feedback module 424 is configured for students to provide feedback information (such as course suggestions or operation opinions, etc.) to the server end 20 .

The synchronous learning module 425 is set to realize the collaborative operation between multiple learning terminals 21 or between the teaching terminal 31 and the learning terminal 21 to realize online multi-person interaction in the virtual environment.

It is worth mentioning that the modules 311, 200, and 211 shown in FIG. 4, the modules 400-404, the modules 410-412, and the modules 420-425 are connected to each other (can be It is an electrical connection or an information connection), and it can be a hardware module (such as an electronic circuit module, an integrated circuit module, a SoC, etc.), a software module, or a mix of software and hardware modules, without limitation.

When the aforementioned modules are software modules (such as firmware, operating system or application program), the corresponding storage unit (such as the storage of the server terminal 20, the storage unit 214 of the learning terminal 21 and the storage unit 314 of the teaching terminal 31 ) may include a non-transitory computer-readable recording medium. The aforementioned non-transitory computer-readable recording medium stores a computer program, and the computer program records computer-executable codes. The controller, the processing unit 212 of the learning terminal 21 and the processing unit 312 of the teaching terminal 31) execute the aforementioned program codes to realize the control functions of the aforementioned modules.

Please refer to FIG. 5 . FIG. 5 is a flowchart of a disassembly and inspection method according to an embodiment of the present invention. The disassembly and inspection methods of each embodiment of the present invention can be implemented by the virtual reality system of any embodiment.

The disassembly and inspection method of this embodiment may include steps S10-S16.

In step S10 , the learning terminal 21 obtains the teaching disassembly record 201 and the corresponding verification data 202 through the server terminal 20 . The teaching and disassembly record 201 is set with one or more checkpoints, and the check data 202 includes the teaching and disassembly status of multiple virtual objects in the teaching and disassembly record 201 at each checkpoint.

In step S11 , the student starts to disassemble and assemble a plurality of virtual objects through the learning terminal virtual reality device 210 .

In one embodiment, the virtual reality device 210 at the learning end can play the teaching disassembly record 201 for students to perform the disassembly process according to the screen.

In step S12 , the learning terminal 21 detects the real-time learning disassembly status through the checking execution module 423 .

Specifically, when students can disassemble and disassemble multiple virtual objects through the virtual reality device 210 of the learning terminal to change the learning disassembly and assembly status of multiple virtual objects, the learning terminal module 211 can detect the aforementioned changes in real time To obtain the latest learning disassembly status of multiple virtual objects.

While executing step S12, in step S13, the learning terminal 21 determines whether the current playback progress has reached any detection point of the teaching disassembly record 201 through the checking execution module 423.

If no detection point is detected, the learning terminal 21 executes step S16.

If the detection point is detected, the learning terminal 21 executes step S14. In step S14 , the learning terminal 21 obtains the teaching disassembly status of the check data 202 at the current detection point through the check execution module 423 , and compares whether the teaching disassembly status is consistent with the current learning disassembly status.

If the teaching disassembly state is consistent with the current learning disassembly state, the learning terminal 21 executes step S16.

If the teaching disassembly state does not match the current learning disassembly state, the learning terminal 21 executes step S15. In step S15 , the learning terminal 21 issues disassembly error prompts (such as voice prompts, text prompts, or video prompts, etc.) through the check execution module 423 to instantly remind students that the current disassembly process is wrong.

Next, step S16 is executed. In step S16, the learning terminal 21 judges whether to end the disassembly, such as whether the teaching disassembly record 201 has been played or exited, whether the student has turned off the virtual reality device 210 of the learning terminal, and so on.

If the learning terminal 21 judges that the assembly and disassembly is finished, then the assembly and disassembly inspection method ends. Otherwise, the learning terminal 21 performs step S12 and step S13 again.

The present invention can immediately confirm whether the disassembly state of the learning terminal is correct, and can greatly improve the learning efficiency and accuracy of students.

Please refer to FIG. 5 and FIG. 6 at the same time. FIG. 6 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method in FIG. 5 , the disassembly and inspection method of this embodiment includes steps S20-S24 before starting the disassembly (step S12).

In step S20 , the learning terminal 212 determines the position of the virtual hand in the virtual reality space by positioning the virtual hand through the learning terminal interaction module 421 .

In one embodiment, students can operate the controller 2101 of the learning terminal virtual reality device 210, so that the learning terminal 212 can operate based on the student's (user's) operation (such as moving the controller 2101 in the real space or operating the controller 2101) to correspond to the position of moving the virtual hand in the virtual reality space.

In step S21 , the learning terminal 212 sets the physical attributes of a plurality of virtual objects based on the teaching disassembly record 201 through the record playback module 420 .

In step S22 , the learning terminal 212 initializes the renderer component based on the teaching disassembly record 201 through the recording and playback module 420 , so as to complete the preparation for rendering the virtual object in the virtual reality environment.

In step S23 , the learning terminal 212 initializes the collider component based on the teaching disassembly record 201 through the record playback module 420 , so as to complete the preparation for detecting object collision in the virtual reality.

In step S24 , the learning terminal 212 reproduces the teacher's operation process by playing the teaching disassembly record 201 on the display 2100 of the virtual reality device 210 of the learning terminal through the record playback module 420 .

In this way, the present invention can automatically reproduce the teaching disassembly record 201, and automatically build a virtual interactive environment for students.

Please refer to FIG. 5 to FIG. 7 at the same time. FIG. 7 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method in FIG. 5 , the step S12 of the disassembly and inspection method of this embodiment may include steps S30-S33.

In step S30 , the learning terminal 21 creates a boundary table through the learning terminal interaction module 421 . The aforementioned boundary table is used to record the boundaries of each virtual object, and whether each virtual object contacts or collides with other virtual objects can be determined by detecting whether the aforementioned boundaries overlap.

In step S31, the learning terminal 21 sets collision detection for each virtual object through the learning terminal interaction module 421, so as to detect whether there is a collision between the virtual objects or whether the virtual hand collides with the virtual object (collision detection result) .

In one embodiment, virtual objects may include strokes (such as 3D annotations), line segments (such as cables) and regular objects (such as various components).

The present invention can perform different collision detections for different types of virtual objects through steps S310 and S311, so as to improve the accuracy and efficiency of collision detection.

In step S310, when the virtual object is a line segment or a stroke, the learning terminal 21 detects the collision of the line segment or stroke through the learning terminal interactive module 421 through the line renderer, such as determining the boundary of the line segment or stroke based on the rendering boundary of the line renderer .

In step S311, when the virtual object is a normal object, the learning terminal 21 calculates the boundary of the normal object through the learning terminal interactive module 421 through the grid renderer, and detects the collision of the normal object based on the boundary, such as based on the grid renderer The rendering bounds of determine the bounds of regular objects.

In step S32, the learning terminal 21 captures the collision detection result detected in step S31 by the learning terminal interaction module 421.

In one embodiment, when the virtual hand collides with the virtual object, the learning terminal 21 determines that the virtual hand grasps the virtual object, and controls the grasped virtual object to move along with the virtual hand.

In one embodiment, when the virtual hand collides with the virtual object and the grab button is triggered, the learning terminal 21 determines that the virtual hand grabs the virtual object, and controls the grabbed object to move with the virtual hand until the object is detected ( if the grab key is released or pressed again).

In step S33 , the learning terminal 21 performs assembly detection based on the collision detection result detected in step S31 through the learning terminal interaction module 421 .

In one embodiment, when the grasped virtual object collides with other virtual objects, the learning terminal 21 assembles the collided virtual object and controls the grasped virtual object to be fixed at the assembly position.

In one embodiment, the learning terminal 21 disassembles the collided virtual object when the captured virtual object and other virtual objects change from a collision state to a separated state, and controls the captured virtual object hand moves.

In one embodiment, the aforementioned determination of assembly and disassembly takes effect only when the assembly button or the disassembly button is triggered simultaneously.

Therefore, the present invention allows users to assemble and disassemble virtual objects in a virtual reality environment.

Please refer to FIG. 13 . FIG. 13 is a data structure of a virtual object according to an embodiment of the present invention.

In one embodiment, a virtual object (especially a regular object) has a data structure as shown in FIG. 13 , where + is an internal component, and - is an external component.

The main category (game object) of the virtual object in FIG. 13 has rigid body (rigid body) components and graspable (grabbable) components, that is, rigid body and graspable two attributes that endow virtual objects with collision effects.

The virtual object in FIG. 13 also has a mesh sub-object and a collider sub-object. The mesh sub-object is used to represent the rendering boundary, and the collider sub-object is used to represent the collision boundary.

Please refer to FIG. 14A and FIG. 14B , FIG. 14A is a schematic diagram of a virtual hand according to an embodiment of the present invention, and FIG. 14B is a schematic diagram of a virtual hand grasping a virtual object according to an embodiment of the present invention.

In order to allow users to freely pick up and operate virtual objects in the virtual reality, the present invention can set the anchor point 50 of the virtual hand in the virtual reality, by judging whether the anchor point 50 enters the boundary of the virtual object 51 (such as a bounding box) 510) to determine whether the virtual hand grabs the virtual object 51.

Please refer to FIG. 8 and FIG. 15A to FIG. 15D. FIG. 8 is a partial flow chart of a disassembly and inspection method according to an embodiment of the present invention. FIG. 15A is a first schematic diagram of a calculation object bounding box according to an embodiment of the present invention. FIG. 15B FIG. 15C is a third schematic diagram of a computing object bounding box according to an embodiment of the present invention. FIG. 15D is a fourth schematic diagram of a computing object bounding box according to an embodiment of the present invention. schematic diagram.

As shown in Figure 15A, in the current 3D boundary calculation method, when the virtual object 52 rotates, the grid boundary (bounding box 520) will automatically calculate the maximum value, so that the actual boundary between the bounding box 520 and the virtual object 52 has obvious drop, resulting in inaccurate capture events.

In this regard, the present invention proposes a 3D boundary calculation method, which can continuously calculate the minimum bounding box of the virtual object 52 to be grasped each time the virtual object 52 is picked or rotated, so as to greatly improve the accuracy of the grasping event.

The aforementioned 3D boundary computing method may include steps S40-S43.

In step S40 , the teaching end 31 or the learning end 21 acquires the object pose of the virtual object 52 (eg, a normal object). The aforesaid object posture includes the rotation angle of the virtual object 52, and the rotation angle can be represented by a Euler angle, but not limited thereto.

In the example of FIG. 15A, the position of the virtual object 52 is BP (0, 0, 0), the rotation angle is BR (0, 0, 45), and the size is BS (2, 2, 2).

In step S41 , the teaching end 31 or the learning end 21 calculates the bounding box 521 of the virtual object 52 in the non-rotation state.

In the example of FIG. 15B , the non-rotated bounding box 521 of the virtual object 52 can be obtained through the Quaternion.identity function of unity 3D, but it is not limited thereto.

In step S42 , the teaching end 31 or the learning end 21 rotates the bounding box 521 based on the object pose to obtain the rotated bounding box 521 .

In the example of FIG. 15C , the teaching end 31 or the learning end 21 can calculate the vector of the vertices P0-P7 of the bounding box 521, and rotate the bounding box 521 with the vector to obtain the rotated bounding box 521 as shown in FIG. 15D.

In step S43 , the teaching end 31 or the learning end 21 uses the rotated bounding box 521 as the object bounding box of the virtual object (such as a regular object). The aforementioned object bounding box serves as the collision boundary of the virtual object 52 and is used to detect the collision event of the virtual object 52 . For example, when other virtual objects enter the collision boundary of the virtual object 52, it is determined that the virtual object 52 collides with other virtual objects.

For example, as shown in FIG. 15C , when the positioning point PV of the virtual hand enters the object bounding box 521 of the virtual object 52 , it can be determined that the virtual hand collides with the virtual object 52 .

Please refer to FIG. 5 to FIG. 9 . FIG. 9 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method shown in FIG. 5 , the disassembly and inspection method of this embodiment may include steps SS50 - S54 for realizing assembly and disassembly functions.

In step S50 , the teaching terminal 31 or the learning terminal 21 can accept the user's moving operation through the teaching terminal virtual reality device 310 or the learning terminal virtual reality device 210 to move the virtual object grasped by the virtual hand.

Next, the teaching terminal 31 or the learning terminal 21 can execute step S51 and step S53.

In step S51 , the teaching end 31 or the learning end 21 detects whether the preset assembly conditions are satisfied.

In one embodiment, the assembling condition includes that the virtual object grasped by the virtual hand collides with other virtual objects.

In one embodiment, the assembly condition further includes that the assembly key of the controller 3101 or the controller 2101 is pressed.

If the assembling condition is not satisfied, the teaching end 31 or the learning end 21 ends the detection.

If the assembling condition is satisfied, the teaching end 31 or the learning end 21 executes step S52. In step S52 , the teaching end 31 or the learning end 21 executes group processing to assemble a plurality of colliding virtual objects into a group object (first group object).

In one embodiment, the group processing may include steps S520-S521.

In step S520 , the teaching end 31 or the learning end 21 generates a graspable group object, and sets a plurality of virtual objects that collide with as a plurality of sub-objects of the group object.

In step S521 , the teaching terminal 31 or the learning terminal 21 sets multiple sub-objects of the group object as ungraspable.

Please refer to FIG. 16A to FIG. 16B , FIG. 16A is a first schematic diagram of group processing according to an embodiment of the present invention, and FIG. 16B is a data structure after group processing according to an embodiment of the present invention.

As shown in FIGS. 16A and 16B , if there are three virtual objects A, B, and C in the virtual reality environment, the user can assemble them sequentially.

When virtual object A touches virtual object B, when the collider objects of the two generate collision detection (On Collision Enter), a new virtual object will be created, that is, group object AB.

The new group object AB will add a Grabbable (grabable) component, make the virtual object A and virtual object B the sub-objects of the group object AB, and set the Grabbable components of virtual object A and virtual object B to be invalid, so that Prevent further detection.

Next, when the group object AB touches the virtual object C, the above settings are also performed to obtain the group object ABC.

In this way, the tree view of group objects will grow from bottom to top, and when a new group object is created, only the Grabbable component of the topmost node (such as virtual object ABC) will be activated.

Please refer to FIG. 17A and FIG. 17B , FIG. 17A is a second schematic diagram of group processing according to an embodiment of the present invention, and FIG. 17B is a third schematic diagram of group processing according to an embodiment of the present invention.

As shown in Figure 17A, regular objects are composed of a Mesh object and a Collider. When the objects touch, that is, when the collision range (white grid area) of two objects (vfd5a5ms43ansaa_001 and vfd5a5ms43ansaa_002) touches, That is, group processing is performed.

As shown in Figure 17B, after the group judgment is established, a new group object (vfd5a5ms43ansaa_001&vfd5a5ms43ansaa_002) is added, and the two objects (vfd5a5ms43ansaa_001 and vfd5a5ms43ansaa_002) are placed in the sub-level of the new virtual object, and the Grabbable components of the two objects are canceled.

In addition, the Grabbable component on the new group object will integrate the collision range of all child objects into a new collision range (white grid area), and the grabbing range (gray grid area) is the same.

Referring again to FIG. 9 , in step S53 , the teaching terminal 31 or the learning terminal 21 detects whether the preset disassembly condition is satisfied.

In one embodiment, the dismantling condition includes that the virtual object grasped by the virtual hand is separated from other virtual objects.

In one embodiment, the dismantling condition further includes that the dismantling key of the controller 3101 or 2101 is pressed.

If the dismantling condition is not met, then the teaching end 31 or the learning end 21 ends this detection.

If the dismantling condition is satisfied, the teaching terminal 31 or the learning terminal 21 executes step S54. In step S54 , the teaching end 31 or the learning end 21 performs ungrouping processing to separate the separated group objects (second group objects) into a plurality of virtual objects.

In one embodiment, the degrouping process may include steps S540-S541.

In step S540 , the teaching terminal 31 or the learning terminal 21 deletes multiple links between multiple sub-objects of the group object, and deletes the group object.

In step S541 , the teaching terminal 31 or the learning terminal 21 sets a plurality of child objects as multiple virtual objects that can be grasped.

Please refer to FIG. 18A to FIG. 19 , FIG. 18A is a first schematic diagram of degrouping processing according to an embodiment of the present invention, FIG. 18B is a second schematic diagram of degrouping processing according to an embodiment of the present invention, and FIG. 18C is a schematic diagram of degrouping processing according to an embodiment of the present invention. The third schematic diagram of the ungrouping process of the embodiment, FIG. 19 is the fifth schematic diagram of the ungrouping process of an embodiment of the present invention.

As shown in Figures 18A-18C and Figure 19, when the ungroup event occurs on the group object ABC, all non-original objects (the original objects are virtual objects A, virtual objects B, and virtual objects C, and the non-original objects are group The Grabbable components of object AB and group object ABC) will be deleted, and the original object will release the parent object hierarchy and enter the clickable state.

The aforementioned clickable state escapes the Grabbable capture function, allowing users to click on the object to be removed, and then drag to remove the group.

For example, if the user wants to separate the virtual object C, when the dismantling condition is satisfied, the virtual object C is confirmed to be separated, and the virtual object A and the virtual object B are grouped into the group object AB again because the distance has not changed.

Please refer to FIGS. 20A-20C , FIG. 20A is a data structure after degrouping processing according to an embodiment of the present invention, FIG. 20B is a sixth schematic diagram of degrouping processing according to an embodiment of the present invention, and FIG. 20C is an implementation of the present invention The seventh schematic diagram of the ungrouping process of the example.

As shown in FIG. 20A, when two objects (vfd5a5ms43ansaa_001 and vfd5a5ms43ansaa_002) are assembled and judged, a new group of objects (vfd5a5ms43ansaa_001&vfd5a5ms43ansaa_002) will be created, and both of them will be placed as child objects of the new object.

If the user wants to cancel the group object (vfd5a5ms43ansaa_001&vfd5a5ms43ansaa_002), he can click on the group object, and the system will cancel the parent object hierarchy of the two virtual objects and delete the parent object (group object).

Next, as shown in FIG. 20B , the user can select and remove the virtual object to be released.

Finally, as shown in FIG. 20C, the two virtual objects (vfd5a5ms43ansaa_001 and vfd5a5ms43ansaa_002) are restored to the state before grouping.

Through the above grouping and ungrouping processing, the present invention can effectively realize the assembly and disassembly of virtual objects in a virtual environment.

Please refer to FIG. 5 to FIG. 10 . FIG. 10 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method in FIG. 5 , the steps S14-S15 of the disassembly and inspection method of this embodiment may include steps S60-S67.

In step S60 , the learning terminal 21 detects whether the student's action on the virtual object is incorrect through the checking execution module 423 .

In one embodiment, the check data 202 may record the corresponding action type of the check point, such as an assembly action or a disassembly action. The learning terminal 21 can judge whether the current action is consistent with the verification data 202 through the verification execution module 423 .

If the detection action is correct, the learning terminal 21 executes step S62.

If the detection action is incorrect, the learning terminal 21 executes step S61. In step S61 , the learning terminal 21 issues an operation error prompt through the check execution module 423 .

In step S62 , the learning terminal 21 detects whether the composition of the virtual object currently completed by the student is correct through the checking execution module 423 .

In one embodiment, the check data can record the structure tree of each group object of the check point (the tree structure shown in FIG. 16 ). The learning terminal 21 can judge whether the composition of the current group object is consistent with the verification data 202 through the verification execution module 423 .

If the detected composition is correct, the learning terminal 21 executes step S64.

If the detected composition is incorrect, the learning terminal 21 executes step S63. In step S63 , the learning terminal 21 issues a composition error prompt through the checking execution module 423 .

In step S64 , the learning terminal 21 detects whether the current position of any virtual object is incorrect through the checking execution module 423 .

In one embodiment, the check data 202 can record the position of each virtual object of the check point. The learning terminal 21 can judge whether the current position of the virtual object is consistent with the verification data 202 through the verification execution module 423 .

In one embodiment, the present invention considers that each student has different assembling habits. Some people are used to assemble multiple virtual objects into group objects on the right hand side first, and then install the group objects in the correct position, while others are used to first Assembly is done on the left hand side. In order to eliminate the misjudgment of position errors that may be caused by the above-mentioned different assembly habits, an embodiment of the present invention uses the relative positions of virtual objects for detection.

Please refer to FIGS. 21-23. FIG. 21 is a schematic diagram of calculating relative positions according to an embodiment of the present invention. FIG. 22 is a data structure of relative positions according to an embodiment of the present invention. FIG. 23 is a recording group event according to an embodiment of the present invention. , Schematic diagram of solving group events and tool events.

As shown in FIG. 21 , in order to obtain the relative position between the virtual object A (object A) and the virtual object B (point B), the present invention can calculate the relative position of the virtual object B to the virtual object A.

In addition, since the position, rotation angle, and object size of the virtual object A may change the final value of the relative position, the position, rotation angle, and object size of the virtual object A must be considered together.

For example, the position BP of virtual object B is (0,3,1), the position AP of virtual object A is (2,4,0), the rotation angle AR is (0,0,0), and the object size AS is (1,1,1). The aforementioned relative position can be, for example, relative position coordinates (0, 3, 1)-(2, 4, 0).

As shown in FIG. 22 , the present invention also proposes an assembly format, which can record and store the relative position coordinates of two virtual objects.

As shown in Figure 23, when a group event occurs, such as virtual object A and virtual object B are assembled into a group object AB, the present invention can first focus on virtual object A, and calculate the relative relationship between virtual object B and virtual object A position, and then based on virtual object B, determine the relative position coordinates of virtual object A to virtual object B, and so on.

In another example, for group objects ABC, the present invention can first focus on virtual object A, calculate the relative position of virtual object B and virtual object C to virtual object A, and use virtual object B as the main one to determine the virtual object The relative position coordinates of virtual object A and virtual object C to virtual object B, and finally virtual object C is the main point to determine the relative position of virtual object A and virtual object B to virtual object C.

When the ungroup event occurs, such as group object ABC ungrouped into group object AB and virtual object C, the present invention can delete all relative positions to make virtual object A, virtual object B, and virtual object C independent virtual objects object A and virtual object B, and execute a group event on virtual object A and virtual object B to establish the relative position of virtual object A and virtual object B.

When a tool event occurs, it is only necessary to record the position of the virtual tool D.

If the detected position is correct, the learning terminal 21 executes step S66.

If the detected position is incorrect, the learning terminal 21 executes step S65. In step S65 , the learning terminal 21 issues a position error prompt through the checking execution module 423 .

In step S66 , the learning terminal 21 detects whether the assembly sequence of any virtual object is incorrect through the checking execution module 423 .

In one embodiment, the check data can record the assembly sequence of each virtual object with check points. The learning terminal 21 can judge whether the assembly sequence of the current virtual object is consistent with the verification data through the verification execution module 423 .

If the detection sequence is correct, the learning terminal 21 ends the detection.

If the detection sequence is incorrect, the learning terminal 21 executes step S67. In step S67 , the learning terminal 21 issues a sequence error prompt through the check execution module 423 .

It is worth mentioning that there are four types of action checks in steps S60-S61, composition checks in steps S62-S63, position checks in steps S64-S65, and sequence checks in steps S66-S67 proposed in this embodiment. There is no absolute order between checks, and those skilled in the technical field of the present invention can change the check order or check content arbitrarily according to requirements (such as deleting component checks).

Please refer to FIG. 5 to FIG. 11 . FIG. 11 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method in FIG. 5 , the disassembly and inspection method of this embodiment further includes steps S70 - S74 for generating a teaching disassembly record 201 on the teaching terminal 31 .

In step S70 , the teaching terminal 31 executes an initialization process.

In one embodiment, the initialization processing may be, for example, the same or similar processing as steps S20 - S23 in FIG. 6 .

In step S71 , the teaching terminal 31 accepts the teacher's operation through the virtual reality device 310 of the teaching terminal, and creates one or more virtual objects and line segments based on the operation.

In one embodiment, the teacher can input an object creation operation to create one or more virtual objects.

In one embodiment, the teacher can input a line segment creation operation to create a line segment connecting multiple virtual objects.

For example, when the multiple virtual objects are virtual electronic devices, the teacher can create line segments between the multiple virtual objects to simulate connecting multiple virtual electronic devices.

In step S72 , the teaching terminal 31 accepts the teacher's operation through the virtual reality device 310 of the teaching terminal, and changes the teaching detachment status of a plurality of virtual objects based on the teaching detachment operation.

Specifically, after creating the virtual objects and line segments, the teacher can operate the teaching terminal virtual reality device 310 to demonstrate the assembly and disassembly process.

Moreover, during the demonstration of the disassembly process, the teaching and disassembly states of multiple virtual objects will change continuously, and the teaching terminal 31 can continuously record the changes of the teaching and disassembly states.

In step S73 , the teaching terminal 31 sets checkpoints and records the teaching and dismounting status of each checkpoint.

In step S74 , after the demonstration of the disassembly process is completed, the teaching end 31 generates a teaching disassembly record, which can be further released to the server end 20 .

In this way, the present invention allows teachers to quickly generate teaching disassembly records.

Please refer to FIG. 5 to FIG. 12 . FIG. 12 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Compared with the disassembly and inspection method in FIG. 11 , step S71 of the disassembly and inspection method of this embodiment further includes steps S80 - S82 for generating virtual line segments.

In step S80 , the teacher may input a line segment creation operation, and the teaching end 31 creates a line segment with matching nodes based on the head node number, body node number, and tail node number of the line segment creation operation.

In one embodiment, the aforementioned line segment creation operation includes setting the number of head nodes, the number of body nodes and the number of tail nodes.

In one embodiment, the head node and the tail node of the above-mentioned line segment are used to connect virtual objects, and are used to meet wiring requirements.

Please refer to FIG. 24 . FIG. 24 is a schematic diagram of multiple line segment types according to an embodiment of the present invention.

In this embodiment, the line segments can be one-to-one (the number of head nodes is 1, and the number of tail nodes is 1), one-to-many (the number of head nodes is 1, and the number of tail nodes is more than 2), many-to-one (head nodes The number is more than 2, and the number of tail nodes is 1), or many-to-many (the number of head nodes and the number of tail nodes are connected to be more than 2).

Please refer to FIG. 25 . FIG. 25 is a data structure of a line segment according to an embodiment of the present invention.

In this embodiment, the original line segment includes two endpoints (Node) and a line (Wire).

In order to make the endpoints of the line segments graspable, in this embodiment, the Grabbable component is used to replace the original line segment endpoints (Nodes) in the combination of line segments, so as to achieve the routing function.

Referring again to FIG. 12 , in step S81 , the teaching end 31 sets the harness relationship of the line segment to convert multiple body nodes into a harness.

Please refer to FIG. 26 . FIG. 26 is a schematic diagram of a many-to-many line segment according to an embodiment of the present invention.

Since all line segments will not have a ring state, this embodiment allows the user to directly input (number of head nodes, number of body nodes, number of tail nodes) = (2, 1, 2) to quickly generate corresponding line segments.

Please refer to FIG. 27 . FIG. 27 is a data structure of a wire harness according to an embodiment of the present invention.

This embodiment considers the situation that a bundle may occur in actual wiring, and proposes a set of bundle categories to create line segments under the bundle level.

Please refer to FIGS. 26-28 . FIG. 28 is a structure diagram of a generated many-to-many line segment according to an embodiment of the present invention.

When the user inputs (the number of head nodes, the number of body nodes, the number of tail nodes) = (2, 1, 2), the teaching end 31 can set each wire bundle as two line segments.

Referring again to FIG. 12 , in step S82 , the teaching end 31 sets the coordinates of each node of the line segment.

Please refer to FIGS. 28-30 . FIG. 29 is a data structure of a many-to-many line segment generated according to an embodiment of the present invention, and FIG. 30 is a schematic diagram of a many-to-many line segment generated according to an embodiment of the present invention.

As shown in FIG. 30 , each node of the initially created line segment is like a general virtual object, and can be virtual grabbed.

Figure 29 shows the data structure in Figure 28. Each line segment initially created will follow a node coordinate to achieve the effect of wiring.

Please refer to Figures 31-33, Figure 31 is a structure diagram of a recorded many-to-many line segment according to an embodiment of the present invention, Figure 32 is a data structure of a recorded many-to-many line segment according to an embodiment of the present invention, and Figure 33 is a structure diagram of the recorded many-to-many line segment according to an embodiment of the present invention. A schematic diagram of a recorded many-to-many line segment of an embodiment.

As shown in the figure, when recording, the objects created by the line segment module can disassemble the graspable node objects, and create a substitute empty object to record the coordinates, so as to ensure that there will be no errors during playback.

Please refer to FIGS. 34-36. FIG. 34 is a structure diagram of a many-to-many line segment played back in an embodiment of the present invention. FIG. 35 is a data structure of a many-to-many line segment played back in an embodiment of the present invention. FIG. A schematic diagram of a many-to-many line segment for playback according to an embodiment.

As shown in Figure 36, when playing back, its structure is exactly the same as that of Figure 33.

Moreover, comparing Figures 34-35 with Figures 31-32, the connection relationship and position of the line module will not change, that is, no error will occur.

The present invention can ensure correct recording and playback functions of the line segment module by formulating the strict object structure of the line segment module during recording and playback.

The above descriptions are only preferred specific examples of the present invention, and are not intended to limit the patent scope of the present invention. Therefore, all equivalent changes made by using the content of the present invention are all included in the scope of the present invention in the same way. Chen Ming.

100: Teacher 111: Teaching end virtual reality device 112: Virtual objects 120: Student 121: Learning terminal virtual reality device 130: Student 131: Learning terminal virtual reality device 132: Virtual objects 2: Virtual reality system 20: Server side 200: Server side modules 201: Teach disassembly record 202: Check data 203: Account Information 204: Learning Disassembly Records 205: Course Templates 21: Learning end 210: Learning terminal virtual reality device 2100: display 2101: Controller 211: Learning terminal module 212: processing unit 213: storage unit 214: Communication unit 215: Man-machine interface 3: Virtual reality system 30: Network Database 31: teaching end 310: Teaching end virtual reality device 3100: display 3101: Controller 311: teaching end module 312: processing unit 313: Communication unit 314: storage unit 315: Human-machine interface 400: Course Creation Module 401: Teaching terminal interactive module 402: teaching end recording module 403: Check the setting module 404: Teaching end feedback module 410: Account Management Module 411: Teaching Management Module 412: Record Management Module 420: Record playback module 421: Learning Terminal Interactive Module 422: Learning terminal recording module 423: Check execution module 424: Learning End Feedback Module 425: Synchronous Learning Module 50: anchor point 51: Virtual objects 510: Bounding box 52: Virtual objects 520: Bounding box 521: Bounding box P0-P7: vertices PV: anchor point S10-S16: Disassembly and inspection steps S20-S24: initialization and playback steps S30-S33: virtual disassembly steps S310-311: Collision detection steps S40-S43: Calculating Boundary Steps S50-S54: virtual disassembly steps S520-S521: Assembly steps S540-S541: Disassembly steps S60-S67: Checking steps S70-S74: Teaching steps S80-S82: Create line segment steps

FIG. 1A is a schematic diagram of the operation of the teaching end of the present invention.

FIG. 1B is a schematic diagram of the operation of the learning terminal of the present invention.

FIG. 2 is a structural diagram of a virtual reality system according to an embodiment of the present invention.

FIG. 3 is a structural diagram of a virtual reality system according to an embodiment of the present invention.

FIG. 4 is a structural diagram of a teaching terminal module, a server terminal module and a learning terminal module according to an embodiment of the present invention.

FIG. 5 is a flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 6 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 7 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 8 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 9 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 10 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Fig. 11 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

Fig. 12 is a partial flowchart of a disassembly and inspection method according to an embodiment of the present invention.

FIG. 13 is a data structure of a virtual object according to an embodiment of the present invention.

FIG. 14A is a schematic diagram of a virtual hand according to an embodiment of the present invention.

FIG. 14B is a schematic diagram of a virtual hand grabbing a virtual object according to an embodiment of the present invention.

FIG. 15A is a first schematic diagram of computing object bounding boxes according to an embodiment of the present invention.

FIG. 15B is a second schematic diagram of computing object bounding boxes according to an embodiment of the present invention.

FIG. 15C is a third schematic diagram of computing object bounding boxes according to an embodiment of the present invention.

FIG. 15D is a fourth schematic diagram of computing object bounding boxes according to an embodiment of the present invention.

FIG. 16A is a first schematic diagram of group processing according to an embodiment of the present invention.

FIG. 16B is a data structure after group processing according to an embodiment of the present invention.

FIG. 17A is a second schematic diagram of group processing according to an embodiment of the present invention.

FIG. 17B is a third schematic diagram of group processing according to an embodiment of the present invention.

FIG. 18A is a first schematic diagram of a degrouping process according to an embodiment of the present invention.

FIG. 18B is a second schematic diagram of the degrouping process according to an embodiment of the present invention.

FIG. 18C is a third schematic diagram of the degrouping process according to an embodiment of the present invention.

FIG. 19 is a fifth schematic diagram of degrouping processing according to an embodiment of the present invention.

FIG. 20A is a data structure after ungrouping processing according to an embodiment of the present invention.

FIG. 20B is a sixth schematic diagram of the degrouping process according to an embodiment of the present invention.

FIG. 20C is a seventh schematic diagram of the degrouping process according to an embodiment of the present invention.

FIG. 21 is a schematic diagram of calculating relative positions according to an embodiment of the present invention.

FIG. 22 is a data structure of relative positions according to an embodiment of the present invention.

FIG. 23 is a schematic diagram of recording group events, ungroup events and tool events according to an embodiment of the present invention.

Fig. 24 is a schematic diagram of multiple line segment types according to an embodiment of the present invention.

Fig. 25 is a data structure of a line segment according to an embodiment of the present invention.

Fig. 26 is a schematic diagram of a many-to-many line segment according to an embodiment of the present invention.

Fig. 27 is a data structure of a wire harness according to an embodiment of the present invention.

FIG. 28 is a structural diagram of a generated many-to-many line segment according to an embodiment of the present invention.

Fig. 29 is a data structure of many-to-many line segments generated according to an embodiment of the present invention.

FIG. 30 is a schematic diagram of generating many-to-many line segments according to an embodiment of the present invention.

Fig. 31 is a structure diagram of a recorded many-to-many line segment according to an embodiment of the present invention.

Fig. 32 is a data structure of recorded many-to-many line segments according to an embodiment of the present invention.

Fig. 33 is a schematic diagram of recorded many-to-many line segments according to an embodiment of the present invention.

FIG. 34 is a structural diagram of a many-to-many line segment playback according to an embodiment of the present invention.

Fig. 35 is a data structure of many-to-many line segments played back according to an embodiment of the present invention.

Fig. 36 is a schematic diagram of a many-to-many line segment played back according to an embodiment of the present invention.

S10-S16: Disassembly and inspection steps

Claims (20)

A virtual reality system with a disassembly and inspection function, comprising: A server end is used to provide a teaching disassembly record and a check data, wherein the check data includes a teaching disassembly status of a plurality of virtual objects at each check point; and A learning terminal includes a learning terminal module and a learning terminal virtual reality device. The learning terminal virtual reality device is used to play the teaching disassembly record and accept operations. A virtual object is disassembled to change a learning disassembly state of the plurality of virtual objects, and when the teaching disassembly state of any of the detection points is inconsistent with the learning disassembly state, a disassembly error prompt is issued. The virtual reality system as described in claim 1, wherein the learning terminal module includes: A record playback module, which is used to play the teaching disassembly record through the learning terminal virtual reality device, and is used to set the physical properties of the plurality of virtual objects, initialize the renderer component, and initialize the collider component; A learning terminal interaction module, used to position a virtual hand and move the virtual hand based on the user's operation of the learning terminal virtual reality device; and A check execution module is used to detect whether the teaching disassembly state is consistent with the learning disassembly state at each detection point. The virtual reality system as described in claim 1, wherein the learning terminal module is configured to create a boundary table for recording a boundary of each virtual object; Wherein, the learning terminal module is set to detect a collision of the line segment or the stroke through the line renderer when any of the virtual objects is a line segment or a stroke; Wherein, the learning terminal module is configured to calculate the boundary of the normal object through a grid renderer when any of the virtual objects is a normal object, and detect a collision of the normal object based on the boundary; Wherein, the learning terminal module is set to perform a grasping detection and an assembly detection based on the collision detection result. The virtual reality system as described in claim 3, wherein the learning terminal module is configured to obtain an object pose of the normal object, calculate a bounding box of the virtual object in a non-rotation state, and rotate the virtual object based on the object pose bounding box to obtain a rotated bounding box, and use the rotated bounding box as an object bounding box of the regular object; Wherein, the object bounding box is used as a collision boundary of the normal object, and is used to detect the collision of the normal object. The virtual reality system as described in Claim 1, wherein the learning terminal module is configured to accept a moving operation through the learning terminal virtual reality device, and move at least one of the plurality of virtual objects based on the moving operation one; Wherein, the learning terminal module is set to execute a group process when an assembly condition is satisfied, so as to assemble the plurality of virtual objects that collide into a first group object; and Wherein, the learning terminal module is set to perform a degrouping process when a disassembly condition is met, so as to separate a second group of objects that has been separated into the plurality of virtual objects. The virtual reality system as described in claim 5, wherein the learning terminal module is configured to generate the first group of objects that can be grasped, and the plurality of virtual objects that collide are set as objects of the first group of objects multiple sub-objects, and set the multiple sub-objects of the first group of objects to be ungraspable. The virtual reality system as described in claim 5, wherein the learning terminal module is configured to delete multiple links between multiple sub-objects of the second group object, delete the second group object, and set the The sub-objects are the virtual objects that can be grasped. The virtual reality system as described in Claim 1, wherein the learning terminal module is set to issue an action error prompt when detecting an incorrect action on any virtual object, and to send out an action error prompt when detecting any virtual object When the composition is incorrect, issue a composition error prompt, issue a position error prompt when detecting an incorrect position of any of the virtual objects, or issue a sequence when detecting an incorrect assembly sequence of any of the virtual objects Error message. The virtual reality system as described in claim 1, further comprising a teaching terminal; Wherein, the teaching terminal includes a teaching terminal module and a teaching terminal virtual reality device; Wherein, the teaching end virtual reality device is used to display the plurality of virtual objects and accept operations; Wherein, the teaching terminal module is used to perform an initialization process, create the plurality of virtual objects based on an object creation operation, create a line segment connecting the plurality of virtual objects based on a segment creation operation, and create a line segment connecting the plurality of virtual objects based on a teaching disassembly operation to change the teaching disassembly state of the plurality of virtual objects, set the checkpoint and record the teaching disassembly state of the checkpoint, and generate the teaching disassembly record. The virtual reality system as described in claim item 9, wherein the line segment creation operation includes setting the number of a head node, the number of body nodes and the number of a tail node; Wherein, the teaching terminal module is used to create the line segment based on the number of the head node, the number of the body node and the number of the tail node, set a bundle relationship of the line segment to convert multiple body nodes into a bundle, and set the Multiple node coordinates; Wherein, at least one head node and at least one tail node of the line segment are respectively connected to the plurality of virtual objects. A disassembly and assembly inspection method based on virtual reality, comprising: a) Obtain a teaching disassembly record with multiple check points and a check data on a learning terminal, wherein the check data includes a teaching disassembly state of multiple virtual objects at each check point; b) Play the teaching disassembly record on the learning terminal; c) Accepting operations on the learning terminal to disassemble the plurality of virtual objects to change a learning disassembly state of the plurality of virtual objects; and d) When the teaching disassembly status of any of the detection points is inconsistent with the learning disassembly status, a disassembly error prompt is issued. The disassembly and inspection method as described in claim 11, which further includes before the step c): e1) positioning a virtual hand; e2) Setting physical attributes of the plurality of virtual objects; e3) Initialize the renderer component; and e4) Initialize the collider component. The disassembly inspection method as described in claim 11, wherein the step c) includes: c11) establishing a boundary table, wherein the boundary table is used to record a boundary of each virtual object; c12) When any of the virtual objects is a line segment or a stroke, detect a collision of the line segment or the stroke through a line renderer; c13) when any of the virtual objects is a regular object, calculating the boundary of the regular object by a mesh renderer, and detecting a collision of the regular object based on the boundary; and c14) Perform a grasp detection and an assembly detection based on the collision detection results. The disassembly and inspection method as described in claim 13, wherein calculating the boundary of the conventional object further includes c131) Get an object pose of the regular object; c132) Compute a bounding box of the virtual object in a non-rotated state; c133) rotating the bounding box based on the object pose to obtain a rotated bounding box; and c134) Using the rotated bounding box as an object bounding box of the regular object, wherein the object bounding box is used as a collision boundary of the regular object, and is used to detect the collision of the regular object. The disassembly inspection method as described in claim 11, wherein the step c) includes: c21) moving at least one of the plurality of virtual objects based on a move operation; c22) when an assembly condition is satisfied, perform a group process to assemble the plurality of virtual objects that collide into a first group object; and c23) When a disassembly condition is satisfied, perform a degrouping process to separate a second group of objects that has been separated into the plurality of virtual objects. The disassembly and inspection method as described in claim 15, wherein the group processing includes c221) Generate the first group of objects that can be grabbed, and set the plurality of virtual objects that collide as the plurality of child objects of the first group of objects; and c222) Setting the plurality of child objects of the first group of objects to be ungraspable. The disassembly and inspection method as described in claim 15, wherein the ungrouping process includes: c231) delete links between child objects of the second group of objects, and delete the second group of objects; and c232) Set the multiple child objects as the multiple virtual objects that can be grabbed. The disassembly and inspection method as described in Claim 11, wherein the step d) includes at least one of the following: d1) When an incorrect action on any virtual object is detected, an action error prompt is issued; d2) When detecting that the composition of any virtual object is incorrect, issue a composition error prompt; d3) When an incorrect position of any virtual object is detected, a position error prompt is issued; and d4) When it is detected that an assembly sequence of any of the virtual objects is incorrect, a sequence error prompt is issued. The disassembly and inspection method as described in claim 11, which further includes before the step a): f1) performing an initialization process on a teaching end; f2) creating the plurality of virtual objects based on an object creation operation; f3) Create a segment connecting the plurality of virtual objects based on a segment creation operation; f4) changing the teaching detachment state of the plurality of virtual objects based on a teaching detachment operation; f5) Set the checkpoint and record the teaching disassembly status of the checkpoint; and f6) Generate the teaching disassembly record. The disassembly and inspection method as described in claim 19, wherein the line segment creation operation includes setting the number of a head node, the number of body nodes and the number of a tail node; This step f3) comprises: f31) creating the line segment based on the number of head nodes, the number of body nodes and the number of tail nodes, wherein at least one head node and at least one tail node of the line segment are respectively connected to the plurality of virtual objects; and f32) Set a bundle relationship of the line segment to convert multiple body nodes into a bundle; and f33) Set multiple node coordinates of the line segment.

Images