KR102469149B1 - Real object multi-directional recognition training system based on 3D design data and the method using it - Google Patents

Abstract

The present invention relates to a real object multidirectional recognition system based on 3D design data, and more particularly, to a real object multidirectional recognition system that provides a general equipment maintenance practice system based on 3D design data using virtual reality and an operation method thereof. it's about
The present invention provides a virtual maintenance practice providing device that obtains positional information on a three-dimensional space, recognizes a practitioner's motion, and displays a haptic signal generated by a practitioner in a virtual failure simulation procedure in virtual reality; a communication unit providing the haptic signal to the virtual maintenance practice providing device; A controller configured to configure a practice situation so that a mechanic can find a failure using the virtual maintenance practice providing device for a failure of a component when a failure situation is input in response to the practitioner's operation, wherein the control unit includes the A virtual failure practice scenario converting unit that converts the scenario according to the virtual failure simulation procedure to suit the purpose and target of the virtual failure simulation;
a facility/device and background 3D modeling unit for 3D modeling the facility/device and background to be suitable for the maintenance scenario for the virtual failure simulation practice; It includes; a practice tool 3D modeling unit for 3D modeling a practice tool for scenario progress for virtual failure simulation practice using the facility/device and background for which the 3D modeling was performed.

Description

Real object multi-directional recognition training system based on 3D design data and the method using it}

The present invention relates to a real object multi-directional recognition system based on 3D design data, and more particularly, using a model tool for a 3D rendered maintenance target object in virtual reality without the actual maintenance target using virtual reality. It relates to a real object multi-directional recognition system that provides a general equipment system based on 3D design data and an operating method thereof so that practitioners can perform maintenance practice.

Virtual reality refers to a human-computer interface that creates a specific environment or situation as a computer and makes it as if a person using it is interacting with a real surrounding situation or environment.

This virtual reality has been applied to the practice system and developed into an immersive virtual environment technology that increases the immersion effect of practice by allowing practitioners to wear virtual reality devices and participate in virtual practice environments.

Virtual reality technologies have been developed to enable practitioners to interact with computer simulation environments, and with the recent emergence of games, marketing, and educational contents to which these virtual reality technologies are applied, virtual three-dimensional (3D) phenomena can be easily and dynamically simulated by practitioners. Techniques for control have been developed.

As a representative of these technologies, the HMD (Head Mounted Display) method dynamically controls a virtual 3D shape by a practitioner manipulating a separate VR controller, and the display method dynamically controls a virtual 3D shape through a finger touch on the screen by a practitioner. Control.

In order to provide maintenance practice to practitioners in such a conventional virtual reality environment, a method in which the practitioner simply touches a maintenance position (marker) in the virtual reality environment and makes the practitioner master the maintenance procedure is used.

However, in the case of such a conventional virtual maintenance practice method in virtual reality, it is possible to learn maintenance procedures for practitioners, but there is a problem that it is difficult to master the use of tools used for actual maintenance.

On the other hand, haptics (haptic stimulation) is a study that understands human sensorimotor ability and studies the system for physical interaction between a user and a computer in various situations such as real environment, virtual environment, and augmented environment based on this understanding. Simply put, it allows users to touch virtual objects with their hands.

And haptics is a new field in which many studies have been conducted since the late 1990s.

It has developed from research on communication aids for the hearing-blind, and is currently being actively researched in academia and industry, and is being applied to various fields such as virtual/augmented reality, robots, general consumer electronics, and medical care.

However, there was a problem in that it was difficult to increase the immersion effect of practitioners because it was not yet actively used in design data-based general equipment maintenance practice systems.

Korean Registered Patent Publication No. 10-2012835 (Announced on August 21, 2019)

delete

The present invention has been made to solve the above problems, according to the expensive equipment maintenance practice system and its operation method using virtual reality, a model for a maintenance target object rendered three-dimensionally in virtual reality without the actual maintenance target. The purpose is to provide a real object multi-directional recognition system based on 3D design data that allows practitioners to perform maintenance practice using tools.

The present invention is difficult to authorize and handle due to the nature of expensive equipment, and multidirectional recognition of real objects based on 3D design data that enables practitioners to practice maintenance through virtual reality without learning through actual maintenance for very expensive equipment The purpose is to provide a system.

In order to solve the above problems, the present invention obtains location information in a three-dimensional space, recognizes the practitioner's operation, and displays a haptic signal generated by the practitioner of the virtual failure simulation procedure in virtual reality. A virtual maintenance practice providing device; a communication unit providing the haptic signal to the virtual maintenance practice providing device; A controller configured to configure a practice situation so that a mechanic can find a failure using the virtual maintenance practice providing device for a failure of a component when a failure situation is input in response to the practitioner's operation, wherein the control unit includes the A virtual failure practice scenario converting unit that converts the scenario according to the virtual failure simulation procedure to suit the purpose and target of the virtual failure simulation; a facility/device and background 3D modeling unit for 3D modeling the facility/device and background to be suitable for the maintenance scenario for the virtual failure simulation practice; It includes; a practice tool 3D modeling unit for 3D modeling a practice tool for scenario progress for virtual failure simulation practice using the facility/device and background for which the 3D modeling was performed.

The device for providing virtual maintenance practice may include a gripper gripped by a practitioner's hand, a hollow and a vibration motor are formed on the upper side of the gripper, and a virtual failure practice scenario converting unit 111 is provided by the practitioner. A haptic glove that generates a first haptic signal generated by manipulating a practice model tool provided according to a maintenance scenario, wherein the practice tool of the practice tool 3D modeling unit receives operation information of the haptic glove, and The control unit transmits a control command so that the maintenance training proceeds normally based on a maintenance scenario for virtual failure simulation using operation information, facility/device, and background provided by the 3D modeling unit.

The device for providing virtual maintenance practice is connected to a Head Mounded Display (HMD) mounted on the practitioner's head to provide the generated maintenance practice content to the practitioner; and guides tools necessary for maintenance of the virtual failure simulation practice. When a voice is recognized by the voice recognition unit while providing content including data and tool usage procedure guidance data, the controller corresponding to the recognized voice performs a control command to control the content, and the HMD simulates a virtual failure procedure VR. If contents including photos, videos, and text messages are transmitted from the control unit while providing images, they are displayed together with VR images according to virtual failure simulation, and the practice tools of the 3D modeling unit of the practice tools receive operation information of the HMD, The control unit controls the maintenance training to proceed normally based on the scenario provided by the virtual failure practice scenario conversion unit for virtual failure simulation practice using the operation information, facilities/devices, and background provided by the 3D modeling unit send the command

The device for providing virtual maintenance practice is connected to a plurality of virtual reality controllers that generate a second haptic signal, which is a control signal of the practitioner for the maintenance object in the virtual reality, and the practice tool of the practice tool 3D modeling unit is the virtual Receiving operation information of a real controller, based on a scenario for virtual failure simulation using the operation information, facility/device, and background, and using at least one of the practitioner's career information related to the virtual failure simulation to generate prior knowledge level information, the value obtained by applying the first weight to information selected from among the first weight, the second weight, and the evaluation information, and the value obtained by applying the second weight to the career information The control unit provides the prior knowledge level information by using.

The apparatus for providing virtual maintenance practice includes a database including management information of items subject to failure, management information of jigs, management information of inspection items, and other item management information according to the virtual failure simulation practice, wherein the database is obtained from a motion sensor. Operation information on the practitioner's operation is received and stored, and based on the operation information, the control unit controls to perform maintenance training extracted from the database.

The virtual failure practice scenario converting unit judges the performance of the virtual failure simulation practice, determines and presents a practice scenario to be practiced next, and as the number of practices accumulates, the practice history and accident history stored in the practice result DB server. And based on the practice pattern, a practice scenario to be practiced by the operator is determined and presented, and the maintenance details of the equipment to be repaired, the maintenance scenario of the equipment to be repaired, information on the maintenance guide for the equipment to be repaired, and the equipment to be maintained It includes location information (Marker) where the model tool should be located on the maintenance scenario.

The present invention includes the steps of displaying, in virtual reality, a haptic signal generated by a practitioner of a virtual failure simulation procedure by an apparatus for providing virtual maintenance practice; providing the haptic signal to the virtual maintenance practice providing device by a communication unit; Recognizing, by a motion sensor, a practitioner's motion by obtaining location information of the virtual maintenance practice providing device in a 3-dimensional space; converting, by a virtual failure simulation scenario converting unit, a scenario according to the virtual failure simulation procedure to be suitable for the purpose and subject of the virtual failure simulation; 3D modeling of facilities/devices and backgrounds by a facility/device and background 3D modeling unit to fit the scenario for the virtual failure simulation; The practice tool 3D modeling unit 3D modeling a practice tool for scenario progress for virtual failure simulation practice using the facilities/devices and backgrounds in which the 3D modeling was performed.

The present invention made as described above can provide the practitioner with the same effect as the maintenance proficiency through the use of tools in the maintenance practice procedure in a real environment based on the three-dimensional design data.

According to the expensive equipment maintenance practice system and its operating method according to an embodiment of the present invention configured as described above, it is difficult to apply and handle due to its characteristics, and maintenance through virtual reality without learning through actual maintenance for very expensive equipment. It has the effect of making practice available to practitioners.

1 is a diagram showing a virtual reality system according to the prior art.
2 is a diagram showing a schematic configuration of a real object multidirectional recognition system based on 3D design data according to an embodiment of the present invention.
3 is a diagram showing a detailed configuration of a real object multidirectional recognition system based on 3D design data according to an embodiment of the present invention.
4 is a diagram illustrating receiving operation information of a haptic glove and transmitting a control command according to an embodiment of the present invention.
5 is a diagram illustrating receiving operation information of an HMD and transmitting a control command according to another embodiment of the present invention.
6 is a diagram showing control using a first weight, a second weight, and the like of a virtual reality controller according to another embodiment of the present invention.
7 is a diagram showing control to perform maintenance training extracted from a database according to another embodiment of the present invention.
8 is a graph showing a haptic stimulus according to another embodiment of the present invention.

In order to fully understand the present invention, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Embodiments of the present invention may be modified in various forms, and the scope of the present invention should not be construed as being limited to the examples described in detail below. This embodiment is provided to more completely explain the present invention to those skilled in the art. Therefore, the shapes of elements in the drawings may be exaggerated to emphasize a clearer description. It should be noted that in each drawing, the same members are sometimes indicated by the same reference numerals. In addition, detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention are omitted.

As shown in FIGS. 2 and 3 , the real object multidirectional recognition system 100 of the present invention includes a virtual maintenance practice providing device 110 , a communication unit 120 , and a control unit 130 .

The virtual maintenance practice providing device 110 is a device that acquires location information in a three-dimensional space, recognizes a practitioner's motion, and displays a haptic signal generated by a practitioner in a virtual failure simulation procedure in virtual reality.

The device for providing virtual maintenance practice 110 includes a virtual failure practice scenario converting unit 111, a facility/device and background 3D modeling unit 112, and a training tool 3D modeling unit 113.

In addition, the device for providing virtual maintenance practice 110 is connected to a haptic glove 210 , a head mounted display (HMD) 220 , a virtual reality controller 230 , and a motion sensor, which are tactile interfaces.

Each configuration of the virtual maintenance practice providing device 100 is exemplary, and some configurations may be added or omitted as needed.

The haptic glove 210 generates a first haptic signal generated as a practitioner manipulates a model tool required when virtual maintenance is performed on the equipment to be serviced according to a maintenance scenario in virtual reality.

Here, the first haptic signal may include a signal generated when the practitioner wears the haptic glove 210 and grabs a model tool, or a maintenance operation according to the purpose of the model tool (eg, rotation operation using a screwdriver, etc.) In addition, the HMD 220 according to an embodiment of the present invention is mounted on the practitioner's head and provides the generated maintenance education content in virtual reality.

The virtual reality controller 230 generates a second haptic signal, which is a control signal of the practitioner for an object on the virtual reality learning content provided by the HMD.

Here, the second haptic signal is a signal for the trainee to select an object on the virtual reality learning content and to control the virtual reality environment.

The motion sensor acquires positional information of the haptic glove 210, the HMD 220, and the virtual reality controller 230 in a 3D space and recognizes the practitioner's motion.

The communication unit 120 provides the first haptic signal, the second haptic signal, and the location information to the virtual maintenance practice providing device 110 .

The method described above can be implemented as computer readable codes on a computer readable recording medium.

Computer-readable recording media includes all types of recording media in which data that can be decoded by a computer system is stored.

For example, there may be read only memory (ROM), random access memory (RAM), flash memory, and an optical data storage device.

In addition, the computer-readable recording medium may be distributed to computer systems connected through a computer communication network, and stored and executed as readable codes in a distributed manner.

When a failure situation is input in response to the practitioner's operation, the control unit 130 configures a practice situation so that a mechanic can find a failure using the virtual maintenance practice providing device for component failure.

Specifically, the control unit 130 converts the scenario according to the virtual failure simulation procedure to be suitable for the purpose and object of the virtual failure simulation, the virtual failure practice scenario converting unit 111, and the scenario for the virtual failure simulation. Facility/device and background 3D modeling unit 112 that 3D models facilities/devices and backgrounds to be suitable, 3D modeling practice tools for scenario progress for virtual failure simulation practice using the 3D modeled facilities/devices and backgrounds It includes a practice tool 3D modeling unit 113 to do.

Hereinafter, various embodiments of the present invention will be described with reference to FIGS. 4 and 7 .

The device for providing virtual maintenance practice 110 may include a gripper gripped by a practitioner's hand, a hollow and a vibration motor are formed on the upper side of the gripper, and the practitioner can use the virtual failure practice scenario converting unit 111 ) is connected to the haptic glove 210 that generates a first haptic signal generated by manipulating a practice model tool provided according to a maintenance scenario provided by

The practice tool of the practice tool 3D modeling unit 113 receives operation information of the haptic glove.

The facility/device and background 3D modeling unit 112 performs the maintenance training based on the maintenance scenario provided by the virtual failure practice scenario conversion unit 111 for virtual failure simulation practice using the operation information, facility/device, and background. The control unit 130 transmits a control command to proceed normally.

The virtual maintenance practice providing device 110 is connected to the HMD 220 mounted on the practitioner's head and providing the generated maintenance practice content to the practitioner; guides tools necessary for maintenance of the virtual failure simulation practice. When a voice is recognized by the voice recognition unit while providing content including data and tool usage procedure guidance data, the controller 130 corresponding to the recognized voice executes a control command to control the content.

When contents including photos, videos, and text messages related to external maintenance are transmitted from the control unit 130 while providing VR images of a virtual failure simulation procedure, the HMD 220 displays the VR images according to the virtual failure simulation.

The practice tool of the practice tool 3D modeling unit 113 receives the operation information of the HMD, and simulates a virtual failure using the operation information, facilities/devices, and background facilities/devices and background provided by the 3D modeling unit 112. Based on the maintenance scenario provided by the virtual failure practice scenario conversion unit 111 for practice, the control unit 130 transmits a control command so that the maintenance training proceeds normally.

The virtual maintenance practice providing device 110 is connected to a plurality of virtual reality controllers 230 that generate a second haptic signal, which is a control signal of the practitioner for the maintenance object in the virtual reality.

The practice tool of the practice tool 3D modeling unit 113 receives operation information of the virtual reality controller, based on a scenario for virtual failure simulation using the operation information, facility/device, and background, and simulates the virtual failure. Prior knowledge level information is generated using at least one of the practitioner's career information related to the practice, and a value obtained by applying the first weight to information selected from among first weight, second weight, and evaluation information, And the control unit 130 provides the prior knowledge level information by using a value obtained by applying the second weight to the career information.

The virtual maintenance practice providing device 110 includes a database 114 including management information of items subject to failure, management information of jigs, management information of inspection items, and other item management information according to the virtual failure simulation practice.

The database 114 receives and stores motion information about the practitioner's motion from a motion sensor, and based on the motion information, the control unit 130 controls to perform maintenance training extracted from the database.

The virtual failure practice scenario converting unit 111 determines and presents a practice scenario to be practiced next by determining the performance of the virtual failure simulation practice, and as the number of practices accumulates, the practice history stored in the practice result DB server. In addition, based on the accident history and practice pattern, the practice scenario to be practiced by the operator is determined and presented.

It includes maintenance details of the equipment to be repaired, maintenance scenarios of the equipment to be repaired, maintenance guide information on the equipment to be repaired, and location information (marker) at which the model tool should be located on the maintenance scenario of the equipment to be repaired.

The 3D design data-based real object multidirectional recognition system described above may be implemented as hardware components, 3D design data based real object multidirectional recognition system components, and/or a combination of hardware components and software components. have.

For example, the hardware components of a real object multi-directional recognition system based on 3D design data include a processor that can execute and respond to AI algorithm instructions, a controller, an arithmetic logic unit (ALU), and a digital signal processor. processor, microcomputer, field programmable array (FPA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. can be implemented using

The real object multi-directional recognition system based on 3D design data may execute an operating system (OS) and one or more software applications running on the operating system.

A processing device may also access, store, manipulate, process, and generate data in response to execution of software.

Although the real object multi-directional recognition system based on 3D design data is sometimes described as being used, those skilled in the art will understand that the processing device is a plurality of processing elements and/or a plurality of types. It can be seen that it may include processing elements of.

For example, a real object multidirectional recognition system based on 3D design data may include a plurality of processors or one processor and one controller.

In addition, the real object multi-directional recognition system based on 3D design data is also capable of other processing configurations, such as a parallel processor.

In particular, the real object multi-directional recognition system based on 3D design data may include a computer program, code, instruction, or a combination of one or more of them, and configures a processing device to operate as desired. or command the processing units independently or collectively.

The 3D design data-based real object multi-directional recognition system and/or data may be interpreted by a processing device or used to provide commands or data to a processing device, any type of machine, component, physical device, or virtual device. It can be permanently or temporarily embodied in virtual equipment, computer storage media or devices.

As shown in FIG. 8 , Weber-Fechner's law refers to the theory that in order to feel a change in a stimulus in a sensory organ, a stimulus must be received at a certain rate or more with respect to the initial stimulus.

That is, when a weak stimulus is received at first, the change of the stimulus can be recognized at least.

However, when a strong stimulus is initially received, the change in the stimulus must be large to recognize the change.

Therefore, according to Weber's law according to the present invention, the real object multidirectional recognition system based on 3D design data can use the equation defined as follows.

In Equation 1, p is a numerical value of the degree of recognition of the stimulus, and S is the intensity of the stimulus. And k is a constant of proportionality, called Weber's constant.

The increase in intensity of the physical stimulus required to cause a Just Noticeable Difference (JND) in the haptic response varies in a consistent manner across different stimulus levels.

For example, as shown in Equation 2 above, a smaller intensity change is required to cause JND at a lower intensity level than at a higher intensity level.

At this time, if "I" is the intensity of the stimulus and the constant "k" is the Weber's fraction, the following formula is obtained.

In other words, the value obtained by comparing the change in stimulus intensity with the initial intensity is the Weber prime number, and the smaller this number is, the smaller the change is required to cause JND.

The software having the AI 3D design data generation function used in the present invention described above may be distributed on a computer system connected to a network and stored or executed in a distributed manner.

Computer readable media may include program instructions, data files, data structures, etc. alone or in combination.

The step of pre-processing data by linking the multi-directional recognition system information of the plurality of real objects, the program command for generating the data set that has undergone the pre-processing process, the individual information of the 3-dimensional design data, and the 3-dimensional multi-directional recognition of the real object Examples of computer-readable recording media in which θ of the design data and Z value information indicating the magnitude of the slope are stored include hard disks, magneto-optical media, and ROM, RAM, Hardware devices specially configured to store and execute program instructions, such as flash memory, are included.

It is determined whether there is missing data among the preprocessed data, and if there is missing data, a value corresponding to the missing data is calculated using an interpolation method, and the missing data is additionally input into the memory.

Examples of the 3D design data-based real object multi-directional recognition system command include not only machine code generated by a compiler but also high-level language code that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Hereinafter, a maintenance practice method using a real object multidirectional recognition system based on 3D design data for the implementation of the present invention will be described in detail.

First, the virtual maintenance practice providing device displays the haptic signal generated by the practitioner of the virtual failure simulation procedure in virtual reality.

The communication unit provides the haptic signal to the virtual maintenance practice providing device.

In addition, the motion sensor acquires positional information on the 3D space of the virtual maintenance practice providing device to recognize the practitioner's motion.

In addition, the virtual failure simulation scenario converting unit converts the scenario according to the virtual failure simulation procedure to be suitable for the purpose and object of the virtual failure simulation.

Subsequently, the facility/device and background 3D modeling unit 3D models the facility/device and the background to fit the scenario for the virtual failure simulation practice.

Finally, the practice tool 3D modeling unit performs 3D modeling of practice tools for scenario progress for virtual failure simulation practice using the facilities/devices and backgrounds for which the 3D modeling was performed.

100: real object multi-directional recognition system
110: Device for providing virtual maintenance practice
111: virtual failure practice scenario conversion unit
112: Facility/equipment and background 3D modeling unit
113: practice tool 3D modeling department
210: haptic glove as a tactile interface
220: Head Mounted Display (HMD)
230: virtual reality controller
120: communication department
130: control unit

Claims (7)

A virtual maintenance practice providing device that acquires positional information in a three-dimensional space, recognizes a practitioner's motion, and displays a haptic signal generated by a practitioner in a virtual failure simulation procedure in virtual reality;
a communication unit providing the haptic signal to the virtual maintenance practice providing device;
A controller configured to configure a practice situation so that a mechanic can find a failure using the virtual maintenance practice providing device for a failure of a component when a failure situation is input in response to the practitioner's operation,
The control unit,
a virtual failure practice scenario converting unit that converts the scenario according to the virtual failure simulation procedure to suit the purpose and object of the virtual failure simulation;
a facility/device and background 3D modeling unit for 3D modeling the facility/device and background to be suitable for the maintenance scenario for the virtual failure simulation practice;
A practice tool 3D modeling unit for 3D modeling a practice tool for scenario progress for virtual failure simulation practice using the 3D modeled facility/device and background,
The device for providing virtual maintenance practice,
It is connected to a Head Mounded Display (HMD) mounted on the practitioner's head and providing the generated maintenance practice content to the practitioner;
When a voice is recognized from the voice recognition unit while providing contents including tool guide data and tool use procedure guide data necessary for maintenance of the virtual failure simulation, the controller corresponding to the recognized voice executes a control command to control the contents. do,
When contents including photos, videos, and text messages are transmitted from the control unit while providing VR images for the virtual failure simulation procedure, the HMD displays the VR images according to the virtual failure simulation,
The practice tool of the practice tool 3D modeling unit receives the operation information of the HMD, and converts virtual failure practice scenarios for virtual failure simulation practice using the operation information, facilities/devices, and background facilities/devices and backgrounds provided by the 3D modeling unit A real object multi-directional recognition system based on three-dimensional design data, characterized in that the control unit transmits a control command so that maintenance education proceeds normally based on a scenario provided by the department. The method of claim 1,
The device for providing virtual maintenance practice,
It may include a gripping part gripped by a practitioner's hand, and a hollow and a vibration motor are formed on the upper side of the gripping part, and practice provided by the practitioner according to the maintenance scenario provided by the virtual failure practice scenario conversion unit 111 A haptic glove that generates a first haptic signal generated by manipulating the model tool; is connected to;
The practice tool of the practice tool 3D modeling unit receives operation information of the haptic glove and uses the operation information, facilities/devices, and background facilities/devices and backgrounds provided by the 3D modeling unit Based on a maintenance scenario for simulation practice of a virtual failure The three-dimensional design data-based real object multi-directional recognition system, characterized in that the control unit transmits a control command so that the maintenance education proceeds normally. delete The method of claim 1,
The device for providing virtual maintenance practice,
Connected to a plurality of virtual reality controllers that generate a second haptic signal, which is a control signal of the practitioner for maintenance objects in virtual reality;
The practice tool of the 3D modeling unit receives the operation information of the virtual reality controller, and is based on a scenario for virtual failure simulation using the operation information, facility/device, and background,
Prior knowledge level information is generated using at least one of the practitioner's career information related to the virtual failure simulation, and the first weight is applied to information selected from among first weight, second weight, and evaluation information. 3D design data-based real object multi-directional recognition system, characterized in that the controller provides the prior knowledge level information using the obtained value and the value obtained by applying the second weight to the career information. The method of claim 1,
The device for providing virtual maintenance practice,
A database including management information of items subject to failure according to the virtual failure simulation, tool management information, inspection item management information, and other item management information,
The database receives and stores motion information about the practitioner's motion from a motion sensor, and based on the motion information, the control unit controls to perform maintenance education extracted from the database. Object multidirectional recognition system. The method of claim 1,
The virtual failure practice scenario conversion unit,
The practice performance of the virtual failure simulation is judged to determine and present the practice scenario to be practiced next. Determine and present practice scenarios to be performed,
Characterized in that it includes the maintenance details of the equipment to be repaired, the maintenance scenario of the equipment to be repaired, the maintenance guide information on the equipment to be repaired, and the location information (Marker) where the model tool should be located on the maintenance scenario of the equipment to be repaired. 3D design data-based real object multi-directional recognition system.

delete

Images