KR20200109271A - System for supporting parts maintenace based on mixed reality and method thereof - Google Patents

Abstract

According to an embodiment, disclosed are a system for supporting component maintenance based on mixed reality and a method thereof. The component maintenance support system includes: HMD equipment obtaining an image for a target object existing in a pre-built virtual working space; and a worker terminal recognizing at least one component included in the target object by analyzing the obtained image, and providing component information for the recognized component to the HMD equipment. The HMD equipment displays the image for the recognized target object and then, if one of the at least one component included in the target object is selected in the displayed image, a technical data list for the maintenance of the selected component is displayed. According to the present invention, since various components can automatically be recognized in real-time, it is possible to remarkably improve work efficiency.

Description

A system and its method to support mixed reality based parts maintenance {SYSTEM FOR SUPPORTING PARTS MAINTENACE BASED ON MIXED REALITY AND METHOD THEREOF}

The embodiment relates to a system and method for supporting mixed reality-based component maintenance.

In the case of institutions and enterprises that operate large-scale equipment such as aircraft and railroads, annual equipment maintenance costs are increasing.Especially, when large-scale equipment is operated, there is a market demand for systems and solutions for securing availability and reducing maintenance costs. Is increasing.

The embodiment may provide a system and method for supporting mixed reality-based component maintenance.

The parts maintenance support system according to the embodiment includes: a Head Mounted Display (HMD) device for acquiring an image of a target object existing in a virtual work space built in advance; And an operator terminal that analyzes the acquired image to recognize at least one part included in the target object, and provides part information on the recognized part to the HMD equipment, wherein the HMD equipment is the recognized When an image of a target object is displayed and any one of at least one part included in the target object is selected within the displayed image, a list of technical data for maintenance of the selected part may be displayed.

When the worker terminal receives a spatial image of the work space from the HMD device, the virtual work space is constructed using the received spatial image, and a reference coordinate value at at least one point of a target object in the virtual work space Can be assigned.

The HMD device may display a virtual work space to which coordinate values are allocated from the worker terminal.

When the HMD device receives parts information on a recognized part from the operator terminal, the received part information may match and display the received part information with the virtual work space.

When one part is selected based on part information displayed in the virtual work space, the HMD device may display the selected part to be identifiable from other parts.

The operator terminal stores a machine learning model for recognizing at least one part included in the target object, analyzes the image obtained from the HMD using the stored machine learning model, and at least one included in the target object Can recognize the parts of.

The parts maintenance support system may further include a service server that trains the machine learning model using predetermined learning data and provides the learned machine learning model to the operator terminal.

The list of technical data may include a maintenance manual, a construction drawing, a part specification, an exploded assembly drawing, and a maintenance video.

The parts maintenance support method according to the embodiment includes the steps of obtaining an image of a target object existing in a virtual work space in which a head mounted display (HMD) equipment is pre-built; An operator terminal analyzing the acquired image to recognize at least one part included in the target object, and providing part information about the recognized part to the HMD equipment; And when the HMD device displays an image of the recognized target object and one of at least one part included in the target object is selected within the displayed image, a list of technical data for maintenance of the selected part. It may include the step of displaying.

In the parts maintenance support method, when the worker terminal receives a spatial image of a work space from the HMD equipment, the virtual work space is constructed using the received spatial image, and at least a target object in the virtual work space is The step of allocating a reference coordinate value to one point may further be included.

The parts maintenance support method may further include displaying, by the HMD equipment, a virtual work space to which coordinate values are allocated from the operator terminal.

In the displaying step, when receiving part information on the recognized part from the operator terminal, the received part information may be matched with the virtual work space and displayed.

In the displaying step, when one part is selected based on part information displayed on the virtual work space, the selected part may be displayed to be identifiable from other parts.

According to the embodiment, since it is possible to receive maintenance-related information through an HMD worn by an operator, workers who are unskilled in maintenance of complex devices or facilities stop working and look at manuals, or inquire information on related detailed parts. Or, you can solve the problem of low productivity in terms of maintenance time efficiency, such as contacting an expert to check the assembly work status of the parts.

According to the embodiment, in tasks such as heavy maintenance such as disassembly and assembly of complex facilities, it is possible to dramatically improve work productivity by real-time reference of standard maintenance work methods and related technical data, and real-time automatic recognition of various parts through artificial intelligence technology. I can.

According to the embodiment, the cost of constructing a large amount of learning data DB for part recognition is significantly reduced through the learning data and BOM-based labeling for part recognition in actual operation. It can be possible.

Various and beneficial advantages and effects of the present invention are not limited to the above-described contents, and may be more easily understood in the course of describing specific embodiments of the present invention.

1 is a view showing a parts maintenance support system according to an embodiment of the present invention.
2A to 2C are diagrams showing a detailed configuration of the HMD device shown in FIG. 1.
3 is a view showing a detailed configuration of the operator terminal shown in FIG.
4 is a diagram showing a detailed configuration of the service server shown in FIG. 1.
5A to 5C are views for explaining a process of recognizing a part object according to an embodiment of the present invention.
6 is a diagram illustrating a method of constructing a virtual work space according to an embodiment of the present invention.
7 is a view showing a component maintenance support method according to an embodiment of the present invention.
8A to 8C are views for explaining a process of displaying and selecting a recognized part object.
9 is a diagram for explaining information provided during emergency maintenance.
10A to 10F are diagrams for explaining information provided during precise maintenance.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

However, the technical idea of the present invention is not limited to some embodiments to be described, but may be implemented in various different forms, and within the scope of the technical idea of the present invention, one or more of the constituent elements may be selectively selected between the embodiments. It can be combined with and substituted for use.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention are generally understood by those of ordinary skill in the art, unless explicitly defined and described. It can be interpreted as a meaning, and terms generally used, such as terms defined in a dictionary, may be interpreted in consideration of the meaning in the context of the related technology.

In addition, terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In the present specification, the singular form may include the plural form unless specifically stated in the phrase, and when described as "at least one (or more than one) of A and (and) B and C", it is combined with A, B, and C. It may contain one or more of all possible combinations.

In addition, in describing the constituent elements of the embodiments of the present invention, terms such as first, second, A, B, (a), and (b) may be used.

These terms are only for distinguishing the component from other components, and are not limited to the nature, order, or order of the component by the term.

And, when a component is described as being'connected','coupled' or'connected' to another component, the component is not only directly connected, coupled or connected to the other component, but also the component and It may also include the case of being'connected','coupled' or'connected' due to another component between the other components.

In addition, when it is described as being formed or disposed in the "top (top) or bottom (bottom)" of each component, the top (top) or bottom (bottom) is one as well as when the two components are in direct contact with each other. It also includes a case in which the above other component is formed or disposed between the two components. In addition, when expressed as "upper (upper) or lower (lower)", the meaning of not only an upward direction but also a downward direction based on one component may be included.

In an embodiment, a worker wearing a HMD (Head Mounted Display) scans a work space to create a virtual work space, analyzes an image of a target object existing at a viewing angle in the virtual work space, We propose a new method that recognizes one part, checks the condition of the recognized part, displays maintenance-related information according to the checked state, and performs maintenance work according to the displayed maintenance-related information. Here, the target object may mean a composite part composed of a plurality of parts.

1 is a view showing a parts maintenance support system according to an embodiment of the present invention.

Referring to FIG. 1, a system for supporting maintenance of parts based on mixed reality according to an embodiment of the present invention may include an HMD equipment 100, an operator terminal 200, and a service server 300.

The HMD device 100 may be worn on a user's body, that is, a head, and may acquire a spatial image by scanning a work space, and obtain an object image of a target object existing in the work space.

The HMD device 100 may display information on at least one part included in a target object in a virtual work space and display detailed data on a selected part from among the displayed parts. Here, the detailed data may include detailed data for defective repairs in which emergency measures are possible and detailed data for maintenance and repairs in which emergency measures are impossible.

The HMD device 100 may recognize a hand gesture gesture of an operator in the process of performing parts maintenance work and perform a predetermined function according to the recognized hand gesture gesture.

The HMD device 100 refers to a display device that can be mounted on a head and present an image directly in front of a worker's eyes. The HMD device 100 may display part information on an actual target object using augmented reality technology. Augmented reality technology is being applied in industrial fields such as manufacturing, distribution, and medical care under the name of industrial AR. In theory, augmented reality requires three requirements:'combination of reality and virtual images','real-time interaction', and'development into 3D space', but it is fundamentally to visualize information based on the real world. For this reason, industrial augmented reality technology is playing a role of helping to improve productivity and work efficiency in the fields of assembly, repair, verification, monitoring, quality control, and education by providing data to workers. Accordingly, HMD equipment such as small optical projection displays, cameras, microphones, and smart glasses with built-in communication modules are changing their product positions from initial personal use to production innovation tools in the industrial field. Using HMD equipment such as smart glasses at the actual industrial site, you can easily read digital manuals in front of your eyes without interrupting work, watch educational videos or connect experts directly by voice, and experts can view the situation as if they were in the field. Grasp and provide accurate guides to workers.

The worker terminal 200 may interwork with the HMD device 100, receive a spatial image of a space where the worker works from the HMD device 100, and build a virtual work space based on the received spatial image. In addition, the operator terminal 200 may recognize at least one part included in the target object by receiving an object image of the target object to be worked from the HMD device 100 and analyzing the received object image using a machine learning model. have.

The worker terminal 200 may interwork with the service server 300 and update a machine learning model or a machine learning algorithm for recognizing a part through the service server 300.

Here, the worker terminal 200 is a device that enables communication using a mobile communication network while the user is carrying, and may include, for example, a smart phone or a tablet PC.

The service server 300 may build training data including an image for recognizing a predetermined part, and train a machine learning model based on the built training data.

The service server 300 may interwork with the worker terminal 200, authenticate a worker who uses the worker terminal 200, and periodically update the machine learning model.

2A to 2C are diagrams showing a detailed configuration of the HMD device shown in FIG. 1.

Referring to FIG. 2A, the HMD device 100 according to an embodiment of the present invention includes a display unit 110, a photographing unit 120, a control unit 130, a sensor unit 140, and a communication unit 150. I can.

The display unit 110 may display a target object existing in the virtual work space and part information about at least one part included in the target object together.

The display unit 110 is a mixed reality user interface information, such as information necessary to maintain a part selected by an operator in an actual maintenance work, such as a maintenance manual, an exploded assembly drawing (video, 3D animation), a part specification, a completion drawing, a maintenance video Can be displayed.

The photographing unit 120 may acquire an image of a viewing angle according to the position and direction of the worker.

The controller 130 may match virtual information to an image of a target object existing in the virtual work space according to the user's location and direction, and provide it to the display 110.

The controller 130 may include an application module 130a and a motion tracking module 130b. The application module 130a stores part information on at least one part included in the target object, and the motion tracking module 130b provides an image of a target object existing in a virtual work space and a part included in the target object. You can match the parts information for.

The motion tracking module 130b may track the position and direction of a worker based on the sensing information.

The sensor unit 140 may acquire various sensing information for tracking the location and direction of a worker. The sensor unit 140 may include, for example, an acceleration sensor, a geomagnetic sensor, and an angular velocity sensor.

The communication unit 150 may transmit or receive various types of data in connection with the worker terminal 200. For example, the communication unit 150 transmits an image of a target object existing in the virtual work space to the worker terminal 200, and the at least one part included in the target object recognized in the image from the worker terminal 200 Part information can be received.

Referring to FIG. 2B, in the embodiment of the present invention, a controller 160 interlocking with the HMD device 100 is further included, and the controller 160 includes a communication unit 161, a control unit 162, and an input unit 163. can do.

The communication unit 161 may interwork with the HMD device 100 and transmit a control signal according to an operator's manipulation.

The control unit 162 may generate a control signal according to an operator's manipulation and provide it to the communication unit 161. The controller 162 may generate a direction control signal when an operator manipulates a direction button, and may generate a selection control signal when an operator manipulates a selection button.

The input unit 163 may include a plurality of manipulation buttons that an operator can manipulate. The plurality of operation buttons may receive operation information according to an operator's operation. Here, the operation button may include four direction buttons b1, b2, b3, and b4 including up, down, left, and right, and one selection button s1.

The controller 160 for controlling such HMD equipment may be implemented as shown in FIG. 2C.

3 is a view showing a detailed configuration of the operator terminal shown in FIG.

Referring to FIG. 3, the operator terminal 200 according to an embodiment of the present invention includes a first communication unit 210, a second communication unit 212, an input unit 220, a control unit 230, a display unit 240, and a storage unit. It may include a unit 250.

The first communication unit 210 may transmit and receive various data in connection with the HMD device 100, and the second communication unit 212 may transmit and receive various data in connection with the service server 300.

The input unit 220 may receive input information according to a user's menu or key manipulation.

The control unit 230 may activate the application and receive a spatial image of a space in which a worker works from the paired HMD device 100 through the activated application, and build a virtual work space based on the received spatial image. .

The controller 230 may recognize at least one part included in the target object by receiving an object image of the target object from the HMD device 100 and analyzing the received object image using a machine learning model.

The control unit 230 may include an edge computing engine module 230a and an algorithm update module 230b. The edge computing engine module 230a may analyze the object image using a machine learning model to recognize at least one part included in the target object, and the algorithm update module 230b may update the machine learning model.

Edge computing technology is a technology that processes vast amounts of data in real time through distributed small servers rather than centralized servers. 'Edge' means edge, meaning that data is processed at the edge of the network, unlike cloud computing where a central server processes all data. As Internet of Things (IoT) devices were spread in earnest, the amount of data exploded, and cloud computing was hitting a limit, and edge computing technology was developed to compensate. In other words, it is a technology for processing important data in real time, instead of sending all data to the cloud for analysis. This technology plays a key role in realizing the fourth industrial revolution, such as autonomous vehicles, smart factories, and virtual reality (AR) that must respond in real time.

The display unit 240 may display various types of information related to the operator performing maintenance work on the screen.

The storage unit 250 may store part information and a machine learning model for each part.

4 is a diagram showing a detailed configuration of the service server shown in FIG. 1, and FIGS. 5A to 5C are diagrams for explaining a process of recognizing a part object according to an embodiment of the present invention.

Referring to FIG. 4, the service server 300 according to an embodiment of the present invention may include a communication unit 310, a control unit 320, and a storage unit 330.

The communication unit 310 may interwork with the worker terminal 200 and transmit or receive various data.

The controller 320 may build training data including an image for recognizing a predetermined part, and train a machine learning model based on the built training data.

The controller 320 may include an artificial intelligence module 320a, an image classification module 320b, a data authoring module 320c, and a machine learning module 320d. The artificial intelligence module 320a may extract a part object by converting an image of a real part into an object through computer vision technology. Referring to FIG. 5A, the artificial intelligence module 320a preprocesses the image of the part, detects the image pattern through an edge detection algorithm, recognizes the image of the part, and converts the image of the part into an object. Objects can be extracted.

Here, the development of computer vision technology, a technology of artificial intelligence, has developed algorithms with excellent performance in each field such as object recognition, tracking, 3D shape restoration, and video understanding. , Autonomous driving system, photo and base search system, robot visual recognition system, smart factory-based work efficiency system, etc., and is used in the field of preventive maintenance through fault diagnosis for large-scale devices.

For example, in order to further evolve maintenance for complex devices such as aircraft and high-speed railways, maintenance records, technicians' memos, and data from IoT sensors are collected/analyzed, so that maintenance engineers take the time to analyze the fault history. It is effective in shortening the time taken to analyze the fault history of the aircraft and reducing the delay time by specifying the cause of the failure or suggesting an appropriate response policy.

The image classification module 320b may recognize a part object of the extracted image.

The data authoring module 320c may generate learning data for component recognition correction by inserting a label, that is, a BOM code, for an image of an unrecognized or error-recognized component.

The machine learning module 320d may train a machine learning model based on the built learning data.

The storage unit 330 may store BOM data, learning data, and machine learning models for various parts. The storage unit 330 may be one physically coupled storage unit, but is not limited thereto and may be implemented as a plurality of physically separated storage units.

Referring to FIG. 5B, in an embodiment, a method for constructing a learning data set to improve the initial recognition rate of parts, and during periodic light maintenance and heavy maintenance to operate complex devices such as aircraft, railroads, etc., when disassembling and assembling parts A picture can be taken through a rotating camera and transmitted to the server to be temporarily stored in the learning data storage unit.

At this time, the data acquired for learning is stored in a folder and compressed file format in which images of various angles are rotated through 360 degrees, and basic data for part recognition can be generated by mapping and labeling the saved data with the BOM code. have.

Referring to FIG. 5C, in an embodiment, in order to improve the image recognition rate, the image is corrected through a pre-processing process, grayscale, binarization, enlargement/reduction, rotation/transformation, and a 3D shape pattern image is extracted through 3D registration. Learning data can be constructed through detection.

6 is a diagram illustrating a method of constructing a virtual work space according to an embodiment of the present invention.

Referring to FIG. 6, the HMD device 100 according to an embodiment of the present invention may attempt to pair with the operator terminal 200 when power is applied according to the operator's menu or key operation (S601) ( S602).

Next, when the pairing with the worker terminal 200 is successful, the HMD device 100 may display a message informing that it is connected to the worker terminal (S603). Likewise, if the operator terminal 200 is successfully paired with the HMD device 100, a message indicating that the HMD device 100 is connected may be displayed.

The HMD device 100 may be paired with the operator terminal 200 using, for example, Bluetooth 3.0 technology, which is a low-power communication method, to transmit or receive data.

Next, the HMD device 100 may scan the work space while rotating 360 degrees according to the movement of the worker (S604) and transmit the image acquired in the scanning process to the worker terminal 200 (S605).

Next, the worker terminal 200 may construct a virtual work space based on the transmitted image (S606), and allocate a predetermined reference coordinate value to at least one point existing in the virtual work space (S607). ). The reason for assigning the reference coordinate value is to prevent confusion of the operator by fixing the virtual work space even if the operator moves based on the reference coordinate value.

Next, the worker terminal 200 may transmit the data of the built virtual work space to the HMD device 100 (S608).

Next, the HMD device 100 may display a virtual workspace based on the transmitted data (S609). Operators can perform maintenance work by recognizing parts within the displayed virtual workspace.

7 is a diagram illustrating a component maintenance support method according to an embodiment of the present invention, FIGS. 8A to 8C are diagrams for explaining a process of displaying and selecting a recognized component object, and FIG. 9 is a diagram provided for emergency maintenance. It is a diagram for explaining information, and FIGS. 10A to 10F are diagrams for explaining information provided during precise maintenance.

Referring to FIG. 7, the worker terminal 200 according to an embodiment of the present invention activates an application of the worker terminal when the virtual work space is built (S701), and the service server 300 through the activated application The worker can be authenticated by synchronizing with the application account (S702).

Next, if the worker's authentication is successful, the worker terminal 200 may receive the maintenance work details assigned to the worker (S703) and transmit the received maintenance work details to the HMD equipment 100 (S704).

For example, the maintenance work may be divided into fields of a roof device, a traveling device, a cab device, and a machine room device, and may be divided into maintenance work details including work procedures for each field and assigned to a corresponding worker.

Next, the HMD equipment 100 may display the transmitted maintenance work details (S705). The operator can check the displayed maintenance work history by using hand gestures to check the process for each work procedure. Here, as an algorithm for recognizing hand gestures, a general well-known algorithm may be used.

Next, the HMD equipment 100 may acquire an object image of a target object for maintenance work according to the displayed maintenance work details (S706) and transmit it to the operator terminal 200 (S707).

Next, the operator terminal 200 may recognize at least one part included in the target object by analyzing (S708) using a machine learning model previously learned based on the transmitted object image (S709).

Next, the operator terminal 200 may transmit the generated part information to the HMD equipment 100 (S710). Here, the part information may include bill of material (BOM) data of the part.

Next, when receiving the part information, the HMD device 100 may display at least one part included in the target object to be identifiable based on the received part information (S711).

When there is a target object including at least one part as shown in FIG. 8A, at least one part recognized from the target object is displayed as shown in FIG. 8B, so that the number of the part selected by the operator can be identified as shown in FIG. 8C. While displayed, part information can also be displayed. Here, the operator can select a part using a finger from the image displayed on the HMD equipment, or select a part by operating the direction buttons of the controller.

For example, in this embodiment, providing too much information during mixed reality matching causes confusion for the operator, so if the numbered parts are initially matched and the desired number is selected through a controller connected to the HMD equipment, the selected number is selected during the mixed reality display. It can be implemented so that the color of the part is changed and the highlight and the part name are displayed together.

Next, the HMD equipment 100 may display a list of technical data for maintenance and repair according to the emergency maintenance or precise maintenance selected by the operator, when there is a part with a defective state among the displayed parts (S712).

Next, the HMD equipment 100 displays detailed data according to an item selected by the operator from the list of displayed technical data, so that parts maintenance by the operator may proceed (S713).

As an example, in case of emergency maintenance, a list of technical data for emergency maintenance of a corresponding part is displayed, and if detailed data according to an item selected by an operator from the displayed technical data list is displayed, Emergency maintenance may proceed.

As shown in FIG. 9, when an emergency measure manual item is selected from the list of technical data required for emergency maintenance and the list of technical data, detailed data according to the corresponding item is displayed.

As another example, in the case of precision maintenance, a list of technical data for precise maintenance of the corresponding part is displayed, and detailed data is displayed according to the item selected by the operator from the displayed technical data list, Maintenance can proceed.

As shown in FIG. 10A, when displayed on the list of technical data required for precise maintenance, one item from the displayed list of technical data may be selected. FIG. 10B shows a case in which the'light maintenance manual' item is selected, and in FIG. Drawing' item is selected, Fig. 10D shows a case where'Parts specification' item is selected, Fig. 10E shows a case where'exploded assembly view' item is selected, and Fig. 10F shows a'maintenance video' 'Shows the case where the item is selected.

In addition, in the embodiment, in addition to the above data, a UI that can be searched by linking technical data to HMD equipment or operator terminals is provided, or by linking with the company's legacy system, the corresponding inventory, ordering, and compatible parts location are displayed in detail information of parts. Can be made to provide.

The term'~ unit' used in this embodiment refers to software or hardware components such as field-programmable gate array (FPGA) or ASIC, and'~ unit' performs certain roles. However,'~ part' is not limited to software or hardware. The'~ unit' may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays, and variables. The components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further divided into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

Although described above with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

100: HMD equipment
160: controller
200: operator terminal
300: service server

Claims (13)

Head Mounted Display (HMD) equipment for obtaining an image of a target object existing in a pre-built virtual work space; And
Analyzing the acquired image to recognize at least one part included in the target object, and including a worker terminal for providing part information on the recognized part to the HMD equipment,
The HMD equipment is
A part that displays an image of the recognized target object, and when any one of at least one part included in the target object is selected within the displayed image, a list of technical data for maintenance of the selected part is displayed Maintenance support system. The method of claim 1,
The worker terminal
When receiving a spatial image of the working space from the HMD device, the virtual working space is constructed using the received spatial image,
The parts maintenance support system for allocating a reference coordinate value to at least one point of the target object in the virtual work space. The method of claim 2,
The HMD equipment,
Parts maintenance support system for displaying a virtual work space to which coordinate values are allocated from the operator terminal. The method of claim 1,
The HMD equipment,
When receiving part information on the recognized part from the operator terminal, the part maintenance support system for matching and displaying the received part information with the virtual work space. The method of claim 4,
The HMD equipment,
When any one part is selected based on part information displayed in the virtual work space, the part maintenance support system for displaying the selected part to be identifiable from other parts. The method of claim 1,
The worker terminal,
Storing a machine learning model for recognizing at least one part included in the target object,
A parts maintenance support system for recognizing at least one part included in the target object by analyzing the image acquired from the HMD using the stored machine learning model. The method of claim 6,
Further comprising a service server that trains the machine learning model using predetermined learning data and provides the learned machine learning model to the worker terminal. The method of claim 1,
The above list of technical data,
Parts maintenance support system, including maintenance manuals, construction drawings, parts specifications, exploded drawings, and maintenance videos. Acquiring an image of a target object existing in a virtual work space in which a head mounted display (HMD) device is built in advance; And
Analyzing, by an operator terminal, the acquired image to recognize at least one part included in the target object, and providing part information on the recognized part to the HMD equipment; And
When the HMD device displays an image of the recognized target object and any one of at least one part included in the target object is selected within the displayed image, a list of technical data for maintenance of the selected part is displayed. Including the step of displaying, parts maintenance support method. The method of claim 9,
When the worker terminal receives a spatial image of the work space from the HMD device, the virtual work space is constructed using the received spatial image, and a reference coordinate value at at least one point of a target object in the virtual work space The method of supporting parts maintenance, further comprising the step of allocating. The method of claim 10,
The HMD equipment further comprises the step of displaying a virtual work space to which a coordinate value is allocated from the operator terminal. The method of claim 9,
In the displaying step,
When receiving part information on the recognized part from the operator terminal, the received part information is matched with the virtual work space and displayed. The method of claim 12,
In the displaying step,
When any one part is selected based on part information displayed in the virtual work space, the selected part is displayed to be identifiable from other parts.

Images