KR101891992B1 - Server, method, wearable device for supporting maintenance of military apparatus based reasoning, classification and decision of case on augmented reality - Google Patents

Abstract

The present invention provides a server, a method and a wearable device for supporting maintenance of a military device, which predict a maintenance object of a military device and stochastically derive a relative approach property, thereby enabling preemptive service support. The server for supporting maintenance of a military device comprises: an image receiving part receiving an image for the military device from the wearable device worn by a body of a maintenance person; an object recognizing part detecting a plurality of maintenance objects from the image, and recognizing at least one component object corresponding to at least one among the detected plurality of maintenance objects; a distance measuring part measuring a distance between the recognized at least one component object and each of the maintenance objects; a maintenance target object extracting part applying information on the distance between the recognized at least one component object and each of the maintenance objects to a case-based reasoning algorithm, and determining at least one maintenance target object among the detected plurality of maintenance objects; and a transmitting part transmitting maintenance information on the extracted maintenance target object to the wearable device.

Description

Technical Field [0001] The present invention relates to a server, a method, and a wearable device for supporting reasoning, classification, and decision-based military equipment maintenance in augmented reality,

The present invention relates to a server, a method and a wearable device for supporting reasoning, classification, decision-based military equipment maintenance in augmented reality.

As the range of data types varies, a variety of methodologies are emerging to analyze vast amounts of data. In the conventional text format data range, useful correlations to 2D, 3D, and virtual / augmented / mixed reality based unstructured data are found, extracting executable information in the future and using it for decision making It utilizes General Object Recognition technology. This means a technique of identifying what is in a video using various features of the object. Conventional object recognition techniques generally use a method of estimating an object using colors, minutiae, and patterns of an object.

However, in the case of military equipment, there is a problem in that it is difficult to apply the conventional object recognition technology to military equipment because the outside of the military equipment is formed of the same color, pattern, or the like.

On the other hand, an HMD (Head Mounted Display) is a head mounted display. When a user mounts an HMD on a head, a user can watch a video directly in front of the user through an HMD. The HMD mainly displays a virtual image or a virtual UI overlaid on the real world.

Related to this HMD, Korean Unexamined Patent Publication No. 2009-0105485 discloses a multimedia providing system using HMD and a method for providing the same.

In recent years, HMD has been providing services to support the maintenance of military equipment. However, in the case of repairing military equipments using HMD, it takes a lot of time to analyze the recognition rate and performance in the object recognition process due to parts having a size similar to that of the military color. In addition, there is a disadvantage in that a delay in a real-time service that carries recognition results of recognized maintenance objects and parts objects occurs.

The present invention provides a server, method and wearable device for supporting maintenance of military equipment that enables preemptive service support by predicting the maintenance objects of military equipment and deducing relative approach characteristics stochastically.

The present invention provides a server, method and wearable device for supporting maintenance of military equipments which allow a maintenance person to improve the understanding of operation and maintenance of equipment in order to repair military equipment, thereby enabling maintenance support.

A server, a method and a wearable device for supporting maintenance of a military equipment providing case-based maintenance information through a three-dimensional screen of a wearable device in order to provide convenience to a maintenance person in performing maintenance of the military equipment.

A server, a method, and a wearable device for supporting the maintenance of military equipment that increases the efficiency of maintenance through accident prevention and malfunction prediction in the machine maintenance industry such as maintenance of military equipment.

It is to be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided an image processing apparatus including an image receiving unit that receives an image of a military equipment from a wearable device worn on a body of a maintenance person, a plurality of maintenance objects from the image, An object recognition unit for recognizing at least one part object corresponding to at least one of the plurality of detected maintenance objects, a distance measurement unit for measuring a distance between the recognized at least one part object and each maintenance object, A maintenance object extracting unit that applies information on a distance between at least one part object and each maintenance object to a case based reasoning algorithm to determine at least one repair object among the plurality of repair objects detected; The maintenance information for the extracted maintenance object is stored in the wearable device And a transmitting unit for transmitting the data to the server.

Another embodiment of the present invention is directed to a method of controlling a vehicle, comprising: receiving an image of a military equipment from a wearable device worn on the body of a mechanic; detecting a plurality of maintenance objects from the image; The method comprising the steps of: recognizing at least one part object, at least one part object recognized by the at least one part object, measuring a distance between the recognized at least one part object and each maintenance object, Determining at least one repair object among the plurality of detected repair objects by applying the repair information to the wearable device, and transmitting maintenance information about the extracted repair object to the wearable device, .

According to another aspect of the present invention, there is provided a wearable device including a photographing unit photographing a military equipment through a camera provided in a wearable device, a transmitting unit transmitting an image photographed by the equipment to a maintenance support server, And a display unit for displaying the maintenance information on the received maintenance object on the display, wherein the image includes a plurality of maintenance objects detected by the maintenance support server, Wherein at least one part object corresponding to at least one of the plurality of detected maintenance objects is recognized and the maintenance object is configured to detect at least one part object and information on the distance between the recognized at least one part object and each maintenance object, Wherein at least one of the plurality of detected maintenance objects Can provide a wearable device to the target object is determined.

The above-described task solution is merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and the detailed description of the invention.

According to any one of the above-mentioned objects of the present invention, there is provided a server, a method and a system for supporting the maintenance of military equipment that enables preemptive service support by predicting a maintenance object of military equipment and deriving a relative approach characteristic stochastically A wearable device can be provided.

A server, a method, and a wearable device for supporting maintenance of a military equipment that enables maintenance to be supported by improving the understanding of operation and maintenance of the equipment in maintenance of the military equipment can be provided.

A server, a method, and a wearable device for supporting maintenance of military equipment that provides case-based maintenance information through a three-dimensional screen of a wearable device in order to provide convenience to a maintenance person while performing maintenance of the military equipment.

A server, a method, and a wearable device for supporting the maintenance of military equipment that increases the utility of maintenance through accident prevention and malfunction prediction in the machine maintenance industry, such as maintenance of military equipment, can be provided.

1 is an exemplary diagram illustrating a maintenance support system for a military equipment in accordance with an embodiment of the present invention.
2 is a configuration diagram of a wearable device according to an embodiment of the present invention.
3 is an exemplary diagram illustrating a process of displaying maintenance information on a maintenance object in a wearable device according to an exemplary embodiment of the present invention.
4 is a flowchart of a method for receiving maintenance support for military equipment in a wearable device according to an embodiment of the present invention.
5 is a configuration diagram of a maintenance support server according to an embodiment of the present invention.
FIG. 6 is an exemplary diagram for explaining a process of determining a maintenance object by applying information on a distance between a recognized part object and each maintenance object according to an embodiment of the present invention to a case-based reasoning algorithm.
7A and 7B illustrate a process of determining a maintenance target object based on information on a distance between a part object and each maintenance object and a case classification tree generated based on the similarity table according to an embodiment of the present invention Fig.
8 is a flowchart of a method of supporting maintenance of military equipment in a maintenance support server according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "including" an element, it is to be understood that the element may include other elements as well as other elements, And does not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

In this specification, the term " part " includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be implemented using two or more hardware, or two or more units may be implemented by one hardware.

In this specification, some of the operations or functions described as being performed by the terminal or the device may be performed in the server connected to the terminal or the device instead. Similarly, some of the operations or functions described as being performed by the server may also be performed on a terminal or device connected to the server.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram illustrating a maintenance support system for a military equipment in accordance with an embodiment of the present invention. Referring to FIG. 1, a maintenance support system 1 for a military equipment may include a wearable device 110 and a maintenance support server 120. The wearable device 110 and the maintenance support server 120 illustratively illustrate components that can be controlled by the maintenance support system 1 for military equipment.

Each component of the maintenance support system 1 for the military equipment of FIG. 1 is generally connected via a network. For example, as shown in FIG. 1, the wearable device 110 may be connected to the maintenance support server 120 at the same time or at intervals.

The network refers to a connection structure in which information can be exchanged between each node such as terminals and servers. The network includes a local area network (LAN), a wide area network (WAN), a wide area network (WWW) Wide Web, wired / wireless data communication network, telephone network, wired / wireless television communication network, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP, LTE, WIMAX, Wi-Fi, Bluetooth, infrared, Communications, Visible Light Communication (VLC), LiFi, and the like.

The wearable device 110 can photograph the military equipment 100 through a camera provided in the wearable device 110. [

The wearable device 110 may transmit the image of the military equipment 100 to the maintenance support server 120. [

The wearable device 110 may receive maintenance information on the object to be serviced of the military equipment 100 from the maintenance support server 120. [ The maintenance information may include, for example, a maintenance guide, a maintenance history, a list of part objects used to maintain the target maintenance object, and the like.

The wearable device 110 may display maintenance information on the received maintenance object on the display. For example, the wearable device 110 may display maintenance information in a plurality of output areas in the form of augmented reality, virtual reality, or mixed reality.

One example of such a wearable device 110 may include a HoloLens, a Smart Glass, a HMD (Head Mounted Display), or a Head Up Display (HUD) that can be worn on the body of a maintenance person.

The maintenance support server 120 stores information related to maintenance of military equipment in the database, maintenance history information, similarity information between tasks and detail work items related to maintenance, and feedback information on the determined target maintenance objects inputted from the maintenance person .

If the mechanic of the military equipment 100 intends to repair, it is possible to recall and repair the military equipment 100 having similar symptoms through pattern analysis of the past maintenance, So that the solution can be imitated or copied and solved.

The maintenance support server 120 can receive images of the military equipment 100 from the wearable device 110 worn on the body of the maintenance person.

The maintenance support server 120 may detect a plurality of maintenance objects from the image and recognize at least one part object corresponding to at least one of the plurality of detected maintenance objects.

The maintenance support server 120 can measure the distance between at least one recognized part object and each maintenance object. At this time, the maintenance support server 120 can determine the access status of each of at least one or more component objects moving by the maintenance person from the image to the maintenance object.

For example, the maintenance support server 120 may determine the state of access to the maintenance object of each of at least one part object in the first unit time and the state of access to each of the at least one part object in the second unit time after the first unit time The maintenance target object can be extracted based on the access state to the maintenance object.

The maintenance support server 120 may extract pixels from at least one recognized part object. At this time, the maintenance support server 120 may group the extracted at least one or more pixels into the similar pixel group.

The maintenance support server 120 extracts the clustered pseudo pixel group into a plurality of candidate maintenance areas, and calculates a distance between the at least one part object recognized among the extracted plurality of candidate maintenance areas and the distance between each maintenance object The first maintenance area or the second maintenance area. For example, the first maintenance area includes a maintenance object for performing maintenance, and the second maintenance area includes a maintenance object for proceeding maintenance. The maintenance support server 120 stores the maintenance object in the first maintenance area The corresponding maintenance object can be extracted as the maintenance object.

The maintenance support server 120 can determine at least one of the plurality of detected maintenance objects by applying information on the distance between the recognized at least one part object and each maintenance object to the case based reasoning algorithm. Here, the case-based reasoning algorithm may include a similarity table based on the access state between the part object and the maintenance object.

For example, the maintenance support server 120 generates a case classification tree based on the information about the distances between the recognized at least one part object and each maintenance object and the similarity table, and generates at least One or more objects to be repaired can be determined.

The maintenance support server 120 may transmit maintenance information on the extracted maintenance object to the wearable device 110. [ The maintenance information may include, for example, a maintenance guide, a maintenance history, a list of part objects used to maintain the target maintenance object, and the like.

2 is a configuration diagram of a wearable device according to an embodiment of the present invention. Referring to FIGS. 1 and 2, the wearable device 110 may include a photographing unit 210, a transmitting unit 220, a receiving unit 230, and a display unit 240.

The photographing unit 210 can photograph the military equipment 100 through a camera provided in the wearable device 110.

The transmission unit 220 may transmit the image of the military equipment 100 to the maintenance support server 120. For example, the transmitting unit 220 may transmit the holographic lens image of the military equipment 100 to the maintenance support server 120.

The receiving unit 230 may receive maintenance information on the object to be serviced of the military equipment 100 from the maintenance support server 120. [ The maintenance information may include, for example, a maintenance guide, a maintenance history, a list of part objects used to maintain the target maintenance object, and the like.

The display unit 240 may display maintenance information on the received maintenance object on the display. For example, the display unit 240 may display maintenance information in a plurality of output areas of the wearable device 110 in the form of an augmented reality, a virtual reality, or a mixed reality.

3 is an exemplary diagram illustrating a process of displaying maintenance information on a maintenance object in a wearable device according to an exemplary embodiment of the present invention. Referring to FIG. 3, the wearable device 110 may display maintenance information on a maintenance object on the display 300. FIG.

For example, the wearable device 110 may display maintenance guide information for a maintenance object in a first area 310 of the display 300 in the form of a virtual reality (VR).

The wearable device 110 can display a maintenance guide image for a maintenance target object in the form of a virtual reality for a maintenance person, a maintenance manual for supporting voice / image / text through the first area 310 of the display 300 have.

For example, the wearable device 110 may display the maintenance history of the object to be protected in the first area 310 of the display 300 in the form of an Augmented Reality (AR).

The wearable device 110 may output the augmented reality, virtual reality, or mixed reality-based maintenance support image to the second area 320 of the display 300. [

For example, the wearable device 110 displays a nearest-neighbor distance depiction technique for application of a maintenance object recognition technique through case-based reasoning about maintenance status for maintenance support, The maintenance can be supported by easily expressing the access object and the parts access within the maintenance history of the base.

In another example, the wearable device 110 may display a maintenance toolbox (including part objects and information about it) in a third area 330 of the display 300. The wearable device 110 displays parts necessary for the maintenance target object extracted from the third area 330 of the display 300 and parts that the maintenance person can select and provides an interaction matrix related to maintenance support can do.

4 is a flowchart of a method for receiving maintenance support for military equipment in a wearable device according to an embodiment of the present invention. The method for receiving the maintenance support of the military equipment 100 in the wearable device 110 shown in FIG. 4 may be performed by the maintenance support system 1 for the military equipment according to the embodiment shown in FIGS. Lt; / RTI > Therefore, the present invention is also applied to a method of providing maintenance support for the military equipment 100 in the wearable device 110 according to the embodiment shown in FIGS. 1 to 3 even if omitted from the following description.

In step S410, the wearable device 110 may photograph the military equipment 100 through a camera provided in the wearable device 110. [

In step S420, the wearable device 110 may transmit the image of the military equipment 100 to the maintenance support server 120. [

In step S430, the wearable device 110 may receive maintenance information on the object to be serviced of the military equipment 100 from the maintenance support server 120. [ Here, the maintenance information may include a maintenance guide, a maintenance history, a list of part objects used for maintenance of the target maintenance object, and the like.

In step S440, the wearable device 110 may display maintenance information on the received maintenance object on the display. For example, it is possible to output each of the plurality of output regions of the wearable device 110 in the form of an augmented reality, a virtual reality, or a mixed reality.

In the above description, steps S410 to S440 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some of the steps may be omitted as necessary, and the order between the steps may be switched.

5 is a configuration diagram of a maintenance support server according to an embodiment of the present invention. 1 and 5, the maintenance support server 120 includes a database 510, an image receiving unit 520, an object recognizing unit 530, a pixel extracting unit 540, a distance measuring unit 550, A target object extracting unit 560 and a transmitting unit 570.

The database 510 may include information related to maintenance of the military equipment, maintenance history information, similarity information between task and detail work items for maintenance related jobs, and feedback information on the determined target maintenance objects received from the wearable device .

The information related to the maintenance of the military equipment (100) refers to practical knowledge emphasizing the operational aspects of the general knowledge of the task, It can provide accumulated know-how, methodology and experience.

The maintenance history information is a record of tasks related to the maintenance of all the various military equipment 100. The maintenance history information is composed of metadata constituted by core information about each of the records. The records include, for example, digital data, Reports, books, minutes, logbooks, nostalgia, notes, and the like.

The similarity information means the relation between task and detail work items identified as a result of maintenance related job analysis. The initial score of the similarity table is set from the lowest score to the highest score according to the degree of relation between items, Quot; < / RTI >

The feedback information can provide a feedback on the case where the maintenance person is recommended (the determined object to be repaired and the maintenance information on the object), so that the value of the existing similarity table can be continuously and automatically evolved from the initial value. Here, the feedback may be a user judgment / evaluation of how the maintenance person can refer to the original data of the recommendation case and then help solve the problem.

Thus, by constructing the maintenance situation or knowledge obtained from the experience of the maintenance process in the database 510, it is possible to provide information on the situation through the solution associated with the problem. In addition, by considering similar problems or cases among similar experiences in the past, it is possible to find a clue for solving the current problem from the case or to control the addition or subtraction as a reference point of judgment.

In addition, if an empirical case is extracted and tracked from a specialist or an administrator, and any new judgment (decision making) is required, a comparative analysis is made of cases similar to those currently encountered among various experiences accumulated in the database 510 It is possible to improve the ability to solve pending issues.

The image receiving unit 520 can receive images of the military equipment 100 from the wearable device 110 worn on the body of the maintenance person. For example, the image receiving unit 520 can receive a holographic lens image of the military equipment 100 photographed from the wearable device 110 worn on the body of a maintenance person.

The object recognition unit 530 may detect a plurality of maintenance objects from the image and recognize at least one part object corresponding to at least one of the plurality of detected maintenance objects. For example, the object recognition unit 530 may divide a frame of an image into a plurality of cells and detect at least one part object by detecting an object and a boundary line in the image through a plurality of divided cells .

The object recognition unit 530 recognizes at least one part object included in the object to be maintained and extracts the position of the recognized part object. For example, the object recognition unit 530 may detect the boundary of at least one part object by applying the extracted position of at least one part object to the cells of the image frame.

The pixel extracting unit 540 may extract pixels from at least one recognized part object. At this time, the pixel extracting unit 540 may group the extracted at least one or more pixels into a similar pixel group.

For example, the pixel extractor 540 may extract pixels from the recognized part object using the k-means technique, and group them into similar pixel groups. The reason for clustering is that features and states are required for each position of the recognized component object, and there may be differences in the number of features and states and precise rules of prediction depending on the size of the position unit. Therefore, it is possible to classify clusters by defining the number of clusters in anticipation that they will be divided into predetermined clusters in a method of clustering two-dimensionally without consideration of clusters.

The distance measuring unit 550 may measure the distance between at least one recognized part object and each maintenance object.

The distance measuring unit 550 can determine the access state of each of at least one or more component objects moving by the maintenance person from the image to the maintenance object. Here, the approach state means a distance, a distance, and a time-dependent degree of change between objects to be maintained of the military equipment 100. The distance measuring unit 550 calculates a relative state of access to each maintenance object of the component object can do.

For example, the distance measuring unit 550 may determine the access state of each of the part objects to the maintenance object as one of Neutral, Inward, and Outward. Neutral means that the maintenance object is not detected in the image, Inward means that the part object is detected in the image and is approaching the maintenance object, and Outward type means that the part object in the image moves away from the maintenance object And the resources required for the computation and storage of the access state can be reduced by using these three state values. In this case, variables indicating various state transitions can be varied by considering the mobility state value at the immediately preceding time point and the mobility state value at the present time point.

For example, in the case of the [Newtral, Inward] state or the [Outward, Inward] state indicating the mobility state value at the previous time point and the mobility state value at the current time point, And the relative approach state that is gradually approaching the maintenance object as time passes.

Here, the unit of distance when determining the access state may vary depending on the object recognition method, the measuring instrument, and the like. For example, a distance based on a specific length, such as m or cm, may be used for a range that can be measured for image object recognition in a holographic lens display, or a distance between part objects in a maintenance object may be used as a reference. At this time, the object recognition unit 530 can derive a relative approach state between the maintenance object and the part object, for all the maintenance items.

The reason why the access state is determined is that, in the case of the maintenance object and the component object, the accessory state may have stability, but may have mobility, and the wearable device 110 moves, It is also applied when the relative approach state of the part object changes.

The distance measuring unit 550 measures the state of access to the maintenance object of each of at least one part object at the first unit time (previous point in time) and at least one or more of the access state of the at least one part object at the second unit time The object to be repaired can be extracted based on the access state of each of the part objects to the maintenance object.

The access state to each maintenance object of the part object can be judged to be eight in total. Actually, the access state of each part of the component object to the maintenance object can be judged to be 9, but since the possibility of the state transition of [N, O] is very low, State transitions can be generated.

The state transition is an index expression representing the mobility state transition in the eight state sets S, which can be expressed as (moving state value at the previous time point, moving state value at the current time) = (N, N).

That is, the mobility state transition degree of the next time can be expressed by using the mobility state value information of k past and present time points from time t-k + 1 to t. In this case, the set S = {(N, N), (N, I), (I, I) , N)}.

The distance measuring unit 550 extracts the clustered pseudo pixel group into a plurality of candidate maintenance areas, and calculates a distance between the at least one component object recognized among the extracted candidate maintenance areas and the distance between each repair object, The first maintenance area or the second maintenance area. For example, the distance measuring unit 550 may extract a maintenance object corresponding to the first maintenance area as a maintenance object.

Here, the first maintenance area may be a positive maintenance area in which maintenance can proceed or maintenance needs to be carried out, and the second maintenance area may be a negative maintenance area in which maintenance is impossible or maintenance is not required .

Here, the positive maintenance area may be an area including maintenance objects that need maintenance based on the maintenance histories of the plurality of recognized maintenance objects. Further, it may be an area including a maintenance object that is compatible with the recognized spare parts. In addition, the recognized spares can be areas containing maintenance objects that need to be replaced.

On the other hand, the negative maintenance area may be an area including a maintenance object that requires no maintenance based on the maintenance histories of the plurality of recognized maintenance objects. It may also be an area containing maintenance objects that are incompatible with the recognized spare parts. Further, the recognized spare parts may be areas containing maintenance objects that do not need to be replaced.

The maintenance target object extracting unit 560 can determine at least one or more repair target objects among a plurality of detected maintenance objects by applying information on the distance between at least one recognized part object and each maintenance object to the case based reasoning algorithm have.

Here, the case-based reasoning algorithm may include a similarity table based on the access state between the part object and the maintenance object. For example, the maintenance subject object extracting unit 560 generates a case classification tree based on the information about the distances between the recognized at least one part object and each maintenance object and the similarity table, and based on the generated case classification tree So as to determine at least one or more objects to be repaired.

The similarity table includes (O, I) and (I, I) as a first example and (N, N) and (O, N) and (O, O) as the second example, and (N, I) and (I, N) as the third case.

The transmitting unit 570 may transmit the maintenance information about the extracted maintenance object to the wearable device 110. The maintenance information may include a maintenance guide, a maintenance history, a list of part objects used to maintain the target maintenance object, and the like.

FIG. 6 is an exemplary diagram for explaining a process of determining a maintenance object by applying information on a distance between a recognized part object and each maintenance object according to an embodiment of the present invention to a case-based reasoning algorithm.

6, the maintenance support server 120 detects an object in the plurality of maintenance objects 620 (an object in 630) from the image, and detects at least one of the detected objects in the plurality of maintenance objects 620 A part object 610 (which is being moved by a maintenance person) can be recognized. The maintenance support server 120 recognizes the objects in the plurality of maintenance objects 620 as the positive maintenance area or the negative maintenance area according to the modeling data, Can determine whether maintenance is allowed or not, and notify the maintenance object.

For example, the maintenance support server 120 can not perform maintenance on the first maintenance object 620 when the part object 610 approaches the first maintenance object 620 recognized as a negative maintenance area. To the wearable device (110).

In another example, when the parts support object 610 approaches the second maintenance object 630, which is recognized as a positive maintenance area, the maintenance support server 120 performs maintenance for the second maintenance object 630 And transmit the notification message to the wearable device 110.

The maintenance support server 120 can provide a probability-based service according to the access characteristics of the component objects having mobility based on the probability information obtained from the mathematical model when the event is generated by recording the relative access characteristics of the maintenance object have.

7A and 7B illustrate a process of determining a maintenance target object based on information on a distance between a part object and each maintenance object and a case classification tree generated based on the similarity table according to an embodiment of the present invention Fig.

FIG. 7A is an exemplary diagram illustrating a similarity table according to an embodiment of the present invention. In FIG. 7A, there is shown an example in which observations about individual cases are collectively set up and their common properties are established as generalized propositions, It is a drawing that is used to deduce the proposition of the maintenance process.

Referring to FIG. 7A, the maintenance support server 120 can generate a similarity table by classifying the cases for finding the results of the characteristic and characteristic values in the maintenance history. The affinity table may be configured to include a high probability of maintenance 710, very low 711, very high 712, and low 713 for each case 700.

For example, the maintenance support server 120 classifies a case (Inward, Inward) when the possibility of maintenance is high (Outward, Inward) to be very high (Inward, Inward) as the first case 701, Neutral, and Neutral), and Inward and Outward cases (Inward, Outward) are classified as the second case. In the case where the possibility of maintenance is high (Neutral, Inward) (Outward, Neutral), and the possibility of maintenance (Outward, Outward) can be categorized as the fourth case.

That is, the maintenance support server 120 can classify a total of eight access states into four cases consisting of a pair of an access state at a previous time and an access state at a current time.

FIG. 7B is an exemplary diagram illustrating a case classification tree according to an embodiment of the present invention. In FIG. 7B, items to be compared are listed at each node to classify or determine the proposition of inference approached by an inductive method. By arranging the results or conditions, the accessibility of the exploration step is improved, so that the operation speed can be increased and a lot of data can be quickly accessed where specific data is located.

Referring to (a) of FIG. 7, a determination of maintenance cases is performed by applying a binary search tree which determines a sequence for quickly searching for classification and operation of the determined maintenance case. For example, assuming that all the data in the tree should be different from each other, the examples of the node tree on the left side of the parent node consist of the values contained in the maintenance area of the parent node, and the case of the node tree on the right side of the parent node, And a value included in the area other than the area.

Referring to FIG. 7B (b), the determination of the normal state of the determined maintenance case is shown. This is much more time-consuming than searching the entire maintenance area.

8 is a flowchart of a method of supporting maintenance of military equipment in a maintenance support server according to an embodiment of the present invention. The method of supporting the maintenance of the military equipment 100 in the maintenance support server 120 shown in FIG. 8 is performed by the maintenance support system 1 for the military equipment according to the embodiment shown in FIGS. 1 to 7B, Lt; / RTI > Therefore, even if omitted in the following description, the present invention is also applied to a method of supporting maintenance of the military equipment 100 by the maintenance support server 120 according to the embodiment shown in FIGS. 1 to 7B.

In step S810, the maintenance support server 120 may receive the image of the military equipment from the wearable device 110 worn on the body of the maintenance person.

In step S820, the maintenance support server 120 may detect a plurality of maintenance objects from the image and recognize at least one or more part objects corresponding to at least one of the plurality of detected maintenance objects.

In step S830, the maintenance support server 120 may measure the distance between at least one recognized part object and each maintenance object.

In step S840, the maintenance support server 120 determines at least one of the plurality of detected maintenance objects by applying information on the distance between the recognized at least one part object and each maintenance object to the case-based inference algorithm .

In step S850, the maintenance support server 120 may transmit the maintenance information on the extracted maintenance object to the wearable device 110. [

In the above description, steps S810 to S850 may be further divided into further steps, or combined in fewer steps, according to an embodiment of the present invention. In addition, some of the steps may be omitted as necessary, and the order between the steps may be switched.

A method for providing maintenance support for military equipment in the wearable device described in Figures 1 to 8 and a method for providing maintenance support for military equipment in a maintenance support server may be performed by a computer program or computer But may also be embodied in the form of a recording medium including executable instructions. Further, a method of providing maintenance support of military equipment in the wearable device described above with reference to FIGS. 1 to 8 and a method of providing maintenance support of military equipment in the maintenance support server may be implemented in the form of a computer program stored on a medium . ≪ / RTI >

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. The computer-readable medium may also include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: military equipment
110: a wearable device
120: maintenance support server
210:
220:
230: Receiver
240:
510: Database
520:
530: Object recognition unit
540:
550: distance measuring unit
560: a maintenance target object extracting unit
570:

Claims (20)

A server for supporting maintenance of military equipment,
An image receiving unit for receiving an image of a military equipment from a wearable device worn on a body of a maintenance person;
An object recognition unit for detecting a plurality of maintenance objects from the image and recognizing at least one or more part objects corresponding to at least one of the plurality of detected maintenance objects;
A distance measuring unit for measuring a distance between the recognized at least one part object and each maintenance object;
A maintenance object extracting unit for determining at least one maintenance object among the plurality of detected maintenance objects by applying information on the distance between the recognized at least one part object and each maintenance object to the case based reasoning algorithm; And
And a transmission unit for transmitting maintenance information on the extracted maintenance object to the wearable device,
Wherein the distance measuring unit determines an access state of the at least one or more component objects moved by the maintenance person from the image to the respective maintenance objects,
Wherein the distance measuring unit measures the state of access of the at least one component object to the respective maintenance object in a first unit time and the access state of the respective at least one component object in the second unit time after the first unit time And extracts the maintenance target object based on an access state to the maintenance object.
The method according to claim 1,
Wherein the maintenance information includes at least one of a maintenance guide, a maintenance history, and a list of parts objects used for maintenance of the maintenance object.
3. The method of claim 2,
Wherein the maintenance information is output to each of a plurality of output areas of the wearable device in the form of an augmented reality, a virtual reality, or a mixed reality.
The method according to claim 1,
A database including information related to the maintenance of the military equipment, maintenance history information, similarity information between tasks for maintenance-related tasks and detail work items, and feedback information on the determined target maintenance objects inputted from the maintenance personnel In, server.
delete The method according to claim 1,
And a pixel extracting unit for extracting pixels from the recognized at least one part object,
Wherein the pixel extracting unit groups the extracted at least one or more pixels into a group of similar pixels.
delete The method according to claim 1,
Wherein the distance measuring unit calculates the distance between the at least one component object in the first unit time and the access state of the at least one part object in the second unit time after the first unit time, The maintenance object is classified into the first maintenance area or the second maintenance area based on the access state of the maintenance object to the maintenance object and the maintenance object corresponding to the first maintenance area is extracted as the maintenance object,
Wherein the first maintenance area includes a maintenance object for performing maintenance,
And the second maintenance area includes a maintenance object for proceeding maintenance.
The method according to claim 1,
The case-based reasoning algorithm includes a similarity table based on an access state between a part object and a maintenance object,
Wherein the maintenance object extraction unit generates a case classification tree based on the recognized similarity table and information on the distance between the recognized at least one part object and each maintenance object, and based on the generated case classification tree, And determines the object to be serviced as described above.
In a method for supporting maintenance of military equipment in a server,
The method comprising: receiving an image of a military equipment from a wearable device worn on a body of a maintenance person;
Detecting a plurality of maintenance objects from the image, and recognizing at least one part object corresponding to at least one of the plurality of detected maintenance objects;
Measuring a distance between the recognized at least one part object and each maintenance object;
Determining at least one repair object among the plurality of detected repair objects by applying information on the distance between the recognized at least one part object and each repair object to a case based reasoning algorithm; And
And transmitting maintenance information on the determined maintenance object to the wearable device,
Wherein the step of measuring the distance comprises determining an access state of the at least one or more part objects moving by the maintenance person from the image to the respective maintenance object,
Wherein the step of measuring the distance comprises determining the state of access of the at least one part object to the respective maintenance object in a first unit time and the state of access of the at least one part object in the second unit time after the first unit time And the maintenance object is extracted based on an access state to each of the maintenance objects.
11. The method of claim 10,
Wherein the maintenance information includes at least one of a maintenance guide, a maintenance history, and a list of parts objects used for maintenance of the maintenance object.
12. The method of claim 11,
Wherein the maintenance information is output to each of a plurality of output areas of the wearable device in the form of an augmented reality, a virtual reality, or a mixed reality.
delete 11. The method of claim 10,
Further comprising extracting pixels from the recognized at least one part object,
Wherein extracting the pixels comprises grouping the extracted at least one or more pixels into a group of similar pixels.
delete 11. The method of claim 10,
Wherein the step of measuring the distance includes determining a state of access to the respective maintenance object of the at least one component object in the first unit time and a state of access to the at least one part object in the second unit time after the first unit time, The maintenance object is classified into the first maintenance area or the second maintenance area based on the access state of the maintenance object to the maintenance object and the maintenance object corresponding to the first maintenance area is extracted as the repair object However,
Wherein the first maintenance area includes a maintenance object for performing maintenance,
Wherein the second maintenance area includes a maintenance object for proceeding maintenance.
11. The method of claim 10,
The case-based reasoning algorithm includes a similarity table based on an access state between a part object and a maintenance object,
Wherein the step of determining the object to be maintained comprises the steps of: generating a case classification tree based on the recognized distance information between the at least one component object and each of the maintenance objects and the similarity table; And determines the at least one or more objects to be serviced.
delete delete delete

Images