KR20210062156A - System for maintenance of facilities on site - Google Patents

Abstract

An on-site facility maintenance system according to one embodiment of the present invention comprises: a facility management server that identifies a location of the facility by receiving a fail signal for the facility, acquires fail information for the facility by analyzing the fail signal, and extracts AR contents corresponding to the acquired fail information; and an AR device that receives and provides AR contents extracted from the facility management server from the facility management server, wherein the AR contents may be for providing guide information for maintenance of the facility based on the fail information. Therefore, the present invention is capable of having an effect of constructing quick information confirmation and remote augmentation support services.

Description

Field facility maintenance system {SYSTEM FOR MAINTENANCE OF FACILITIES ON SITE}

The present invention relates to a maintenance support system capable of providing guide information required to maintain field facilities using an augmented reality (AR) display device. More specifically, based on an augmented reality that can provide guide information for facility maintenance by using augmented reality (AR) content corresponding to the fault information by receiving a fault signal from the facility to identify the location of the facility and obtaining the fault information. It relates to the maintenance system of the.

In recent years, with the advent of the industrial revolution, which establishes an inter-device communication system through the Internet of Things (IoT) and optimizes the entire process, the role of smart factories is emerging. A smart factory is an intelligent factory that can improve productivity, quality, and customer satisfaction by applying information and communication technology to the entire revolution, such as product design, development, manufacturing, distribution, and logistics. According to this, it meant that the facilities were passively moving, but in the 4th Industrial Revolution, various facilities were changing to an active working method depending on the product and the situation.

Nevertheless, in order to maintain and manage various and complex facilities in a smart factory, maintenance and repair procedures by skilled workers with many years of experience were essential.

However, even if it is an experienced person, it is difficult to understand the maintenance process for all facilities, and in an environment where it is difficult to predict when and what maintenance requests will occur, maintenance workers with manuals are required for facility maintenance and repair in a smart factory. have.

Conventional manuals for facility maintenance have been generally produced in the form of a book or a two-dimensional image. The problem with these manuals is that even one part has to find the page that corresponds to a breakdown, and if the manual is old, there is a risk of discoloration or damage, and in the case of devices with complex mechanical structures such as aircraft, a considerable amount of manuals are required. Therefore, there is also a problem that is difficult to carry.

In order to solve the above problem, technologies for providing a manual as contents produced using virtual reality (VR) or augmented reality (AR) technology have been developed.

Virtual reality (VR) technology blocks the real world and shows only virtual images such as cyberspace. Using computer graphics or camera technology, images having various three-dimensional effects can be created. In addition, augmented reality technology is a technology that displays virtual information in addition to the information seen in reality as a technology that displays virtual information over the real world. In addition, recently, a mixed reality (MR) technology, which is a technology that combines or utilizes the advantages of augmented reality (AR) and virtual reality (VR), has also been developed.

That is, a technology that provides maintenance guide information along with a manual in temporal and spatial terms to workers who understand the operating relationship of complex equipment or facilities using the above virtual reality or augmented reality technology and check the maintenance or normality. Are also being developed.

Republic of Korea Patent Publication No. 10-2005-0055507 (published date: 2005.06.13)

The present invention is intended to augment facility status and IoT sensor information in a smart factory in real time on an augmented reality (AR) display device, and establish a manual support enhancement service to establish a quick information confirmation and remote enhancement support service through on-site personnel. There is a purpose.

In addition, the purpose of the autonomous driving robot is to provide an on-site facility maintenance system capable of accurately specifying the location of the facility by obtaining and providing a fault signal from the facility, thereby providing guide information for specific facility maintenance.

As an embodiment of the present invention, an on-site facility maintenance system may be provided.

The on-site facility maintenance system according to an embodiment of the present invention receives a fault signal for the facility, identifies the location of the facility, and analyzes the fault signal to obtain fault information for the facility, and an AR corresponding to the acquired fault information. A facility management server that extracts content and an AR device that receives and provides AR content extracted from the facility management server from the facility management server, and the AR content is for providing guide information for facility maintenance based on failure information. I can.

In the field facility maintenance system according to an embodiment of the present invention, the facility management server receives a fault signal from a communication module provided in the facility, or for a facility generated by an autonomous driving robot that moves a factory in which one or more facilities are provided. A fault signal can be received.

In the field facility maintenance system according to an embodiment of the present invention, the received fault signal may be learned in advance based on a deep learning algorithm, and fault information may be generated according to a result analyzed using an analysis model generated.

In the field facility maintenance system according to an embodiment of the present invention, a fault signal is obtained from one or more sensors attached to the facility, and the sensor may include a temperature sensor, a humidity sensor, an IMU sensor, an image sensor, and a positioning sensor. .

In the field facility maintenance system according to an embodiment of the present invention, the AR device includes: a camera for photographing the front, a data storage unit for storing AR contents provided from the facility management server, and a communication device for transmitting and receiving data with the facility management server. And a display for displaying facility guide information generated by matching AR content to the front image captured by the camera.

According to the field facility maintenance system of the present invention, facility status and IoT sensor information in a smart factory are augmented in real time on an augmented reality (AR) display device, and a manual support enhancement service is established to quickly check information and remotely augment through field personnel. It has the effect of building support services.

In addition, since the autonomous driving robot acquires and provides a fault signal from the facility, the location of the facility can be accurately specified, thereby providing an on-site facility maintenance system capable of providing guide information for specific facility maintenance.

1 is an exemplary view showing a field facility maintenance system according to an embodiment of the present invention.
2 is a flow chart showing a maintenance support process using a field facility maintenance system according to an embodiment of the present invention.
3 is an exemplary view showing a state of confirming a location of a facility from a fault signal in a field facility maintenance system according to an embodiment of the present invention.
4 is a block diagram showing a facility management server in a field facility maintenance system according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

Terms used in the present invention have selected general terms that are currently widely used as possible while taking functions of the present invention into consideration, but this may vary according to the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. . In addition, when a part is said to be "connected" with another part throughout the specification, this includes not only the case of being "directly connected", but also the case of being connected "with another element in the middle."

1 is an exemplary view showing a field facility maintenance system according to an embodiment of the present invention, Figure 2 is a flow chart showing a maintenance support process using the field facility maintenance system according to an embodiment of the present invention, Figure 3 is the present In the field facility maintenance system according to an embodiment of the present invention, an exemplary view showing a state of confirming the location of a facility from a fault signal, and FIG. 4 is a facility management in the field facility maintenance system according to an embodiment of the present invention. It is a block diagram showing the server 100. Hereinafter, the present invention will be described in detail with reference to the above-described drawings.

As an embodiment of the present invention, an on-site facility maintenance system may be provided. In the present specification, the field facility maintenance system of the present invention may be a system for directly servicing site facilities, but may also be a maintenance support system for guiding necessary information in the process of a site operator servicing facilities. In addition, the field facilities of the present invention may represent facilities provided in a smart factory. A smart factory may be an intelligent factory that establishes a system of communication between devices through the Internet of Things (IoT) and optimizes the entire process. Of course, in addition to the smart factory, the field facilities of the present invention may represent various facilities provided in a general factory.

1 and 4, the on-site facility maintenance system according to an embodiment of the present invention receives a failure signal for a facility, identifies the location of the facility, and analyzes the failure signal to obtain failure information for the facility. , A facility management server 100 for extracting AR content corresponding to the acquired failure information, and an AR device 200 for receiving and providing AR content extracted from the facility management server 100 from the facility management server 100, and , AR content may be for providing guide information for maintaining facilities based on fault information.

The facility management server 100 according to an embodiment may include a memory (not shown) that stores one or more instructions and a processor 110 that executes one or more instructions stored in the memory. That is, the processor can be responsible for all necessary digital processing of the received data. That is, data processing of the modules may be operated based on a processor.

According to one disclosure, the processor may control at least one other component (eg, hardware or software component) of the facility management server 100 connected to the processor by driving software (eg, a program), for example, , Various data processing and operations can be performed. The processor may load and process commands or data received from other components into the volatile memory, and store result data in the nonvolatile memory. According to an embodiment, the processor is a main processor (eg, a central processing unit or an application processor), and operates independently of the main processor, and additionally or alternatively, uses lower power than the main processor, or a secondary processor specialized for a specified function (eg. : A graphic processing device, an image signal processor, a sensor hub processor, or a communication processor) may be included. Here, the secondary processor may be operated separately or embedded in the main processor.

According to one disclosure, the memory may store a program for processing and controlling the processor, and may store data input to or output from the device of the present invention. Programs stored in the memory may be classified into a plurality of modules according to their functions. Here, the plurality of modules are software, not hardware, and functionally operated modules. Programs stored in the memory may be classified into a plurality of modules according to their functions. Here, the plurality of modules are software, not hardware, and may refer to modules that operate functionally.

According to one disclosure, the facility management server 100 may include a communication unit. The communication unit may support data transmission/reception between components inside the facility management server 100, as well as establishing a wired or wireless communication channel with the AR device 200 and other devices to be described later, and performing communication through the established communication channel. The communication unit may include one or more communication processors that support wired communication or wireless communication operated independently of the processor. The communication unit is a wireless communication module (Ex. a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (Ex. a local area network (LAN) communication module, or a power line communication module). It may include.

According to one disclosure, the facility management server 100 may include a data analysis unit. The data analysis unit may prepare an analysis model generated by learning in advance based on a deep learning algorithm. The deep learning algorithm may include a convolutional neural network (CNN). In addition, the analysis model may be generated according to a learned result by combining with a recurrent neural network (RNN) in addition to the CNN. Hereinafter, it will be described based on the analysis model formed according to the CNN-based learning result. First, briefly explaining CNN, CNN can be a type of artificial neural network used to analyze input data, features are extracted from the input data in the feature extraction layer, and input data based on the features extracted from the classification layer. Can be classified as to which class is applicable. The feature extraction layer may include at least one convolutional layer and a pooling layer, and the classification layer is a fully connected layer including one hidden layer. ) Can be. The convolution layer may correspond to a filter that generates a feature map representing features of an object through a convolution operation. In the pooling layer, a pooling operation may be performed to reduce the size of the output data of the convolution layer or to emphasize specific data. The pooling layer may include a max pooling layer and an average pooling layer. The fully connected layer may correspond to a classifier for classifying input data based on the extracted feature information.

According to one disclosure, the facility management server 100 may generate a control signal generator. The control signal generator may generate a control signal for determining AR content to be provided to the AR device 200 on the basis of fault information according to a result of analyzing the fault signal using the analysis model of the data analyzer described above. In addition, a control signal for controlling the operation of other components of the facility management server 100 may be generated according to an instruction from the processor.

According to one disclosure, the AR content provider 160 may pre-classify and store a plurality of AR content. The AR contents providing unit 160 may store AR contents in a state classified in advance according to various categories such as a type of facility and a maintenance method.

According to one disclosure, the AR device control unit 170 separately controls the AR device 200 to facilitate maintenance of facilities according to communication with the AR device 200 while providing AR content to the AR device 200. In order to do so, it may be included in the facility management server 100.

In the field facility maintenance system according to an embodiment of the present invention, the AR device 200 includes a camera for photographing the front, a data storage unit for storing AR contents provided from the facility management server 100, and a facility management server ( 100) and a communication device for transmitting and receiving data, and a display for displaying facility guide information generated by matching AR content to a front image captured through a camera.

According to one disclosure, the AR device 200 refers to a display device for providing 3D augmented reality content to a user or a worker, and the AR device 200 is formed in the form of a head mounted display. However, the present invention is not limited thereto, and various devices may be included in the AR device 200 as long as a display device capable of providing AR content. The camera provided in the AR device may be fixed so as to look in a predetermined direction, but unlike this, it may be provided in a rotating manner provided with a separate rotating device (not shown) so as to rotate about a predetermined axis.

Hereinafter, a process in which maintenance support is performed using the on-site facility maintenance system according to an embodiment of the present invention will be described with reference to FIG. 2.

First, it is possible to receive a fault signal for the facility (S100).

In the facility, a communication module for communication with other devices such as the facility management server 100 and the AR device 200 may be provided. In addition, the facility may include a temperature sensor, a humidity sensor, an IMU sensor, an image sensor, and a positioning sensor. That is, the facility may sense information about the environment (ex. temperature, humidity, etc.) around the facility as well as operation information of the facility from the sensor and transmit it to the facility management server 100. In addition, a marker for providing an AR image may be provided in the facility at a predetermined location of the facility. That is, when providing AR content to be described later, it may be primarily recognized and tracked by the marker. In addition, facility information including each unique facility number may be set in advance in the facility. Unlike this, the facility management server 100 may generate facility information including a unique facility number for each facility.

In S100, a fault signal may be directly received from the facility (S110), or a fault signal for the facility 20 may be received from the autonomous driving robot 30 (S120). The present invention is applied to a smart factory or an intelligent factory in which a plurality of facilities 20 are provided, and the factory may be provided with an autonomous driving robot 30 provided with various functions such as movement and maintenance, and the autonomous driving robot 30 ) Can receive a fault signal for the facility (20). In particular, although the location of the facility 20 can be checked using GPS, when the autonomous driving robot 30 directly extracts or receives a fault signal from the facility 20, the facility is installed according to the location of the autonomous driving robot 30. The location of (20) can be reconfirmed. In other words, the autonomous driving robot 30 is a mobile robot that can be linked with the facility 20 to check whether the facility 20 is malfunctioning, and the autonomous driving robot 30 transmits a malfunction signal of the facility 20 to the facility management server. In the process of transferring to 100, the location of the facility 20 can be confirmed through the autonomous driving robot 30.

Next, it is possible to check the location of the facility 20 from the fault signal (S200). As described above, in a state where the location of the facility 20 has been confirmed using the autonomous driving robot 30, the facility that generated a fault signal by matching the location of the facility 20 corresponding to the facility database through a field map ( 20) can be specified.

Referring to FIG. 3, the factory can be divided into a plurality of zones, and since one or more facilities 20 may be provided in each zone, in order to specify the location of the facility 20, check the area where the facility 20 is provided. This may be preceded.

Next, a fault signal may be analyzed using an analysis model generated as a result learned according to the deep learning algorithm, and fault information may be derived (S300) according to the analysis result. The failure information may include the presence or absence of a failure of the facility 20, a direction of maintenance of the facility 20, and the presence or absence of remaining parts of the facility 20.

Finally, a facility maintenance guide may be provided to the AR device 200 based on the failure information (S400). The facility maintenance guide is guide information for maintenance, and may be information based on the AR content stored in the AR content providing unit of the facility management server 100 in advance, and information based on a result of mapping a front image captured by the AR device 200 and a result of being mapped. . In addition, maintenance target, maintenance sequence, maintenance direction and difficulty, facility management manual, maintenance manual, etc. may be additionally displayed together.

According to the above-described process (S100-S400), the facility status and sensor information in the factory are transmitted to the AR device in real time to enhance it, and remote maintenance support according to quick failure information confirmation through a field worker may be possible.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

Although the preferred embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also present. It belongs to the scope of rights of That is, the scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

20: equipment 30: autonomous driving robot
100: facility management server 200: AR device

Claims (5)

In the field facility maintenance system,
A facility management server configured to receive a fault signal for a facility, identify a location of the facility, obtain fault information for the facility by analyzing the fault signal, and extract AR content corresponding to the acquired fault information; And
Including an AR device for receiving and providing the AR content extracted from the facility management server from the facility management server,
The AR content is for providing guide information for maintaining the facility based on the fault information,
Field facility maintenance system. The method of claim 1,
The facility management server receives a failure signal from a communication module provided in the facility, or receives a failure signal for a facility generated by an autonomous driving robot moving a factory equipped with one or more facilities,
Field facility maintenance system. The method of claim 1,
The received failure signal is learned in advance based on a deep learning algorithm, and the failure information is generated according to the analysis result using the generated analysis model,
Field facility maintenance system. The method of claim 1,
The fault signal is obtained from one or more sensors attached to the facility,
The sensor includes a temperature sensor, a humidity sensor, an IMU sensor, an image sensor and a positioning sensor,
Field facility maintenance system. The method of claim 1, wherein the AR device,
A camera for photographing the front side;
A data storage unit for storing AR contents provided from the facility management server;
A communication device for transmitting and receiving data to and from the facility management server; And
Including a display for displaying facility guide information generated by matching the AR content to the front image captured by the camera,
Field facility maintenance system.

Images