KR20190114155A - The Mobile Unmanned Monitoring System and Method - Google Patents

Abstract

The present invention provides a method for operating a movable unmanned monitoring system, which is capable of monitoring an object placed in an area difficult to access. According to the present invention, the method for operating a movable unmanned monitoring system comprises the steps of: allowing a pipebot to acquire first input information about a first object; transmitting the first input information to a central monitoring system (CMS); allowing the CMS to generate a first model based on the first input information; allowing the pipebot to acquire second input information about the first object and transmit the same to the CMS; allowing the CMS to compare the first model with the second input information to determine whether the first object is abnormal; and providing output information including information about whether the first object is abnormal, wherein the first and second input information about the first object include real image information and thermal image information, and the first model is a model generated by training a training data set based on the first input information in a machine learning type.

Description

Mobile Unmanned Monitoring System and Method

The present disclosure can provide a mobile unmanned surveillance system and method thereof. In particular, it is possible to provide a mobile unmanned surveillance system that provides surveillance information in consideration of machine learning.

With the development of the industry, various facilities have emerged, and the requirements for the maintenance and management of the various facilities are increasing. For example, pipes that were buried underground with industrial development have already been buried underground for a long time. Problems occur in buried pipes, and accidents are increasing. However, the above-mentioned underground pipes have a limitation in that a person approaches and monitors and maintains the entire area. In addition, as industrialization progresses, there are an increasing number of places where human access is difficult, such as inside a complex building. However, there is no special method for management and maintenance of these areas, and many problems are generated based on this.

Thermal imaging cameras may be used for management and maintenance of the above-described areas, but since thermal imaging cameras are expensive equipment, it may be difficult to install a plurality of equipments in large areas. In addition, since a thermal imaging camera is an image of a thermal image, it may be difficult to accurately recognize a target. In addition, in the case of CCTV, since only the fixed area is photographed at a fixed position, there is a limit in managing and maintaining a large area.

Recently, the technology for artificial intelligence has been developed, and the cases of applying it have been increasing. As an artificial intelligence, machine learning can be a system that collects relevant data to learn by itself and, based on it, produces better results on its own.

The present invention can solve the above problems by providing a mobile monitoring system and method.

It is an object of the present invention to provide a mobile unmanned surveillance system and method.

An object of the present specification is to provide a method for monitoring an object located in an area inaccessible to a person.

According to one embodiment of the present specification, a method of operating a mobile unmanned surveillance system may be provided. In this case, the operation method of the mobile unmanned surveillance system, the pipebot obtains the first input information for the first object, transmitting the first input information to the Central Monitoring System (CMS), the CMS to the first input information Generating a first model based on the method, the pipebot obtaining second input information about the first object, transmitting the second input information to the CMS, and comparing the first model with the second input information by the CMS. The method may include determining whether the first object is abnormal and providing output information including information on the determined abnormality. In this case, the first input information and the second input information for the first object include real image information and thermal image information, and the first model trains the training data set based on the first input information in a machine learning manner. It may be a model generated.

In addition, according to one embodiment of the present specification, a mobile unmanned surveillance system may be provided. The mobile unmanned surveillance system may include a pipebot and a CMS. At this time, the pipebot obtains the first input information about the first object, transmits the first input information to the central monitoring system (CMS), the CMS generates the first model based on the first input information, and the pipe The bot obtains the second input information about the first object, transmits the second input information to the CMS, and the CMS compares the first model and the second input information to determine whether the first object is abnormal, and to determine Output information including information on whether there is a problem may be provided. In this case, the first input information and the second input information for the first object include real image information and thermal image information, and the first model trains the training data set based on the first input information in a machine learning manner. It may be a model generated.

In addition, according to an embodiment of the present disclosure, the pipebot may include a transceiver for transmitting and receiving a signal, an input unit for obtaining input information, and a processor for controlling the transceiver and the input unit. In this case, the input unit may include a high definition (HD) camera and a thermal imaging camera.

In addition, according to one embodiment of the present specification, the CMS may include a transceiver for transmitting and receiving a signal, a machine learning learner applying machine learning to input data, and a processor controlling the transceiver and the machine learning learner.

In addition, the followings may be commonly applied to the mobile unmanned surveillance system and its operation method.

According to one embodiment of the present specification, the first input information and the second input information may further include at least one or more of location information, time information, audio information, voltage information, and current information.

In this case, according to one embodiment of the present specification, the first input information and the second input information may be determined based on the attributes of the first object.

In this case, according to one embodiment of the present specification, an attribute of the first object may be determined based on at least one of location information, size information, time information, and surrounding environment information of the first object.

In addition, according to one embodiment of the present specification, an analysis method for machine learning may be determined based on an attribute of the first object.

In this case, according to one embodiment of the present specification, the CMS may generate the training data set through the first input information, and generate the first model through the generated training data set.

In this case, according to one embodiment of the present specification, the CMS extracts information corresponding to the second input information from the first model, compares the extracted information with the second input information, and determines whether there is an abnormality with respect to the first object. Can be.

Further, according to one embodiment of the present specification, the second input information is real time input information, and the first model may be updated through learning every time the second input information is obtained.

In addition, according to one embodiment of the present specification, when a reference value for an attribute of the first object is preset, the first model may be performed based on a supervised learning analysis method of machine learning.

In addition, according to one embodiment of the present specification, the output information provided may be configured only with information on which an abnormality with respect to the first object is detected based on the second input information and the first model.

The present disclosure can provide a mobile unmanned surveillance system and method.

The present disclosure may provide a method for monitoring an object located in an area inaccessible to a person.

Effects obtained in the present specification are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a view conceptually showing the structure of a mobile unmanned surveillance system according to an embodiment of the present disclosure.
2 is a diagram illustrating a method for performing mobile unattended monitoring according to one embodiment of the present specification.
3 is a flowchart illustrating a method for performing mobile unattended monitoring according to an embodiment of the present specification.
4 is a view briefly illustrating a mobile unmanned surveillance system according to an embodiment of the present disclosure.
5 is a view showing a specific embodiment of a mobile unmanned surveillance system according to an embodiment of the present disclosure.
6 is a view showing the operation sequence of the mobile unmanned surveillance system according to an embodiment of the present disclosure.
7 is a flowchart illustrating a method of interoperating between a pipe bot and a CMS according to one embodiment of the present specification.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description, which will be given below with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, one of ordinary skill in the art appreciates that the present invention may be practiced without these specific details.

The following embodiments combine the components and features of the present invention in a predetermined form. Each component or feature may be considered to be optional unless otherwise stated. Each component or feature may be embodied in a form that is not combined with other components or features. In addition, some components and / or features may be combined to form an embodiment of the present invention. The order of the operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment.

Specific terms used in the following description are provided to help the understanding of the present invention, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present invention.

In some instances, well-known structures and devices are omitted or shown in block diagram form, centering on the core functions of each structure and device, in order to avoid obscuring the concepts of the present invention. In addition, the same components will be described with the same reference numerals throughout the present specification.

In addition, terms such as first and / or second may be used herein to describe various components, but the components should not be limited by the terms. The terms are only for the purpose of distinguishing one component from another component, for example, without departing from the scope of rights in accordance with the concepts herein, the first component may be called a second component, and similarly The second component may also be referred to as a first component.

In addition, throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding the other components unless otherwise stated. In addition, the terms “unit” and “unit” described in the specification mean a unit that processes at least one function or operation, which may be implemented by a combination of hardware and / or software.

1 is a diagram conceptually illustrating a structure of a mobile monitoring system according to an embodiment of the present specification.

In this case, as an example, the mobile monitoring system may include a PipeBot 100 and a Central Monitoring System 200. In addition, apparatuses necessary for performing the mobile monitoring method based on the mobile monitoring system may be further included, and are not limited to the above-described embodiment.

For example, the mobile monitoring system may further include a server or a database. In addition, a communication device for transmitting information may be further included.

In addition, as an example, the pipe bot 100 may be composed of a transceiver 110, a processor 120, and an input unit 130. In addition, the device may further include equipment necessary for memory or data collection, and is not limited to the above-described embodiment. In addition, the pipe bot 100 is only a name, it is not limited to the name. In more detail, the pipe bot 100 may be a movable device. In this case, the pipe bot 100 may have various sizes based on the monitoring area. For example, a pipe bot 100 equipped with a micro camera for a narrow area may be possible. Alternatively, a pipebot 100 equipped with a camera having excellent performance in consideration of a wide area may be possible. Also, as an example, the five bots 100 may refer to a device that is moved in a predetermined movement path. As another example, the pipe bot 100 may refer to a device that recognizes an object through a camera and sets a moving path by itself. That is, the pipe bot 100 may mean a device that is equipped with a camera and can operate unattended, and is not limited to the above-mentioned name.

As another example, the pipe bot 100 may be a device including the input unit 130. In this case, the input unit may include the camera described above, and may receive other sounds or other input information.

For example, the input unit 130 of the pipe bot 100 may include a high definition (HD) camera 131 and a thermal imaging camera 132. Also, as an example, the pipe bot 100 may further include at least one of a position sensor, a microphone, a voltage meter, and a current meter as the input unit 130. As an example, the pipebot 100 may obtain shape information of the object as image information of the object through the HD camera 131. In addition, the pipe bot 100 may obtain temperature information as image information of the object through the thermal imaging camera 132. In addition, the pipe bot 100 may obtain the position information of the pipe bot 100 through the position sensor. In addition, the pipebot 100 may obtain power related information using at least one of a voltage meter and a current meter. Also, as an example, the pipebot 100 may obtain audio information by using the microphone unit. That is, the input unit 130 may refer to a part for performing sensing to obtain information, and is not limited to the above-described embodiment.

In this case, Table 1 may be an example of information that can be obtained by the pipe bot 100 described above. However, Table 1 below is only one example and is not limited to the above-described embodiment.

TABLE 1

In addition, the processor 120 of the pipebot 100 may transmit the above-described information to the CMS 200 through the transceiver 110. For example, the processor 120 may include a machine learning function. The processor 120 may analyze the input information through the machine learning function to check whether the target is abnormal and perform monitoring on the target. However, when performing machine learning, the data processing efficiency needs to be excellent, and considering the processing speed or power consumption, it may not be desirable for the processor 120 of the pipebot 100 to directly process the data. have. Therefore, the processor 120 transmits input information to the CMS 200 through the transceiver 110, and the CMS 200 may analyze data about the input information, which will be described later. However, when the size of the pipe bot 100 is large and the processing power or the power consumption is less limited, it may be possible to directly perform a function for the CMS 200. 100 may also be performed and the present invention is not limited to the above-described embodiment.

Also, as an example, the processor 120 of the pipe bot 100 may control a movement path or a patrol path of the pipe bot. More specifically, the pipe bot 100 may be a device for monitoring a certain area based on a certain period. Therefore, it may be necessary to control the movement hardness of the pipe bot 100, the processor 120 may be to move the pipe bot 100 based on a predetermined path.

As another example, the processor 120 of the pipe bot 100 may set the movement path by itself based on the input information of the input unit 130. In more detail, the input unit 130 may obtain real-time input information as input information. In this case, the pipe bot 100 may transmit the input information of the input unit 130 to the CMS, and then receive a response about whether the target is abnormal from the CMS. At this time, the pipe bot 100 may set the movement path through the information on whether or not received from the CMS. For example, when an abnormality with respect to the object is detected, more detailed input information may be obtained by setting a surrounding of the object as a moving path. On the other hand, if no abnormality is detected, the pipe bot 100 may move to an existing movement path and obtain input information about another object, and is not limited to the above-described embodiment.

In this case, as an example, the CMS 200 may determine whether an object is abnormal through a training data set. In this case, the training data set may be a model generated by machine learning and past input information stored in the CMS 200 database. That is, the CMS 200 may generate a model for determining whether or not abnormality by learning the past input information by machine learning, and compare the real-time input information received from the pipebot 100 to determine whether there is an abnormality. Can be.

Through this, the CMS 200 may determine whether the object is abnormal, and may provide information about the abnormality to the pipebot 100 again. The pipebot 100 may determine whether additional input information about the object is needed based on the information received from the CMS 200. If the pipebot 100 determines that additional input information is needed, the pipebot 100 may obtain more specific information about the object and transmit it to the CMS 200. The CMS 200 may finally determine whether an object is abnormal based on more specific information about the object, and store and provide information about the object. In addition, the model may be updated by relearning additional information in the model of the above-described abnormality.

Next, the CMS 200 may include at least one of the transceiver 210, the processor 220, and the machine learning learner 230. In this case, the transceiver 210 of the CMS 200 may receive input data from the pipebot 100. In this case, as an example, the input data may be image information obtained from the above-described HD camera 131, image temperature information obtained from the thermal camera 132, and the like, and are not limited thereto.

The processor 220 of the CMS 200 may control the transceiver 210 described above. In addition, the processor 220 of the CMS 200 may further control the machine learning learner 230, which is not limited to the above-described embodiment. In this case, as an example, the machine learning unit 230 may be selectively included. For example, the processor 220 may directly perform the function of the machine learning learner 230. That is, the machine learning learning unit 230 may be configured to perform a separate rush machining, or the processor 220 may replace this function, and is not limited to the above-described embodiment. In this case, the machine learning learning 230 of the CMS 200 may monitor and control an object through a machine learning task based on the received input data. In this case, the video summary information may be extracted and provided based on the machine learning task, thereby performing monitoring on the target.

As another example, the CMS 200 may detect whether an object is abnormal through machine learning. That is, the CMS 200 may process the input information received from the pipebot 100 through machine learning, and then check whether the target is abnormal. In this case, the above-described video summary information may be summarized as information in which an abnormality is detected. In addition, as an example, the CMS 200 may feed back information on whether or not the abnormality to the pipe bot 100. The pipebot 100 may further determine whether to obtain input information based on the information on whether the feedback is abnormal. For example, the pipe bot 100 may change the movement path when it detects an abnormality of the object, additionally obtain specific input information about the object, and transmit it to the CMS 200, but is not limited to the above-described embodiment.

That is, the mobile unmanned surveillance system may be a system consisting of the pipe bot 100 and the CMS 200, and a detailed operation thereof will be described later.

2 and 3 are diagrams illustrating a method of monitoring an object based on information received from a pipebot.

More specifically, the CMS can receive input data from a pipebot. In this case, the input data may be shape information of the object measured by the HD camera and thermal image information of the object measured by the thermal image camera.

At this time, it may include information on the target to be monitored. For example, the object to be monitored may be a pipe buried underground. Underground pipes can be difficult for people to access, so unmanned pipebots can be moved within the pipes to determine whether the pipes are abnormal.

As another example, it may be used to manage a static VAR compensator (SVC) system. At this time, in a place where a large amount of power is used, such as a power plant or a furnace, the voltage and current of the entire system may be greatly changed. In preparation for this, the SVC system can be equipped, and pipebots can be combined with the SVC system to check for abnormalities. In addition, the present invention can be applied to devices that are difficult to access, such as a turbine for a thermal / nuclear power plant.

That is, objects to be monitored may be various and are not limited to the above-described embodiment. Accordingly, the pipebot may obtain input information differently based on the attribute or characteristic of the object to be monitored. In addition, as an example, the pipebot may be able to acquire or confirm additional input information based on an attribute or feature of a target to be monitored, and is not limited to the above-described embodiment.

Meanwhile, the CMS may perform machine learning using input data based on an object to be monitored. In this case, machine learning may refer to a methodology for learning a computer using data. For example, the machine learning algorithm may be classified into supervised learning, unsupervised learning, and reinforcement learning. For example, supervised learning may be a method of learning based on data in which information criteria are explicitly determined. In addition, unsupervised learning may be a method of learning through data in a state in which information is not explicitly determined. In addition, reinforcement learning may be a method of learning by feeding back the reward and status information.

In this case, as an example, the CMS may learn the input data received from the pipebot through machine learning, and generate a model of whether the object is abnormal based on the learning. That is, the CMS may generate a model that derives an abnormality using information received from the pipebot, and may update the above-described model by relearning real-time information received from the pipebot. In addition, the CMS may derive an output value of whether the object is abnormal based on the above-described model. In this case, the output value may be derived based on the HD camera and the thermal imaging camera. As an example, a main object and location information thereof to be monitored may be determined through an HD camera. In addition, temperature information about the main object may be obtained through the thermal image sequence obtained from the thermal imager. In this case, the CMS may determine whether there is an abnormality by comparing the model generated on the basis of the above-described machine learning in the image from the camera and the real-time information. Thereafter, the CMS may generate a video summary of whether there is an error. For example, a video summary may be generated by extracting only a case where a condition that is equal to or greater than a reference value among conditions set in the target is generated through an abnormality model based on machine learning. In other words, by comparing the real-time input data with the model of machine learning, it is possible to extract the image of the detected part and provide it as an output value without processing all the images. Can be.

In this case, as an example, another treadmill learning method may be applied according to a target to which the pipebot performs monitoring. For example, if the property or characteristic of the object to be monitored is clear and a reference value based thereon is determined, machine learning may operate based on the above-described supervised learning method. As an example, if the target is a turbine, and whether the abnormality is determined based on a specific temperature value, a model of abnormality is generated through a supervised learning method during machine learning, and the abnormality is detected by comparing real-time input information. Can provide.

As another example, if a comprehensive analysis is required because an attribute or characteristic of a monitoring target is not clear, an unsupervised learning method may be applied. In this case, the CMS may build history information through input information based on machine learning, and derive a model for machine learning using history information. In addition, the CMS may update model information generated through relearning whenever the real-time input information is acquired. In this case, the CMS may detect an abnormality by comparing a difference value between the currently obtained input information and the model information.

That is, the real-time input information received from the pipebot can be used to detect anomalies and can be used for relearning a model generated through machine learning.

For example, when the object is a pipe, the determination on whether an abnormality may be comprehensive. In this case, the pipebot may obtain information about the real image image and the thermal image image as input information. In addition, the pipebot may further use information about pipes around the target area as input information. The CMS may check whether the pipe is abnormal by combining the above-described information. In this case, the model may be generated through a treadmill method based on the unsupervised learning method.

4 is a view schematically showing a mobile unmanned surveillance system.

As mentioned above, the mobile unmanned surveillance system may be composed of a pipebot and a CMS. In this case, the pipebot may acquire a real image camera image (image captured by the HD camera) and a thermal image camera as main input values. As described above, the pipebot may acquire an image image of the object while moving. Thereafter, the pipebot may transmit real-time input values or input data to the CMS, and the CMS may process the input data based on the above-described machine learning method to generate a model of whether there is an error. In this case, the CMS may determine whether the target is abnormal by comparing the generated model with newly input real-time input information. In this case, when the CMS detects the abnormality based on the real-time input information and the model of the abnormality, the CMS may obtain output data. In this case, the output data may be a video summary image as an image acquired based on input data and machine learning. That is, only an image of an area where an abnormality is detected may be extracted and provided as an output value, as described above.

5 is a view showing a specific embodiment of a mobile unmanned surveillance system.

As described above, the mobile unmanned surveillance system may monitor the target. In this case, as an example, the object may be located in an area that is difficult for a person to access. In this case, the inaccessibility may be due to reasons such as underground buried or high temperature, and is not limited to the above-described embodiment.

In addition, whether or not the object is abnormal may be determined according to a predetermined criterion. As an example, it may be confirmed that the target is abnormal when the temperature of the target is greater than or equal to the threshold temperature. As another example, the abnormality of the object may be determined by comprehensive judgment by setting a plurality of criteria. Comprehensive determination may be performed based on machine learning, and for this, the CMS may generate a model of abnormality based on past input information. Thereafter, the CMS may determine whether the object is abnormal by comparing the model with respect to the abnormality and real-time input information. As a specific example, CMS is historical input information, and a pipe embedded in a wide area is a model of abnormality by combining local characteristics of a pipe, a time when a pipe bot performs sensing, history information, surrounding area information, and image information. Can be generated. In this case, when the CMS receives the real-time input information from the pipebot, it may be determined whether or not the abnormality by comparing the real-time input information and the model for the abnormality.

In more detail, as an example, whether or not an abnormality may be sufficiently determined using only the real image image and the thermal image information of the first region as the specific region of the pipe. That is, the CMS may generate a model for abnormality based on only the real image and the thermal image as past input information and compare the result with real-time input information to determine whether the abnormality exists. On the other hand, it may not be enough to determine whether the second region as the specific region of the pipe is abnormal only with the above-described real image and thermal image. For example, if water flows around a pipe (local feature), and the pipebot performs sensing in winter (performed at the time of execution), and there is an internal problem, the real image and the thermal image alone are used to derive the abnormality. It can be difficult. Accordingly, the CMS may comprehensively analyze image information, surrounding area information, and history information at the same point as past input information, generate a model for abnormality, and compare the result with real-time input information to determine whether the abnormality is present. That is, different model information may be set based on an object or region monitored by the pipebot, and thus, efficiency of abnormality may be increased.

In addition, the CMS may output information on whether the abnormality detected through the model of machine learning as summary information.

As another example, the mobile unmanned surveillance system may operate differently according to a monitoring target. As mentioned above, the pipe may be relatively large and continuous in the area to be monitored. On the other hand, an area or object to be monitored is specified, and monitoring of the object may be performed periodically through patrol. That is, it may be different whether the pipe bot is continuously sensing the input while receiving the input value or receiving the input value only at a specific point and time. In this case, as an example, when the pipebot continuously performs sensing on the input and transmits the input value to the CMS, the priority of the surrounding information may be set to be high. In this case, the video summary information may be extracted only with information on a point where an abnormality occurs based on the surrounding information.

On the other hand, if the target to be monitored is determined and the pipebot does not continuously sense the input, the priority may be set as the main information for determining whether the history information of the target is abnormal. That is, the mobile unmanned mobile system may operate differently according to the target to be monitored, and is not limited to the above-described embodiment.

5 may be a specific example as an example. For example, the pipebot may patrol the area where the pipe pipe is installed at a constant speed (or a constant cycle). At this time, the pipe is installed throughout the area, the pipe bot needs to continuously sense the input value during the movement. That is, Pipebot can continuously acquire an image using a real image camera and a thermal image camera, and transmit the image to the CMS. In addition, other inputs can be sent to the CMS. In this case, the CMS may generate a model for abnormality by performing machine learning based on the input information received from the pipebot, and determine the abnormality by comparing the information input in real time with the model for the abnormality. have. Thereafter, the CMS may acquire an output value when it detects an abnormality. For example, since there is no explicitly defined reference value for the pipe, data may be processed through a machine learning method based on unsupervised learning. At this time, whether the abnormality in the processed data may be determined by comparing with the surrounding pipe data based on the movement. In addition, the above-described video summary may be a summary image of a certain area determined to be abnormal, as described above.

6 and 7 are flowcharts showing the operation of the mobile unmanned surveillance system.

6 and 7, in the mobile unmanned surveillance system, the pipebot may obtain first input information about the first object. (S710) In this case, as described above with reference to FIGS. 1 to 4, the first The first input information about the object may be the above-described real image image and thermal image image. In this case, as an example, the first input information may be the above-described past input information. Also, as an example, the CMS may generate a training data set using the first input information. That is, the CMS may generate a training data set to perform training on a model in a machine learning manner using past input information acquired through pipebots. Also, as an example, the first input information may include location information about the first object. In addition, the first input information may include time information indicating a time for obtaining information about the first object. In addition, the first input information may be audio information obtained through a microphone. In addition, the first input information may be voltage information. In addition, the first input information may be current information. That is, the first input information may refer to information measured or sensed with respect to the first object, and is not limited to the above-described embodiment.

For example, the first input information may be determined based on the attribute of the first object. In this case, the attribute of the first object may include at least one of position information and size information of the first object. Also, an attribute of the first object may mean a time point at which input information about the first object is received. In addition, as the attribute of the first object, it may mean specific information such as temperature, voltage, current, etc. of the first object. In addition, as an example, the surrounding environment information about the first object may also be the above-described attribute.

For example, when the first object is a pipe, the property of the first object may be a size, a location of the pipe, information about a surrounding pipe, time information for measuring an image of the pipe, and the like.

In addition, as an example, when the first object is a turbine, the attribute may include temperature information about the first object, voltage and current information about the first object, and the like. That is, the information may be information about a feature of the first object.

Next, the pipebot may transmit first input information about the first object to the CMS (S720). The CMS may then generate a first model based on the first input information. In this case, as described above with reference to FIGS. 1 to 4, the CMS may process the first input information based on the machine learning. In this case, the first model may be a model for determining whether the first object is abnormal. That is, the first input information may constitute the above-described learning data set as past input information. The CMS may train the first model by using a learning data set in a machine learning manner, thereby determining whether the first object is abnormal. As another example, the CMS may determine a method of applying machine learning based on the attributes of the first object. As an example, if the first object is a turbine and the reference value is clear based on the property of the object, for example, when the turbine detects an abnormality when the turbine reaches a certain temperature or more, machine learning may be applied based on a supervised learning method.

On the other hand, if the first object is a pipe, and the pipe is a comprehensive consideration of the attributes of the object, such as when the abnormality is determined as the surrounding environment or comprehensive information, machine learning may be performed based on the unsupervised learning method. More specifically, the CMS may build a training data set based on input information received from the pipebot. In this case, the learning data set may be information obtained by combining the above-described input information. In this case, the CMS may generate the first model through the training data set, and may determine whether the first object deviates from the normal state.

Next, the pipebot may acquire second input information about the first object and transmit it to the CMS (S740). At this time, as described above with reference to FIGS. 1 to 6, the second input information may be real-time input information. Can be. That is, if the first input information means past input information, the second input information may mean real time input information that is currently input. Thereafter, the CMS may determine whether the first object is abnormal by comparing the first model and the second input information (S750). In this case, as described above with reference to FIGS. 1 may be a model generated through machine learning to determine whether the object is abnormal. In this case, the first model may be updated again by learning real-time input information as second input information. Also, by extracting and comparing information corresponding to the second input information among the information based on the first model, it may be determined whether the first object is abnormal. That is, the second input information as the real time information may be information for determining whether the object is abnormal and at the same time, information for updating the model of the object.

Next, the CMS may provide output information including information on the determined abnormality (S760). At this time, as described above with reference to FIGS. 1 to 4, the output information may be a video summary. In more detail, the first input information and the second input information may be obtained as image information based on the HD camera and the thermal image camera. However, if all the image information is provided as output information, it takes a lot of time to determine whether the abnormality, it is necessary to provide the extraction information. To this end, only information indicating whether abnormality is obtained from machine learning may be obtained as output target information, and the information may be provided as a video summary.

Embodiments of the present invention described above may be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

For implementation in hardware, a method according to embodiments of the present invention may include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). It may be implemented by field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, a procedure, or a function that performs the functions or operations described above. The software code may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various known means.

The detailed description of the preferred embodiments of the invention disclosed as described above is provided to enable any person skilled in the art to make and practice the invention. Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention without departing from the spirit and scope of the invention as set forth in the claims below. I can understand that you can. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, while the preferred embodiments of the present specification have been shown and described, the present specification is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present specification claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present specification.

In the present specification, both the object invention and the method invention are described, and the description of both inventions may be supplementarily applied as necessary.

100: Pipebot
110: transceiver of the pipebot
120: Pipebot's Processor
130: pipebot input unit
131: Pipebot HD Camera
132: PipeBot's Thermal Camera
200: CMS
210: transceiver
220: processor
230: machine learning learning unit

Claims (13)

In the operation method of the mobile unmanned surveillance system,
Obtaining, by the pipebot, first input information about the first object;
Transmitting the first input information to a central monitoring system (CMS);
Generating, by the CMS, a first model based on the first input information;
Obtaining, by the pipebot, second input information about the first object, and transmitting the second input information to the CMS;
Determining, by the CMS, whether the first object is abnormal by comparing the first model with the second input information; And
Providing output information including information on the determined abnormality;
The first input information and the second input information for the first object include real image image information and thermal image image information,
The first model is a model generated by learning a training data set based on the first input information by a machine learning method. The method of claim 1,
The first input information and the second input information further comprises at least one or more of position information, time information, audio information, voltage information and current information. The method of claim 2,
And the first input information and the second input information are determined based on attributes of the first object. The method of claim 3, wherein
The attribute of the first object is determined based on at least one of location information, size information, time information, and surrounding environment information of the first object. The method of claim 3, wherein
The method of analyzing the machine learning is determined based on the attributes of the first object. The method of claim 5,
The CMS generates the learning data set through the first input information,
And generating the first model through the generated training data set. The method of claim 6,
The CMS extracts information corresponding to the second input information from the first model,
And comparing the extracted information with the second input information to determine whether there is an abnormality with respect to the first object. The method of claim 7, wherein
The second input information is real time input information,
And the first model is updated through learning every time the second input information is acquired. The method of claim 5,
And when the reference value for the attribute of the first object is preset, the first model is performed based on a supervised learning analysis method of machine learning. The method of claim 1,
And the output information provided is composed only of information on which an abnormality has been detected for the first object based on the second input information and the first model. In the mobile unmanned surveillance system,
Pipebots; And
CMS;
The pipebot obtains first input information about a first object,
Transmitting the first input information to the central monitoring system (CMS),
The CMS generates a first model based on the first input information,
The pipebot obtains second input information about the first object, sends the second input information to the CMS,
The CMS compares the first model with the second input information to determine whether the first object is abnormal; and
Providing output information including information on the determined abnormality,
The first input information and the second input information for the first object include real image image information and thermal image image information,
And the first model is a model generated by learning a training data set based on the first input information in a machine learning manner. The method of claim 11,
The pipe bot,
Transmitting and receiving unit for transmitting and receiving a signal;
An input unit for obtaining input information; And
Includes; Processor for controlling the transceiver and the input unit,
And the input unit includes a high definition (HD) camera and a thermal imaging camera. The method of claim 11,
The CMS,
Transmitting and receiving unit for transmitting and receiving a signal;
A machine learning learner applying machine learning to the input data; And
And a processor for controlling the transceiver and the machine learning unit.

Images